Kubernetes Node Pools

Every cluster on Hypervisor.io has at least one node pool (the default pool created with the cluster) and can have any number of additional pools added later. This page covers what pools are for, how to add and configure them, how to schedule pods onto specific pools, and how the autoscaler handles them.

Overview #

A node pool is a group of worker nodes inside a Kubernetes cluster that all share:

• The same instance plan - so CPU, RAM, storage, and hourly cost are uniform within the pool.
• The same set of Kubernetes labels - applied to every node at boot.
• The same set of taints - so pods need a matching toleration to land here.
• The same autoscaling bounds (min / max) and rate limits.
• The same drain policy on scale-down or node removal.

What a pool is good for

• Mixed workload shapes. Cache pods want 64 GiB RAM, batch jobs want 32 vCPUs, model serving wants a GPU. One plan can't fit all three; one pool per shape can.
• Taint-based isolation. Keep noisy or expensive workloads off the nodes that run ingress or system pods.
• Per-workload scaling profile. Batch nodes tolerate dense packing and slow reclaim; latency-sensitive nodes want fast reclaim. Per-pool autoscaler tuning lets you do both in one cluster.
• Different fault domains. A pool can target a different hypervisor group for blast-radius isolation.

When to use multiple pools #

A single pool with autoscaling is enough for most clusters. Reach for multiple pools when one of these applies:

• Your workloads need different node sizes (e.g. small for ingress, big for caches, GPU for ML).
• You need to guarantee separation between two workload classes via taints + tolerations.
• You want one pool to scale aggressively for batch bursts while another stays small and steady.
• You want to scale a pool to zero between jobs without impacting the rest of the cluster.

If none of those apply, a single default pool with autoscaling on is the right starting point.

The default pool #

Every cluster has exactly one default pool. It is created automatically when the cluster is created and shows up in the Pools tab marked with a Default badge.

You can:

• Rename it. The default flag stays.
• Edit its plan, size, labels, taints, and policies like any other pool.
• Reassign the default flag by editing another pool and ticking Make default. The previous default becomes a regular pool. There is always exactly one default at a time.

You cannot delete the default pool directly. If you want to remove it, first reassign the default flag to another pool, then delete the old one.

Adding a pool #

1. Open your cluster page.
2. Switch to the Pools tab.
3. Click Add Pool.
4. Fill in the form (fields described below).
5. Click Create.

The new pool starts at its min size. If autoscaling is on, the cluster scales up to min immediately; if autoscaling is off, nothing happens until you scale the pool manually.

Field reference #

Advanced & rate limits #

Visible by expanding the advanced section of the pool form. Defaults are conservative and most clusters never hit them.

The limits exist to avoid two failure modes:

• Stampedes - a sudden burst of Pending pods triggering the autoscaler to ask for fifty nodes at once and overwhelming the hypervisor.
• Capacity flapping - rapid alternating scale-ups and scale-downs that churn billing without doing useful work.

If you have a pool that needs to scale fast (for example, a batch pool that processes a daily queue at 09:00), raise max surge per period and shorten scale period. If you have a pool that needs to be slow and steady (for example, a stateful pool that takes a long time to drain), lower max unavailable per period.

Sending pods to a specific pool #

Use Kubernetes scheduling fields on your pod spec. The pool's labels and taints are what you match against.

Example: a label-only pool

Pool config: label workload=memory, no taint.

Deployment that requests this pool:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
spec:
  replicas: 3
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      nodeSelector:
        workload: memory
      containers:
      - name: redis
        image: redis:7

Pods land only on nodes in this pool. Other workloads can also land here (no taint to keep them out).

Example: a tainted GPU pool

Pool config: label accelerator=gpu, taint nvidia.com/gpu=present:NoSchedule.

Deployment that requests this pool:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: inference
spec:
  replicas: 2
  selector:
    matchLabels:
      app: inference
  template:
    metadata:
      labels:
        app: inference
    spec:
      nodeSelector:
        accelerator: gpu
      tolerations:
      - key: nvidia.com/gpu
        operator: Equal
        value: present
        effect: NoSchedule
      containers:
      - name: server
        image: my-org/inference:latest
        resources:
          limits:
            nvidia.com/gpu: 1

The toleration lets the pod schedule on tainted GPU nodes; the nodeSelector keeps it there. Workloads without the toleration cannot land on GPU nodes, so the GPU pool is reserved for pods that actually want a GPU.

Built-in node labels #

The cluster automatically attaches these labels to every node, in addition to any labels you set on the pool. You can use them as nodeSelector targets without configuring them explicitly.

Scale down behaviour #

When the autoscaler decides a node in a pool is no longer needed, the cluster does the following:

1. Cordon the node so no new pods land on it.
2. Drain the pods according to the pool's drain policy (grace period, ignore-DaemonSets flag, emptyDir flag).
3. If drain succeeds within the drain timeout, destroy the underlying VM.
4. If drain fails or times out, leave the node marked and retry on the next cycle.

The cluster will never drain so many nodes at once that it leaves zero workers. If a scale-down would remove the last remaining worker, that node is exempted until at least one other worker exists.

Deleting a pool #

Schedule deletion

The standard path. Click Delete on the pool in the Pools tab. The pool's nodes are cordoned and drained according to the drain policy, then destroyed. Rate limits apply, so a large pool may take a few cycles to fully drain.

While deletion is in progress the pool stays visible in the Pools tab with a Deleting status. New pods that would have scheduled here go to other pools (assuming their selectors and tolerations match).

Delete now (admin only)

Admins can bypass the drain and rate limits using Delete Now on the admin panel. This destroys all the pool's VMs immediately. Use only when the pool is already broken (for example, every node is stuck in NotReady and a graceful drain will never succeed). Pods running on the pool's nodes are killed without a grace period.

Troubleshooting #