3.1 Warm the image cache

Time: 30 seconds now, ~4 min in the background Creates: DaemonSet vllm-image-prepull on every GPU node

The vLLM image is about 9 GB compressed. Pulling it is the single longest step in a GPU cold start (you'll measure that in Lab 2). Image warmers — a DaemonSet whose only job is to make kubelet pull an image — are a standard production technique, and we're going to use one now so Lab 2 starts fast.

modules/03-sharing/prepull-vllm-image.yaml

# Pre-pulls the vLLM image onto every GPU node so Lab 2 doesn't start with a
# 9 GB download. This is a real production technique (image warmers), not a
# workshop hack. Apply it at the START of Module 3 and forget about it.
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: vllm-image-prepull
  labels:
    app: vllm-image-prepull
spec:
  selector:
    matchLabels:
      app: vllm-image-prepull
  template:
    metadata:
      labels:
        app: vllm-image-prepull
    spec:
      nodeSelector:
        karpenter.sh/nodepool: gpu
      tolerations:
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule
      # The init container's only job is to make kubelet pull the image.
      initContainers:
        - name: pull
          image: vllm/vllm-openai:v0.28.0
          command: ["true"]
          resources:
            requests:
              cpu: 10m
              memory: 16Mi
      # A pause container keeps the pod (and therefore the pulled image's
      # "in use" status) alive with near-zero cost.
      containers:
        - name: pause
          image: registry.k8s.io/pause:3.10
          resources:
            requests:
              cpu: 5m
              memory: 8Mi
            limits:
              memory: 32Mi

The init container references the vLLM image and runs true. Kubelet has to pull the image to run it. The main container is pause — 8 MiB of memory to keep the pod alive so the image stays "in use" and is never garbage-collected. No GPU is requested, so it doesn't consume the resource it's warming.

kubectl apply -f modules/03-sharing/prepull-vllm-image.yaml
kubectl get ds vllm-image-prepull

daemonset.apps/vllm-image-prepull created
NAME                 DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                AGE
vllm-image-prepull   1         1         0       1            0           karpenter.sh/nodepool=gpu    2s

READY 0 for the next few minutes while it pulls. Don't wait for it. Go back to the talk; check on it in 3.5.

Why a DaemonSet and not a bigger disk / faster disk / a registry cache?

All of those help and Lab 2 covers them. The DaemonSet is the one that gives you a warm node before the first real pod lands, which is the difference between a 30-second and a 6-minute scale-out.

Back to the talk → · Next: 3.2 Time-slice a node →