Where to go next

Every technical claim in this guide was checked against the sources below. When something here and something there disagree, the source wins — and please open an issue.

Amazon EKS

• EKS-optimised accelerated AMIs (AL2023 NVIDIA) — what's in the AMI, what isn't
• NVIDIA device plugin vs. DRA on EKS — AWS's current recommendation by capacity model
• MIG on EKS
• Retrieve recommended AMI IDs via SSM
• EKS Auto Mode: accelerated instances
• EKS best practices: AI/ML compute
• EC2 instance quotas (vCPU-based) · Spot limits
• Accelerated instance types reference · G5 · G6 · G6e
• Amazon EKS Workshop — the AI/ML chapter has a vLLM chatbot lab that picks up where this one stops

Karpenter

• NodePools · EC2NodeClasses — the amiSelectorTerms alias syntax and pinning
• Managing AMIs — the @latest warning
• Disruption — consolidation policies, budgets, interruption handling
• Scheduling: accelerators — why the device plugin must exist before the node
• Getting started · Compatibility matrix

NVIDIA

• GPU Operator: getting started — pre-installed driver / toolkit sections
• GPU Operator: time-slicing — including the caveats we quoted
• GPU Operator: MIG · MIG supported GPUs
• GPU Operator: CDI · Release notes
• Kubernetes device plugin — sharing configs, MPS, GFD labels
• DCGM exporter · GPU telemetry with kube-prometheus · Grafana dashboard 12239
• DCGM profiling metrics — what SM_ACTIVE and PIPE_TENSOR_ACTIVE actually measure

Kubernetes

• Schedule GPUs · Extended resources
• Dynamic Resource Allocation
• Startup probes · emptyDir (memory-backed)
• Horizontal Pod Autoscaling

vLLM, KEDA, k6, Prometheus

• vLLM: OpenAI-compatible server · Kubernetes deployment · Metrics · Troubleshooting
• KEDA: Prometheus scaler · ScaledObject spec · Scaling deployments
• k6 docs
• kube-prometheus-stack chart · PodMonitor / ServiceMonitor

Models

• Qwen/Qwen2.5-1.5B-Instruct — Apache-2.0, ungated. Try Qwen/Qwen3-1.7B next (add --reasoning-parser qwen3 and think about enable_thinking).