SIVARO
Admission Control in GPU Inference: K8s' Quietest Superpower
URL slug: admit-control-inference-gpu-kubernetes
We spent six weeks in early 2025 building what we thought was the perfect GPU inference platform. We had KServe, we had KEDA, we had all the observability money could buy. Then a single rogue namespace launched 400 pods of a fine-tuned Llama model nobody was using, and our entire production cluster for a fintech client in Singapore went sideways.
The autoscaler did its job. It scaled up. That was the problem.
What we needed wasn't more capacity. We needed to say "no" at the door. That's admission control inference gpu kubernetes — the practice of inspecting, mutating, and rejecting pod requests before the scheduler ever sees them. It's not glamorous. It's not what vendors demo. It's the bouncer at the club who decides who gets in, and in GPU inference, who gets in decides whether your p99 latency stays under 200ms or turns into a distributed apology tour.
In this piece, I'll walk you through what admission control actually means for GPU inference clusters, how it differs from the backpressure and autoscaling mechanisms people confuse it with, and show you the exact YAML and Go code patterns we use at SIVARO to keep GPU clusters honest.
If you've ever watched a shared inference cluster melt down at 2PM on a Tuesday because someone deployed a 70B parameter model to a cluster sized for 7B workloads, this is for you.
What Admission Control Actually Is (And Isn't)
Let's define terms. Admission control inference gpu kubernetes is a Kubernetes mechanism that intercepts API requests — pod creation, pod updates, sometimes even ephemeral containers — and applies policy before the object is persisted to etcd.
It runs as a webhook. Two kinds:
- MutatingAdmissionWebhook — can rewrite a pod spec before it's validated
- ValidatingAdmissionWebhook — can only accept or reject
For GPU inference, you almost always need both. Requesting zero GPU from a node pool that only has A100s? Reject. Requesting 4 GPUs when the namespace quota says 2? Reject. Running a batch job on an interactive inference pool during peak hours? Either reject or mutate the tolerations so it lands on the batch pool.
The critical property: admission control happens synchronously during the API request. Before anything is bound to a node. Before any image is pulled. The scheduler doesn't even see the pod if your webhook rejects it.
People hear "webhook" and think latency. Fine. A typical admission review round-trip costs 5-15ms if your webhook is co-located in the cluster. Compared to a 30-second model load time, that's noise.
But here's what most teams get wrong: they treat admission control as a security feature. It's not — or at least, that's the least interesting thing it does. It's a capacity planning tool. It's the difference between a cluster that fails gracefully and a cluster that fails chaotically.
There's a newer option, by the way: Kubernetes built-in admission policies via ValidatingAdmissionPolicy (CEL-based, no webhook needed). We've been testing this at SIVARO for OPA-style checks. It works for simple rules. For anything involving GPU topology or live cluster state, you still need a webhook.
The Backpressure Confusion
I need to clear something up because it causes real damage at companies building GPU inference platforms.
Admission control inference gpu kubernetes is not backpressure. They're complementary, but they operate at different layers. I see engineers conflate them constantly, and it leads to clusters that either reject too much or queue too much.
Backpressure in GPU inference refers to in-process mechanisms that protect the inference runtime — the model server, the scheduler inside Triton or vLLM, the queue between the HTTP front-end and the GPU kernel. When the model server is processing 12 concurrent requests and you send a 13th, backpressure pushes back. The client retries or the queue fills up.
Admission control operates at the control plane level. It decides whether a pod gets created at all. It doesn't know about request queues. It knows about namespace quotas, node labels, GPU types, and cluster utilization snapshots.
Consider the failure modes:
- No backpressure? A single GPU gets oversubscribed, requests queue internally, latency skyrockets, timeouts cascade.
- No admission control? You get 40 pods trying to schedule across 8 GPUs. The scheduler does its best, but 32 pods sit in Pending state, consuming API server resources, retrying, thrashing etcd. And when the autoscaler kicks in, it does so based on pods pending — adding expensive GPU nodes for requests that might be misconfigured in the first place.
Admission control is the upstream gate. Backpressure is the downstream safety valve. You need both.
A practical pattern we use: the inference front-end (we use LiteLLM or a custom FastAPI gateway depending on the client) tracks in-flight request counts per model. When a model server reports >80% utilization over a 10-second window, the gateway starts returning HTTP 429 for new model loads. Meanwhile the admission webhook rejects new deployments to that pool if the scheduler-reported allocatable GPU is below 20%.
The 429s slow down the rate of new work. The admission controls slow down the rate of new pods. Different levers, same goal: protect the inference pipeline.
Admission Control vs. Autoscaling: They're Fighting Each Other
Here's the contrarian take. Most Kubernetes GPU autoscaling setups I've audited are actively harmful. KEDA and the cluster-autoscaler do one thing well: react. They scale based on demand signals. But demand signals lag. Loading a 13B parameter model takes 20-40 seconds. Autoscaling a GPU node from cold takes 2-5 minutes. The feedback loop is so slow that every spike is either an over-provision (wasteful) or an under-provision (latency SLO breach).
Admission control inference gpu kubernetes can't fix that — it can't make GPUs materialize faster. But it can make autoscaling sane by shaping the demand side.
Admission control vs autoscaling for gpu clusters is a false binary. You need both. But autoscaling must work inside the constraints admission control sets. Otherwise you get the "scale-up spiral of death":
- Fresh team deploys a model with GPU count = 1 per replica.
- Their traffic spikes. Autoscaler adds replicas.
- Those replicas need GPUs. Cluster autoscaler provisions nodes.
- Model's memory footprint is 60GB but they requested 1 GPU (say A10, 24GB). The pod starts, the model crashes, restart loop begins.
- Autoscaler sees crash-looping pods as "pending work" and adds more nodes.
- Your AWS bill quadruples in an afternoon and the model never serves a single successful request.
This actually happened at a client in July of this year — a logistics company in Germany whose platform team didn't have admission control checks for GPU memory vs. model size.
What should have happened: the admission webhook inspects the pod spec, sees a request of nvidia.com/gpu: 1, queries the model registry (we tag images with expected VRAM), finds the model requires ~60GB, and rejects with a clear message: "Model requires 60GB VRAM. Requested 24GB (1x A10). Specify nvidia.com/gpu: 2 or change node selector to A100/H100."
Suddenly autoscaling behaves. The cluster autoscaler only provisions nodes for pods that can actually run. Pending pods with invalid GPU types get rejected at admission time, not scheduled-and-failed.
The Pragmatic Design: What We Run at SIVARO
We've built this pattern into the open-source sivaro-k8s-admission we maintain for our clients. There are three levels of checks, and I recommend you implement them in this order.
Level 1: Static Policy (Cheap, Fast)
Define labels for model class, batch vs. interactive, GPU tier. Mutate the pod's nodeSelector. This is the "don't run a batch job on the interactive money-making pool" check.
yaml
# ConfigMap for our webhook's rule set
apiVersion: v1
kind: ConfigMap
metadata:
name: admission-config
namespace: svr-system
data:
config.yaml: |
policies:
- name: "route-batch-to-batch-pool"
match:
labels:
workload-type: "batch"
mutate:
nodeSelector:
nvidia.com/gpu.pool: "batch"
topology.kubernetes.io/zone: "us-east1-b"
- name: "reject-age-gpu-on-live"
match:
labels:
workload-type: "interactive"
resources:
gpuMemoryGB: ">40"
deny:
message: "Interactive models limited to 40GB. Need more VRAM? Talk to platform team."
This caught our Singapore client's problem. The rogue namespace didn't have workload labels. Deny if labels are missing. Fail closed — not open.
Level 2: Live Cluster State (Necessary)
Static policies can't know if the cluster is 98% full at 9AM on a Monday. You need a webhook that can talk to the Kubernetes API server and check current scheduling conditions.
go
// Simplified Go webhook logic
func (h *admissionHandler) handleCreate(w http.ResponseWriter, req *http.Request) {
review := &admissionv1.AdmissionReview{}
json.NewDecoder(req.Body).Decode(review)
pod := &corev1.Pod{}
json.Unmarshal(review.Request.Object.Raw, pod)
// Extract GPU request count
gpuCount := int64(0)
for _, ctr := range pod.Spec.Containers {
if q, ok := ctr.Resources.Limits["nvidia.com/gpu"]; ok {
gpuCount += q.Value()
}
}
if gpuCount == 0 {
// No GPU requested, allow through
h.allow(review)
return
}
// Check pool capacity — at SIVARO we've seen this story play out
// enough that we've built a sidecar that wraps a cache of node status
poolStatus := h.poolCache.GetPoolStatus("interactive") // A100 pool
// 90% utilization threshold, with 5 min time decay
if poolStatus.Utilization > 0.90 {
h.deny(review, "GPU pool 'interactive' at 92% utilization.
Consider routing to batch pool or reducing replicas.")
return
}
h.mutateNodeSelector(review, "interactive")
h.allow(review)
}
That pool cache is important. I tried our first version querying the API server on every request. Terrible idea — the API server load goes up just as you're trying to reject additional load. Instead, run a controller that syncs node status every 5-10 seconds and serves the webhook from memory. 10ms response times, no extra API pressure.
Level 3: Model-Aware Admission (The Winner)
This is where we started seeing real wins in late 2025. A model registry — we use MLflow but simple metadata sidecars work too — stores required model VRAM. When a pod requests a model, the webhook cross-checks pod GPU request against the model registry:
bash
# Example CLI on our webhook
kubectl apply -f - <<EOF
apiVersion: svr.sivaro.io/v1alpha1
kind: InferenceDeployment
metadata:
name: llama-70b-serving
spec:
modelName: "meta/llama-3-70b"
container:
image: my-registry/llama-server:70b-v4
resources:
requests:
nvidia.com/gpu: "2" # L4 or A10
---
EOF
The webhook says "no". The model metadata says 70B requires 40GB of contiguous VRAM. 2x A10 doesn't cut it. It mutates the replica count to 0 and sends a Kubernetes Event with the reason. Autoscaling never even sees it. You just saved $15/hour in wasted GPU + hours of crash-loop debugging.
The Admission Control vs. Backpressure Sequel
One architectural pattern I've needed to correct across several teams: serving queue depth and admission control radius must be tuned together. This isn't a new problem — concurrency limits in Knative Serving have done this for CPU services for years. GPU inference carries heavier weights because:
- A single bad deployment can strand an entire GPU (memory fragmentation, severe latency spikes)
- What's pending at the cluster level is often a symptom of an internal queue issue
Companies in 2026 have deployed $10M+ GPU fleets with only autoscalers and no admission policies. They get caught in a trap: the cluster is at 90% utilization by error — a service that requests 4 GPUs per pod but runs at 20% utilization because of a poorly designed batching loop.
Our eventual solution for the fintech client was: don't autoscale until utilization is an actual product decision. Combine request-level throttling at the gateway (backpressure), admission rejection at the control plane (static + current-pool checks), and autoscaling as a last resort. You might think you want to scale up to meet demand, but first check if admission control should have rejected the demand in the first place.
That's not a warm feeling for platform engineering teams. "No" sounds like you're blocking innovation. But I've seen better outcomes with a deliberate, explicit rejection policy than organic chaos.
What Goes Wrong (And How to Debug It)
An admission webhook failure should — by default — be a deadbolt, not a sieve.
Set failurePolicy: Fail for GPU-related checks. I know "fail closed" hurts during webhook downtime. But a fail-open policy on GPU means you're not doing admission control at all. If the policy is safety-critical — which it is when a single deploy can cost $4K/day in idle GPU — failure should be loud.
You must watch webhook timeout trends
Default webhook timeout is 10 seconds. Set it to 2 seconds for user-facing checks, 5 for policy checks. If the webhook times out on resource-intensive validation, pods pass through unvetted. We run a Prometheus query on webhook_duration_seconds and alert over 500ms p99 — beyond that, the admission path is interfering with deploy speed.
Be careful with the matchPolicy
Use Equivalent sparingly. It causes your webhook to trigger for all resources that share the same group/version/kind. In GPU clusters with lots of custom resources (NVIDIA's DevicePlugin, knative custom services), that results in unexpected admission calls. Default to Exact unless you have a specific case.
The Full Picture: An Admission Control Policy for a Real Cluster
Let me show you a working baseline that we template for clients using H100 clusters for both training and inference.
yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: gpu-admission-config
data:
checks: |
# For pods in gpu-training namespace
apiVersion: v1
kind: Policy
favorites:
- matchExpressions:
- {key: "type", operator: In, values: ["training"]}
nodeSelector:
nvidia.com/gpu.product: "H100-SXM-80GB"
# validate nodeSelector is not tainted
requiredTaints:
- key: "dedicated"
operator: "Equal"
value: "gpu"
Not impressive, is it? That's the point. The best admission policy I write for a client barely mutates a pod — it makes sure the pod runs where it should, for the resource it should, and doesn't hit the cluster's shared control plane during spikes.
Admission control is a capacity lever. Set it too tight and you waste money (rejecting pods that would have gently shared a pool). Set it too loose, and you're not doing admission at all.
The 5-Step Adoption Roadmap
If you're running GPU inference on Kubernetes and don't have any admission policies, here's the path.
- A week only monitoring, not blocking. Deploy a validating webhook that logs what it would reject. We use
dryRun: trueand send results to a Slack audit channel. - Add namespace and quota checks. Kubernetes
ResourceQuotais admission control-lite, but you're at the height of GPU bursts. Enforce requests and limits. - Build the cluster utilization cache. The key insight from the second half of 2025. Cache the node state in memory, and reject pods if the GPU pool is 95% scheduled over the last 2 minutes.
- Model-aware checks. Spend the time to create a model registry with VRAM metadata. Manually curate a list of critical model quantities. This is where the real ROI is in prevention.
- Wire it into your deployment UI. Your ML engineers don't get errors from a raw webhook — makes no sense. Instead, surface admission errors in their standard deploy path with actionable remediation.
The Bottom Line: It's Not Autoscaling, It's Admission
Admission control inference gpu kubernetes is the Kubernetes service that says "no" in a world built around "yes." In a GPU inference landscape where each H100 costs $4 an hour whether it does useful work or not, the ability to stop bad work from scheduling becomes a financial metric, not just a rollback strategy.
Admission control vs backpressure in GPU serving: they're complementary. Admission control protects the cluster, backpressure protects the runtime, and the two together protect your wallet.
Admission control vs autoscaling for GPU clusters: the former is a guard, the latter an expander — you need to know which mode you're in. If you don't have the guard, autoscaling expands into chaos.
I started 2025 thinking admission control was a policy compliance feature. After the fintech meltdown in March — the 400-pod rogue namespace that cost us a week of trust — I realize it's a system reliability lever. CPUs were shortlived. GPUs are the new financial risk.
We deploy admission control to every GPU cluster we build now. If you're doing GPU serving without it, we need to talk.
FAQ: Admission Control on GPU Clusters
What is admission control inference gpu kubernetes?
Admission control inference gpu kubernetes refers to intercepting pod creation, update, or delete requests with a webhook or built-in ValidatingAdmissionPolicy to enforce policies before scheduling. In GPU inference, that includes constraints on GPU type, GPU count, model VRAM, node pool selection, and namespace quotas. It prevents invalid or harmful GPU workloads from ever entering the scheduling path.
How is admission control different from request-level backpressure in GPU serving?
Backpressure operates at the inference runtime (Triton, vLLM) to limit simultaneous requests within a single model server. Admission control is a control-plane gate before a pod is created. Backpressure protects a running process — admission control protects the cluster, quota, and the operator's bill. They are sequential layers.
How does admission control interact with autoscaling on GPU clusters?
Admission control can prevent the autoscaler from making destructive decisions, since pending pods that will never successfully run are rejected at the API server. It ensures the cluster autoscaler only provisions nodes for work that can actually execute. Autoscaling scales up demand — admission control regulates what demand is allowed to exist.
Do I need a mutating webhook, or is validating enough?
You cannot adjust a pod's GPU request or nodeSelector with validation alone. To route batches to the right pool, correct a wrong GPU request, or add taints/tolerations, you need a mutating webhook. Use a validating webhook for strict rejections. For GPU workloads with strict pool constraints, plan to run both.
What is the latency cost of admission control?
Our admission webhook runs at 10-15ms p99 for core checks. We keep a in-memory node pool cache, so there is no round-trip to the API server or etcd. Compared to a model load time of 20-60s, that's an irrelevant cost.
What happens if my webhook is down?
With failurePolicy: Fail, a webhook outage breaks new deployments until the service recovers. That is intentional. In GPU clusters with strict quotas and billing implications, fail-open is not a safe default. Set up high-availability replica for your webhook and alert on failure.
Should I write admission policies in CEL or Go?
For simple rules — namespace/GPU type constraints — use ValidatingAdmissionPolicy with CEL. You don't have to maintain a webhook. For anything involving a model registry lookup, dynamic cluster state, or node pool capacity checks, use a Go webhook. The former is a shortcut, the latter is the real tool.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.