Karpenter Spot vs On Demand Cost Comparison: Real-World Data from 2026

Stop 3AM Pages

Free K8s Audit

Let me tell you a story. In early 2025, I was staring at an AWS bill for a Kubernetes cluster running 120 nodes. The number was absurd. I blamed Karpenter. Turned out Karpenter was doing exactly what I told it to. The problem wasn't the tool. It was my assumptions about karpenter spot vs on demand cost comparison.

Most engineers think spot instances are always cheaper. They're not. Not when you factor in interruptions, consolidation behavior, and the real cost of rebalancing. I learned that the hard way — and I'm about to save you that lesson.

This guide breaks down exactly when spot saves you money, when on-demand is worth the premium, and how Karpenter's consolidation engine makes or breaks your cost strategy. You'll see real numbers, a working configuration, and the trade-offs no blog post tells you. By the end, you'll know how to set your own spot-to-on-demand ratio without guessing.

Why Most People Get Spot Wrong

The conventional wisdom: spot instances are 60–90% cheaper than on-demand. So run everything on spot. Then wonder why your batch jobs keep dying.

Here's what the benchmarketing doesn't show: spot interruption rates vary wildly by instance family. A c5.large in us-east-1a could last weeks. A p4d.24xlarge for ML training? It might get reclaimed in 15 minutes. And Karpenter's consolidation — the feature that replaces running nodes with cheaper ones — can trigger interruptions by design. Understanding Karpenter Consolidation: Detailed Overview explains the mechanics, but the TL;DR is that consolidation optimizes for cost and will shift workloads from spot to spot (or spot to on-demand) without asking politely.

The real question isn't "cheaper or not?" It's "what's the total cost of running a workload over a week?" That includes re-queuing time, data re-processing, and the occasional 2 AM pager.

How Karpenter Actually Picks Between Spot and On Demand

Karpenter doesn't have a "spot only" or "on demand only" global switch. It evaluates every Pod's requirements and picks the cheapest instance type from its provisioning template (Provisioner) that meets constraints. The decision depends on:

capacityType: spot or on-demand in the Provisioner
karpenter.sh/capacity-type node selector on Pods
Whether consolidation is enabled

By default, if you specify both spot and on-demand as allowed capacity types, Karpenter will try spot first. If none are available (or if spot price exceeds on-demand), it falls back to on-demand. That's smart — but it doesn't account for workload criticality.

Here's a realistic Provisioner that mixes both:

yaml
apiVersion: karpenter.sh/v1
kind: Provisioner
metadata:
  name: mixed
spec:
  provider:
    instanceProfile: karpenter-node
    subnetSelector:
      karpenter.sh/discovery: my-cluster
    securityGroupSelector:
      karpenter.sh/discovery: my-cluster
  requirements:
    - key: karpenter.k8s.aws/instance-category
      operator: In
      values: [c, m, r]
    - key: karpenter.k8s.aws/instance-generation
      operator: Gt
      values: ["4"]
    - key: karpenter.sh/capacity-type
      operator: In
      values: [spot, on-demand]
  limits:
    resources:
      cpu: 1000
  ttlSecondsAfterEmpty: 30
  consolidation:
    enabled: true

This tells Karpenter: "Use spot or on-demand, whichever is cheaper at the moment. Consolidate aggressively."

But here's the nuance: Karpenter's consolidation logic considers the total cost difference between current nodes and the cheapest alternative. If a spot node costs $0.05/hr and an on-demand node costs $0.08/hr, Karpenter will swap them — possibly evicting your pods. AWS Containers Blog notes that consolidation can reduce costs by 20-40%, but only if your pods can handle being moved.

The Cost Model: Breaking Down Savings

Let's run the numbers for a real workload. Say you have 100 pods, each requesting 1 CPU and 2 GB memory. You're using a mix of c6i.large and m6i.large instances.

Spot pricing (July 2026, us-east-1):

c6i.large: ~$0.034/hr
m6i.large: ~$0.046/hr

On-demand pricing:

c6i.large: $0.085/hr
m6i.large: $0.115/hr

At first glance, spot is 60% cheaper. If you run 10 nodes on spot for a month (720 hours), you pay around $245. On-demand would be $612. Savings: $367 per month.

Now add interruption costs. Assume 5% of spot nodes get reclaimed per week, each taking 2 minutes to spin down and 3 minutes for Karpenter to provision a replacement. During that window, pods may need to restart from scratch. If each restart costs 30 seconds of CPU and network overhead, the total loss is small — maybe $0.50 per interruption.

But what about your batch job that runs for 6 hours and gets killed at hour 5? You lose 5 hours of compute and need to rerun. That's real. In that case, on-demand saves you from that tail risk.

The formula: Effective hourly cost = raw spot cost + (interruption probability × rerun cost / average run time).

For short-lived stateless services (microseconds per request), rerun cost is near zero. Spot wins. For long-running ML training or ETL pipelines, the rerun cost can wipe out spot savings. You have to profile your own workloads.

Spot Interruption Reality in 2026

AWS publishes the EC2 Spot Instance Advisor, but it's historical data. Real-time interruption rates depend on supply/demand in your specific availability zone. In Q1 2026, I saw interruptions spike 3x in us-east-1c when a major gaming company launched a new title.

Karpenter itself can trigger interruptions via consolidation. That caught me off guard. I had set up Pod Disruption Budgets (PDBs) thinking they'd prevent evictions. They don't — not for spot replacements. A Personal Take on Pod Disruption Budgets and Karpenter details exactly how PDBs interact with Karpenter's consolidation. Spoiler: PDBs only protect against voluntary disruptions (like node drains). Spot reclamation is involuntary. Karpenter's consolidation is also considered voluntary but can violate PDBs if it needs to — the blog explains the nuances.

The fix: use topologySpreadConstraints and anti-affinity rules to spread pods across multiple capacity types. And set separate Provisioners for critical vs. non-critical workloads.

Example: a Provisioner for stateful workloads that only uses on-demand:

yaml
apiVersion: karpenter.sh/v1
kind: Provisioner
metadata:
  name: stateful-on-demand
spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: [on-demand]
  ttlSecondsAfterEmpty: 300
  consolidation:
    enabled: false

This stops Karpenter from consolidating those nodes, protecting your databases. Yes, you pay the on-demand premium. But your PostgreSQL won't randomly restart.

On Demand: The Hidden Premium

On-demand isn't always more expensive. When spot markets are tight (like during AWS re:Invent or gaming launches), spot prices can spike to near on-demand levels. Some instance families — t3, t4g — have stable on-demand pricing and burstable capabilities that make spot less attractive for CPU-bound workloads.

Also, Karpenter's consolidation can create on-demand nodes. If consolidation determines that splitting a 4xlarge spot into two 2xlarge on-demand is cheaper (because of granularity), it will do that. I've seen this happen. Your spot-only cluster suddenly has 20% on-demand nodes.

The real premium for on-demand isn't just the price. It's the opportunity cost of not using the savings elsewhere. If you save $500/month by using spot for 80% of your workloads, you can invest that in better observability or faster CI/CD. So on-demand is the safety tax.

Karpenter Cost Allocation Kubernetes: Tracking What You Spend

You can't optimize what you can't measure. Traditional Kubernetes cost tools (Kubecost, OpenCost) work with Karpenter, but they rely on node labels. Karpenter adds useful labels to nodes like:

karpenter.sh/capacity-type
karpenter.sh/provisioner-name
node.kubernetes.io/instance-type

These are gold for cost allocation.

Here's a query to get cost per capacity type using Kubernetes metrics:

bash
# Assuming you have OpenCost or a billing exporter
kubectl cost --namespace my-ns --window 7d --label karpenter.sh/capacity-type

If you're not using a cost tool, you can scrape instance pricing and tag nodes:

yaml
# Karpenter NodeClaim adds labels automatically
# Use these in your monitoring
node.kubernetes.io/instance-type: c6i.large
karpenter.sh/capacity-type: spot

I built a simple script at SIVARO that queries AWS Pricing API and divides costs by the pod's request. It's not perfect, but for a team of 15 engineers, it pointed out that 60% of our cost came from one namespace running heavy ML on spot — with 12% waste from interrupted jobs.

Real Numbers: A/B Test from SIVARO

We ran a 30-day experiment in our production cluster (150 nodes, 3,000 pods). Two Provisioners: one with spot and on-demand allowed, one on-demand only.

Spot-heavy setup (80% spot aim):

Average node cost: $0.052/hr
Total node-hours: 108,000
Billed cost: $5,616
Interruption-induced restarts: 47 (avg recovery 3 min → 141 min downtime cost ~$12)
Effective cost: $5,628

On-demand only:

Average node cost: $0.112/hr
Total node-hours: 108,000
Billed cost: $12,096
Interruption cost: $0
Effective cost: $12,096

Savings: $6,468/month (53%). But — those 47 interruptions were mostly batch jobs. Each caused a 10-minute rerun penalty. If those jobs were latency-sensitive, the user impact would have been worse.

We now run critical services on a separate on-demand-only Provisioner (20% of nodes), and everything else on mixed. The total savings dropped to 42%, but our p99 latency didn't budge.

FAQ

Q: Does Karpenter handle spot interruptions automatically?
Yes. When a spot node gets reclaimed, Karpenter detects the termination via AWS health events and marks the node for deletion. It respects PDBs where possible, but for spot, it can't prevent the interruption — only react. Karpenter docs on concepts explain the lifecycle.

Q: How do I set a max spot percentage per namespace?
Use karpenter.sh/capacity-type: on-demand as a node selector in your Pod spec, or create a separate Provisioner for that namespace with capacityType: on-demand. Karpenter doesn't have a percentage limiter per namespace.

Q: Is consolidation safe for production?
It depends. For stateless workloads, generally yes. For stateful workloads, disable consolidation on their Provisioner. The AWS blog on optimization recommends testing consolidation in staging first — I second that.

Q: What's the best way to do karpenter cost allocation kubernetes?
Use Karpenter's built-in labels and a cost tool like Kubecost. If you're small, export node costs from AWS Cost Explorer and join with Karpenter's NodeClaim events. We use a simple Python script that cross-references Pricing API.

Q: Can spot actually be more expensive than on-demand?
Yes, if you run large instances (e.g., GPU) with high interruption rates and expensive reruns. Tinybird's case study shows they cut costs 20% by aggressively using spot for stateless ingestion but still kept on-demand for stateful caches.

Q: Should I use Karpenter's spotToSpotConsolidation?
It's enabled by default. It can reduce cost but increases pod churn. If your pods can't handle frequent moves, set consolidation.enabled: false on critical Provisioners.

Q: What about is kubernetes used in production?
Absolutely. At SIVARO, 100% of our customer-facing APIs run on Kubernetes with Karpenter. Production means predictable cost and reliability — spot helps with cost, but only when you design for interruptions.

Conclusion

The karpenter spot vs on demand cost comparison isn't a simple number. It's a spectrum. On one end, spot saves 50–60% for workloads that survive interruptions. On the other, on-demand buys stability at a premium. The real answer is a hybrid — separate Provisioners with different consolidation policies, and a clear understanding of your interruption cost per workload.

I learned the hard way: Karpenter is powerful, but it's a tool, not a strategy. You set the rules. If you treat it like a magic cost optimizer, you'll get burned. If you design for it — using the right mix of capacity types, Pod labels, and PDBs — Karpenter can save you real money without killing your uptime.

The next time someone tells you "spot is always cheaper," ask them how many interruptions their pods tolerate. Then show them this guide.

Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.