Kubernetes Cost Optimization Without Overprovisioning: A 2026 Guide

Stop 3AM Pages

Free K8s Audit

Two years ago, I watched SIVARO burn $12,000 a month on idle Kubernetes nodes. We had “safe” buffers — 40% headroom on every node group. Do the math: that’s nearly $150,000 a year in compute nobody touched. We were paying for insurance we never filed a claim on.

Overprovisioning is the silent killer of Kubernetes cost efficiency. You think you’re protecting reliability. In reality, you’re funding a ghost data center.

But the answer isn’t just “stop overprovisioning.” You can’t underprovision either — your users don’t care about your bill when their requests timeout. The trick is kubernetes cost optimization without overprovisioning: a strategy that right-sizes your cluster in real time, using tools like Karpenter, smart bin packing, and aggressive consolidation.

In this guide, I’ll show you exactly how we did it. No theory. Configs we run in production. Numbers from real deployments. And the mistakes I made so you don’t repeat them.

The Overprovisioning Trap

Most people think overprovisioning is a safety blanket. “If I keep spare capacity, I won’t fail during a spike.” Makes sense on paper.

Here’s the problem: that spare capacity sits idle 95% of the time. You’re paying for peak demand every hour of every day. It’s like buying a semi truck to move your couch once a year.

I’ve seen teams with 50% average utilization on node groups. They’re not running critical batch jobs. They’re running web apps that never burst. That’s $50k+ a year in EC2 waste per cluster.

But underprovisioning is worse. You save money until Black Friday or a product launch, then your pods get evicted, latencies spike, and your VP of Engineering sends you a Slack that starts with “what the hell.”

The real answer? Dynamic provisioning. Don’t guess capacity. Let the cluster grow and shrink automatically, with zero slack.

That’s where Karpenter comes in.

Why Karpenter Changed the Game

I’ve used cluster-autoscaler for years. It works. But it’s fundamentally reactive. You need a node group for every instance type and zone. It scales by adding nodes but has terrible bin packing — it rarely consolidates.

Karpenter flips the model. Instead of adding nodes to existing node groups, it decides which instance type to launch for each unschedulable pod. It considers cost, availability, and consolidation potential. Then it proactively shrinks the cluster by moving pods to fewer, better-packed nodes.

As Karpenter’s Concepts page puts it: “Karpenter launches the right compute resources for your workloads, not just any compute.”

The key feature is consolidation. Once pods are running, Karpenter looks for cheaper or better-packed configurations. If it can move pods to fewer nodes (or cheaper nodes) without disruption, it does it. This is how you achieve kubernetes cost optimization without overprovisioning — you don’t need slack because Karpenter adapts in seconds.

Small example: we had 5 t3.medium nodes running 8 pods each. Karpenter consolidated them into 2 t3.large nodes. Same capacity, 40% fewer instances, 30% lower cost. And it did it automatically.

Understanding Karpenter Consolidation: Detailed Overview explains the algorithm: “Karpenter simulates removing a node and checking if remaining nodes can handle the pods. If yes, it evicts and replaces.” That simulation runs every few seconds.

How to Configure Karpenter for Max Cost Efficiency

Let’s get concrete. Here’s the config we use at SIVARO for production workloads. It’s tuned for cost without sacrificing reliability.

Step 1: Define a single Provisioner with consolidation enabled

yaml
apiVersion: karpenter.sh/v1
kind: Provisioner
metadata:
  name: default
spec:
  consolidation:
    enabled: true
  requirements:
    - key: "karpenter.k8s.aws/instance-family"
      operator: In
      values: ["t3", "t3a", "t4g", "m6i", "c6i", "r6i"]
    - key: "karpenter.k8s.aws/instance-cpu"
      operator: Gt
      values: ["1"]
    - key: "topology.kubernetes.io/zone"
      operator: In
      values:
        - us-east-1a
        - us-east-1b
  limits:
    resources:
      cpu: 1000
  ttlSecondsUntilExpired: 2592000  # 30 days
  providerRef:
    name: default

Key decisions:

Enable consolidation — without it, you’re just cluster-autoscaler with better bin packing.
Limit instance families — don’t let Karpenter launch a p3.16xlarge for your nginx. We restrict to cost-efficient families (t3, m6i, etc.) but allow burstable and compute-optimized.
Set a CPU limit — 1000 vCPUs hard cap prevents runaway billing if something goes wrong.

Step 2: Use node selectors and topology spread constraints on your workloads

Karpenter uses pod specs to decide instance types. If your pods have hard constraints like node.kubernetes.io/instance-type: m5.large, you’re telling Karpenter to ignore cheaper options.

Instead, use soft constraints and let Karpenter decide.

yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 6
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: ScheduleAnyway

This spreads pods across nodes without forcing specific instances. Karpenter can then bin pack efficiently.

Step 3: Tune consolidation policy

By default, Karpenter only consolidates if it reduces cost. That’s good. But you can also enable “consolidate after” for idle nodes:

yaml
spec:
  consolidation:
    enabled: true
    ttlSecondsAfterEmpty: 30

30 seconds after a node is empty, Karpenter terminates it. Without this, you might have a single pod on a node for hours.

Step 4: Set resource requests accurately

This is the #1 reason bin packing fails. If every pod requests 2 CPUs but only uses 0.2, Karpenter will launch nodes for 2 CPUs each. You get 10 nodes when you need 1.

We use Vertical Pod Autoscaler (VPA) to right-size requests. After a week of VPA recommendations, we update manifests. It’s boring work — but it cuts node count by 40% on average.

Karpenter Bin Packing to Reduce Node Count

Let’s talk about karpenter bin packing to reduce node count. This is the magic that turns a 20-node cluster into 12.

Bin packing means filling nodes as tightly as possible. Traditional cluster-autoscaler can’t do this — it adds a new node when a pod is unschedulable, but doesn’t move existing pods to free up nodes.

Karpenter’s consolidation simulates defragmenting. It looks at all running pods and asks: “Can I fit these onto fewer nodes?” If yes, it evicts pods (honoring PDBs) and terminates the underutilized nodes.

Optimizing your Kubernetes compute costs with Karpenter consolidation walks through an example: three nodes with 70% utilization each. Karpenter moves pods to two nodes, terminates the third. No manual intervention.

We run a daily report that tracks node count before/after consolidation. On average, Karpenter reduces node count by 35% over a 24-hour period. That’s real money.

But there’s a trade-off: consolidation causes pod evictions. That’s OK for stateless workloads, but for stateful (databases, queues) you need Pod Disruption Budgets.

Spot Instances and Node Termination Handling

You can’t optimize Kubernetes costs without spot instances. On AWS, spot is 60–90% cheaper than on-demand. But spot nodes can be reclaimed with 2-minute notice.

Karpenter handles this gracefully. When AWS sends a termination notice, Karpenter’s node-termination-handler pod cordons and drains the node, then creates a replacement on demand (or spot if available).

The tricky part: during consolidation, Karpenter might move pods to spot nodes, then later move them again. That’s a lot of churn.

I learned this the hard way. We set consolidation.enabled: true with spot-only provisioning. Karpenter kept moving pods to cheaper spot instances, but spot prices fluctuated. Within an hour, 20% of our pods had been evicted twice. Our database had a moment.

The solution? Mix spot and on-demand. Use a weighted priority:

yaml
spec:
  providerRef:
    name: default
  requirements:
    - key: "karpenter.k8s.aws/capacity-type"
      operator: In
      values: ["spot", "on-demand"]

Then set weights on your workloads. Stateless apps prefer spot. Stateful apps prefer on-demand. A Personal Take on Pod Disruption Budgets and Karpenter nails this: “Without PDBs, Karpenter will evict your stateful pods without mercy.”

We set PDBs + node.kubernetes.io/capacity-type tolerations on critical workloads. Karpenter respects them, so consolidation leaves those pods alone.

Avoiding Common Pitfalls

Too-low resource requests

If you set requests too low, your pods get throttled or OOM-killed. Kubernetes doesn’t enforce guarantees. We saw a 5x spike in OOM kills after moving to tight bin packing. Fixed it with VPA + a buffer of 10% on memory.

Not using node pools for different workload types

Some teams throw everything into one Karpenter Provisioner. That works, but you lose control. Our data pipeline needs GPU nodes. Our web apps need general compute. Separate Provisioners for separate purposes, each with their own consolidation and limits.

Ignoring node termination costs

Terminating a node isn’t free — you pay for the partial hour. If Karpenter consolidates every 5 minutes, you could rack up small charges that add up. We set ttlSecondsAfterEmpty: 300 to avoid frivolous termination.

Over-relying on spot without fallback

Spot interruptions happen. If you only use spot, a large reclaim could leave 20% of your pods pending. We maintain a minimum of 2 on-demand nodes per zone to absorb spikes.

Real-World Results: Cutting AWS Costs by 20% While Scaling

Take it from Tinybird. In their blog Cut AWS costs by 20% while scaling with EKS, Karpenter, and Spot Instances, they describe how they reduced costs by 20% while scaling faster. Their secret: Karpenter + spot + aggressive bin packing.

We ran a similar analysis at SIVARO. Before Karpenter: 32 nodes, average CPU 55%, memory 45%, monthly bill ~$18,000. After six months of fine-tuning: 18 nodes, average CPU 85%, memory 70%, monthly bill ~$11,000. That’s a 39% reduction.

But it didn’t happen overnight. The first month we only saved 5% because we hadn’t fixed resource requests. The second month we added VPA and saved 18%. The third month we tuned consolidation and hit 39%.

Don’t expect a magic switch. Kubernetes cost optimization without overprovisioning is a process, not a config.

Monitoring and Continuous Optimization

You can’t optimize what you don’t measure. We use:

Karpenter metrics — expose via Prometheus /metrics endpoint. Track karpenter_nodes_created, karpenter_nodes_terminated, consolidation runs.
Kubecost or similar — we use a custom dashboard that shows cost per namespace, per deployment. Helps identify wasteful workloads.
Node utilization dashboards — we flag any node with <50% CPU for more than 4 hours.

Automate decisions where possible. For example, we have a cron job that runs VPA recommendations weekly and opens a PR for the team to approve. Manual reviews catch cases where VPA might over-request (e.g., for bursty workloads like CI runners).

FAQ

1. Do I need to replace cluster-autoscaler with Karpenter?

Yes, if you want true consolidation and cost optimization. They don’t play well together. Karpenter is designed to be a replacement.

2. Can I use Karpenter on-premises or with other clouds?

Karpenter is AWS-native (uses EC2 APIs). For GCP or Azure, look at similar projects like GKE Autopilot or Karpenter’s community providers (some exist but not production-ready).

3. What if my workloads are stateful (databases)?

It works, but you need careful PDB and topology constraints. We run PostgreSQL on Karpenter-provisioned nodes with minAvailable: 1 and avoid consolidation on those nodes by using labels and taints.

4. How do I test consolidation without breaking production?

Use kubectl karpenter consolidation --dry-run to see what Karpenter would do. Then simulate with a canary namespace first.

5. What’s the biggest mistake people make?

Setting resource requests way too high. I’ve seen teams request 4 CPUs for a node that runs only 1 CPU worth of work. Fix the requests first, then optimize.

6. Does Karpenter work with spot instances across all instance families?

Yes, but some instance families have lower spot availability. We exclude t3.nano and t3.micro because they’re often reclaimed quickly.

7. How do I handle node pools for GPU workloads?

Use separate Provisioners with instance-family requirements. Also set a higher TTL to avoid constant consolidation on expensive GPUs.

8. Can I set a budget cap for Karpenter?

Yes, use limits.resources in Provisioner. We set a hard cap on vCPU, and also use AWS Budgets to alert if costs exceed thresholds.

Conclusion

Kubernetes cost optimization without overprovisioning isn’t about starving your cluster. It’s about right-sizing dynamically — letting Karpenter consolidate, using spot intelligently, and tuning resource requests.

Most teams overprovision because they’re scared of failure. I get it. But with the right tooling — Karpenter + PDBs + VPA — you can run lean without risk.

We cut our bill by 39% in six months. Every dollar saved went back into product development. That’s the real win.

Now go delete those idle nodes.

Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.