Kubernetes Cost Optimization: Rightsizing, Spot Nodes, Bin Packing, Kubecost, and Scale-to-Zero

March 26, 2026 16 min read Virtualization & Containers

kubernetescost-optimizationvpakedakubecostfinopskarpenter

Contents

Kubernetes makes it easy to over-provision. Default resource requests are conservative, autoscalers react slowly, and nobody deletes staging workloads on Friday afternoon. The result: clusters running at 20-30% utilization while the cloud bill climbs. This guide covers the full stack of cost optimization — from understanding what you’re paying for, to the tools that cut waste without breaking production.

Understanding Where the Money Goes

Before optimizing, instrument. Cloud Kubernetes costs break down into three main buckets:

Compute (60-80% of total): node instance costs. This is where most waste lives — over-provisioned nodes, idle nodes left running, expensive on-demand instances where spot would work.

Storage (10-20%): persistent volumes, object storage for logs/metrics, etcd backups. Often overlooked but can surprise you at scale.

Network (5-15%): cross-AZ data transfer (expensive), load balancer hours, NAT gateway traffic. Cross-AZ egress in AWS is ~$0.01/GB — trivial per request, significant at millions of requests.

The Utilization Baseline

Measure before changing anything:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


# Node-level CPU and memory utilization
kubectl top nodes

# Pod-level utilization
kubectl top pods -A --sort-by=cpu | head -30
kubectl top pods -A --sort-by=memory | head -30

# Requests vs actual usage per namespace
kubectl resource-capacity -n production --pods --util

# Using kubectl-view-utilization plugin
kubectl view-utilization -h

# Prometheus query: cluster-wide CPU utilization
# (actual usage / total requested)
sum(rate(container_cpu_usage_seconds_total{container!=""}[5m])) /
sum(kube_pod_container_resource_requests{resource="cpu"})

If this ratio is below 0.4 (40%), you have significant over-provisioning. Most greenfield clusters start at 15-25%.

Rightsizing with Vertical Pod Autoscaler (VPA)

The most impactful single change for most clusters: fix your resource requests. Most engineers set requests once at deployment time and never revisit them. VPA watches actual usage and recommends (or automatically sets) better values.

Install VPA

1
2
3
4
5
6
7
8
9


git clone https://github.com/kubernetes/autoscaler.git
cd autoscaler/vertical-pod-autoscaler
./hack/vpa-up.sh

# Verify
kubectl get pods -n kube-system | grep vpa
# vpa-admission-controller-...  Running
# vpa-recommender-...           Running
# vpa-updater-...               Running

VPA Recommendation Mode (Safe Starting Point)

Start in Off mode — VPA observes and recommends but doesn’t change anything:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-api-vpa
  namespace: production
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-api
  updatePolicy:
    updateMode: "Off"    # Recommend only, don't apply
  resourcePolicy:
    containerPolicies:
      - containerName: web-api
        minAllowed:
          cpu: 50m
          memory: 64Mi
        maxAllowed:
          cpu: 4
          memory: 4Gi
        controlledResources: ["cpu", "memory"]

After a few days, check recommendations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


kubectl get vpa web-api-vpa -n production -o yaml

# Output shows:
# status:
#   recommendation:
#     containerRecommendations:
#     - containerName: web-api
#       lowerBound:
#         cpu: 120m
#         memory: 256Mi
#       target:              ← Use this for requests
#         cpu: 250m
#         memory: 512Mi
#       upperBound:          ← Use this for limits
#         cpu: 800m
#         memory: 1Gi
#       uncappedTarget:
#         cpu: 250m
#         memory: 512Mi

Common finding: containers declared with requests: cpu: 500m, memory: 1Gi actually use cpu: 80m, memory: 200Mi. Fixing this immediately frees cluster capacity.

VPA Auto Mode

Once you trust the recommendations, switch to Auto to let VPA apply changes:

1
2
3
4


spec:
  updatePolicy:
    updateMode: "Auto"      # Evict pods to apply new requests
    # Or "Initial" — only set on new pods, don't evict existing ones

Important caveats:

VPA in Auto mode evicts pods to apply changes — ensure your Deployments have minReadySeconds and enough replicas to tolerate rolling updates
VPA and HPA conflict when both target CPU — use VPA for memory and custom metrics-based HPA for CPU, or use the KEDA approach below
VPA needs at least 8 data points (by default) before making recommendations — give it a week of real traffic

Reading VPA Recommendations Programmatically

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


#!/usr/bin/env python3
"""Extract VPA recommendations and output as a rightsizing report."""
import subprocess, json, sys

def get_vpa_recommendations(namespace="--all-namespaces"):
    result = subprocess.run(
        ["kubectl", "get", "vpa", "-n", namespace, "-o", "json"]
        if namespace != "--all-namespaces"
        else ["kubectl", "get", "vpa", "-A", "-o", "json"],
        capture_output=True, text=True
    )
    return json.loads(result.stdout)

vpas = get_vpa_recommendations()
print(f"{'VPA':<40} {'Container':<20} {'CPU Target':<12} {'Mem Target':<12}")
print("-" * 90)

for item in vpas.get("items", []):
    name = item["metadata"]["name"]
    ns = item["metadata"]["namespace"]
    recs = item.get("status", {}).get("recommendation", {}).get("containerRecommendations", [])
    for rec in recs:
        container = rec["containerName"]
        cpu = rec.get("target", {}).get("cpu", "N/A")
        mem = rec.get("target", {}).get("memory", "N/A")
        print(f"{ns}/{name:<38} {container:<20} {cpu:<12} {mem:<12}")

Spot and Preemptible Nodes

Spot instances (AWS) / preemptible VMs (GCP) / spot VMs (Azure) cost 60-90% less than on-demand. The trade-off: the cloud provider can reclaim them with 2 minutes notice. For the right workloads, this is an excellent deal.

Which Workloads Fit Spot

Good for spot:

Batch jobs, data processing, ML training
Stateless web tier (multiple replicas — losing one is fine)
CI/CD runners
Dev and staging environments
Any workload that can tolerate a restart

Keep on on-demand:

Control plane nodes
Stateful databases (unless using a cloud-managed database)
Single-replica critical services
Jobs with tight deadlines

Karpenter: The Modern Node Provisioner

Karpenter replaces the Cluster Autoscaler for AWS (and increasingly GCP/Azure). It’s smarter about node selection — it picks the optimal instance type for the pending pods rather than just scaling a fixed node group.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


# Install Karpenter (AWS)
helm repo add karpenter https://charts.karpenter.sh/
helm repo update

helm upgrade --install karpenter karpenter/karpenter \
  --namespace karpenter --create-namespace \
  --set settings.clusterName=$CLUSTER_NAME \
  --set settings.interruptionQueue=$KARPENTER_SQS_QUEUE \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59


# NodePool: define what nodes Karpenter can provision
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: general-purpose
spec:
  template:
    metadata:
      labels:
        nodepool: general-purpose
    spec:
      nodeClassRef:
        apiVersion: karpenter.k8s.aws/v1beta1
        kind: EC2NodeClass
        name: default
      requirements:
        # Allow many instance families for better spot availability
        - key: karpenter.k8s.aws/instance-family
          operator: In
          values: [m5, m5a, m5n, m6i, m6a, m7i, m7a, c5, c6i, c7i]
        - key: karpenter.k8s.aws/instance-size
          operator: In
          values: [large, xlarge, 2xlarge]
        # Prefer spot, fall back to on-demand
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: [amd64, arm64]    # Allow Graviton for better price/perf
  disruption:
    consolidationPolicy: WhenUnderutilized   # Remove underused nodes
    consolidateAfter: 30s
  limits:
    cpu: 1000                 # Cluster-wide limit on this pool
    memory: 4000Gi

---
apiVersion: karpenter.k8s.aws/v1beta1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiFamily: AL2
  role: KarpenterNodeRole-$CLUSTER_NAME
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: $CLUSTER_NAME
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: $CLUSTER_NAME
  blockDeviceMappings:
    - deviceName: /dev/xvda
      ebs:
        volumeSize: 100Gi
        volumeType: gp3
        iops: 3000
        throughput: 125
        encrypted: true

Tolerations for Spot Nodes

Mark spot nodes with a taint and add tolerations only to workloads that can handle interruption:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


# Karpenter automatically taints spot nodes:
# karpenter.sh/capacity-type=spot:NoSchedule

# Workloads that tolerate spot
spec:
  template:
    spec:
      tolerations:
        - key: karpenter.sh/capacity-type
          operator: Equal
          value: spot
          effect: NoSchedule
      # Also use affinity to prefer spot but allow on-demand
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              preference:
                matchExpressions:
                  - key: karpenter.sh/capacity-type
                    operator: In
                    values: [spot]

Handling Spot Interruptions Gracefully

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


# PodDisruptionBudget — ensure at least N pods stay running during interruption
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web-api-pdb
  namespace: production
spec:
  minAvailable: 2      # Or use maxUnavailable: 1
  selector:
    matchLabels:
      app: web-api

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


# Graceful shutdown — give pods time to drain connections
spec:
  template:
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: web-api
          lifecycle:
            preStop:
              exec:
                # Signal the app to stop accepting new requests
                command: ["/bin/sh", "-c", "sleep 5"]

Karpenter watches for spot interruption notices (via SQS) and drains nodes proactively before the 2-minute deadline, giving pods a clean shutdown.

Bin Packing: Fitting More Pods Per Node

Even with correct resource requests, poor scheduling decisions leave nodes half-empty while others are full. The scheduler’s default LeastAllocated strategy spreads pods across nodes for resilience — good for fault tolerance, bad for cost.

Descheduler: Rebalance After the Fact

The Descheduler evicts pods that landed suboptimally, letting the scheduler re-place them better:

1
2
3


helm repo add descheduler https://kubernetes-sigs.github.io/descheduler/
helm upgrade --install descheduler descheduler/descheduler \
  --namespace kube-system

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43


apiVersion: v1
kind: ConfigMap
metadata:
  name: descheduler-policy
  namespace: kube-system
data:
  policy.yaml: |
    apiVersion: "descheduler/v1alpha2"
    kind: "DeschedulerPolicy"
    profiles:
      - name: default
        pluginConfig:
          - name: DefaultEvictor
            args:
              ignorePvcPods: true
              evictLocalStoragePods: false
              nodeFit: true
        plugins:
          balance:
            enabled:
              - RemovePodsViolatingTopologySpreadConstraints
              - LowNodeUtilization
          deschedule:
            enabled:
              - RemoveDuplicates
              - RemovePodsViolatingNodeAffinity
              - RemovePodsViolatingInterPodAntiAffinity

          # Key plugin: consolidate pods onto fewer nodes
          balance:
            enabled:
              - LowNodeUtilization
        pluginConfig:
          - name: LowNodeUtilization
            args:
              thresholds:
                cpu: 20       # Nodes below 20% CPU are "underutilized"
                memory: 20
                pods: 20
              targetThresholds:
                cpu: 50       # Evict pods until node is below 50%
                memory: 50
                pods: 50

Karpenter Consolidation

Karpenter’s consolidationPolicy: WhenUnderutilized automatically removes underutilized nodes and reschedules their pods. This is bin packing on autopilot:

1
2
3
4
5
6
7


spec:
  disruption:
    consolidationPolicy: WhenUnderutilized
    consolidateAfter: 30s    # How long to wait before consolidating
    # budgets: limit how many nodes can be disrupted simultaneously
    budgets:
      - nodes: "10%"         # Don't disrupt more than 10% of nodes at once

Topology Spread Constraints for Cost-Aware Spreading

Instead of spreading pods evenly across all AZs (expensive cross-AZ traffic), prefer the cheapest AZ while maintaining basic resilience:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


spec:
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 2
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway    # Soft constraint
          labelSelector:
            matchLabels:
              app: web-api

Scale-to-Zero with KEDA

KEDA (Kubernetes Event-Driven Autoscaler) extends HPA to scale workloads to zero when there’s no work to do, and back up instantly when demand returns. For batch workloads, background jobs, and low-traffic services, this can eliminate compute cost entirely during idle periods.

Install KEDA

1
2
3


helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda --create-namespace

Scale to Zero on Queue Depth

The most common KEDA pattern: scale workers based on queue depth.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


# ScaledObject: scale the order-processor Deployment based on SQS queue depth
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor-scaler
  namespace: production
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 0        # Scale to zero when queue is empty
  maxReplicaCount: 50       # Max 50 workers
  pollingInterval: 15       # Check queue every 15 seconds
  cooldownPeriod: 60        # Wait 60s after queue drains before scaling to 0

  triggers:
    - type: aws-sqs-queue
      authenticationRef:
        name: keda-aws-credentials
      metadata:
        queueURL: https://sqs.us-east-1.amazonaws.com/123456789/orders
        queueLength: "5"        # Target: 5 messages per worker
        awsRegion: us-east-1
        identityOwner: operator

1
2
3
4
5
6
7
8
9


# TriggerAuthentication: how KEDA authenticates with AWS
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-aws-credentials
  namespace: production
spec:
  podIdentity:
    provider: aws            # Use IRSA (IAM Roles for Service Accounts)

Scale on Cron Schedule

For predictable traffic patterns, pre-scale before demand arrives:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: business-hours-scaler
  namespace: production
spec:
  scaleTargetRef:
    name: report-generator
  minReplicaCount: 0
  maxReplicaCount: 20
  triggers:
    - type: cron
      metadata:
        timezone: America/New_York
        start: "0 8 * * 1-5"     # 8 AM weekdays — scale up
        end: "0 19 * * 1-5"      # 7 PM weekdays — scale down
        desiredReplicas: "10"

Scale on Prometheus Metrics

Scale based on any metric in Prometheus — request rate, queue latency, custom business metrics:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: api-scaler
  namespace: production
spec:
  scaleTargetRef:
    name: api-server
  minReplicaCount: 0
  maxReplicaCount: 100
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring.svc:9090
        metricName: http_requests_per_second
        threshold: "100"       # Scale up when > 100 req/s per replica
        query: |
          sum(rate(http_requests_total{service="api-server"}[2m]))

Scale on Kafka Lag

For Kafka consumers, scale on consumer group lag:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: kafka-consumer-scaler
  namespace: production
spec:
  scaleTargetRef:
    name: kafka-consumer
  minReplicaCount: 0
  maxReplicaCount: 30
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka.production.svc:9092
        consumerGroup: order-processor-group
        topic: orders
        lagThreshold: "100"     # 100 messages lag per consumer
        offsetResetPolicy: latest

Scale-to-Zero for Preview Environments

One of the highest-ROI KEDA use cases: preview (PR) environments that scale to zero overnight and on weekends:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: preview-pr-123-scaler
  namespace: preview-pr-123
spec:
  scaleTargetRef:
    name: app
  minReplicaCount: 0
  maxReplicaCount: 2
  triggers:
    # Scale up during business hours only
    - type: cron
      metadata:
        timezone: UTC
        start: "0 8 * * 1-5"
        end: "0 19 * * 1-5"
        desiredReplicas: "1"
    # OR: scale up when there's HTTP traffic (using Prometheus/NGINX metrics)
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring.svc:9090
        metricName: nginx_ingress_requests
        threshold: "1"
        query: |
          sum(rate(nginx_ingress_controller_requests{
            namespace="preview-pr-123"
          }[5m]))

Measuring Cost with Kubecost

Kubecost allocates cloud costs to Kubernetes namespaces, deployments, labels, and teams. The free tier covers most needs for a single cluster.

Install Kubecost

1
2
3
4
5
6
7
8
9


helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm upgrade --install kubecost kubecost/cost-analyzer \
  --namespace kubecost --create-namespace \
  --set kubecostToken="your-token" \
  --set global.prometheus.fqdn=http://prometheus.monitoring.svc:9090 \
  --set global.prometheus.enabled=false    # Use existing Prometheus

# Access the UI
kubectl port-forward -n kubecost svc/kubecost-cost-analyzer 9090

Key Kubecost Views

Cost allocation: break down spend by namespace, label, deployment, or team. Use this to answer “which team is spending the most?” and drive accountability.

Savings insights: Kubecost identifies specific underutilized workloads and quantifies the savings from rightsizing.

Cost efficiency: ratio of actual cost to requested resources vs actual usage. A cluster at 30% efficiency has a lot of headroom.

Automated Savings Reports via API

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


import requests

KUBECOST_URL = "http://localhost:9090"

# Get namespace cost breakdown for last 7 days
response = requests.get(
    f"{KUBECOST_URL}/model/allocation",
    params={
        "window": "7d",
        "aggregate": "namespace",
        "accumulate": "true",
    }
)

allocations = response.json()["data"][0]
results = []
for namespace, data in allocations.items():
    results.append({
        "namespace": namespace,
        "total_cost": round(data["totalCost"], 2),
        "cpu_cost": round(data["cpuCost"], 2),
        "memory_cost": round(data["ramCost"], 2),
        "efficiency": round(data.get("totalEfficiency", 0) * 100, 1),
    })

results.sort(key=lambda x: x["total_cost"], reverse=True)
print(f"{'Namespace':<30} {'7d Cost':>10} {'CPU':>8} {'Mem':>8} {'Efficiency':>12}")
print("-" * 72)
for r in results[:20]:
    print(f"{r['namespace']:<30} ${r['total_cost']:>9} ${r['cpu_cost']:>7} ${r['memory_cost']:>7} {r['efficiency']:>10}%")

Cost Allocation Labels

Tag your workloads so Kubecost can group costs by team and product:

1
2
3
4
5
6
7
8


# Standard label set for cost attribution
metadata:
  labels:
    app.kubernetes.io/name: web-api
    app.kubernetes.io/part-of: payments-platform
    team: payments
    cost-center: "cc-1234"
    environment: production

1
2


# Kubecost query by team label
curl "http://localhost:9090/model/allocation?window=30d&aggregate=label:team"

LimitRanges and ResourceQuotas: Governance at Scale

Without governance, a misconfigured deployment can request all cluster resources. LimitRanges and ResourceQuotas prevent this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


# LimitRange: default requests/limits for containers that don't specify them
apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: production
spec:
  limits:
    - type: Container
      default:
        cpu: 500m
        memory: 256Mi
      defaultRequest:
        cpu: 100m
        memory: 128Mi
      max:
        cpu: "4"
        memory: 4Gi
      min:
        cpu: 50m
        memory: 64Mi
    - type: PersistentVolumeClaim
      max:
        storage: 100Gi

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


# ResourceQuota: hard caps per namespace
apiVersion: v1
kind: ResourceQuota
metadata:
  name: production-quota
  namespace: production
spec:
  hard:
    requests.cpu: "100"         # Total requested CPU across all pods
    requests.memory: 200Gi
    limits.cpu: "200"
    limits.memory: 400Gi
    count/pods: "500"
    count/services: "50"
    count/persistentvolumeclaims: "100"
    requests.storage: 5Ti

Practical Cost Reduction Playbook

Apply these in order — earlier items have lower risk and higher ROI:

Week 1: Measure and Identify Waste

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


# Find pods with no resource requests (they get 0 priority from VPA/scheduler)
kubectl get pods -A -o json | jq -r '
  .items[] |
  select(.spec.containers[].resources.requests == null) |
  [.metadata.namespace, .metadata.name] | @tsv
'

# Find deployments with only 1 replica (no HA, no spot tolerance)
kubectl get deployments -A -o json | jq -r '
  .items[] |
  select(.spec.replicas == 1) |
  [.metadata.namespace, .metadata.name, (.spec.replicas | tostring)] | @tsv
'

# Find namespaces without ResourceQuota
kubectl get resourcequota -A -o json | jq -r '[.items[].metadata.namespace] | unique | .[]' > has_quota.txt
kubectl get namespaces -o json | jq -r '.items[].metadata.name' > all_namespaces.txt
comm -23 <(sort all_namespaces.txt) <(sort has_quota.txt)

# Find PVCs that aren't mounted (orphaned storage)
kubectl get pvc -A -o json | jq -r '
  .items[] |
  select(.status.phase == "Bound") |
  select(.metadata.annotations["volume.kubernetes.io/storage-provisioner"] != null) |
  [.metadata.namespace, .metadata.name, .spec.resources.requests.storage] | @tsv
' | head -30

Week 2: Rightsize with VPA

Deploy VPA in Off mode everywhere, collect 1 week of data, apply recommendations to staging first, then production.

Week 3: Enable Spot for Eligible Workloads

Move stateless Deployments (web tier, API servers, background workers) to spot node groups or Karpenter NodePools that prefer spot.

Week 4: KEDA Scale-to-Zero

Enable KEDA for batch workloads, queue consumers, and non-production environments.

Week 5+: Continuous Optimization

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


# Kyverno policy: require resource requests on all containers
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-resource-requests
spec:
  validationFailureAction: enforce
  rules:
    - name: check-requests
      match:
        resources:
          kinds: [Pod]
      validate:
        message: "CPU and memory requests are required"
        pattern:
          spec:
            containers:
              - resources:
                  requests:
                    memory: "?*"
                    cpu: "?*"

Quick Reference: Cost Levers and Expected Impact

Optimization	Typical Savings	Complexity	Risk
VPA rightsizing	20-40%	Low	Low (staging first)
Spot/preemptible nodes	50-70% on eligible workloads	Medium	Medium (interruptions)
KEDA scale-to-zero (non-prod)	60-80% on non-prod	Medium	Low
Karpenter consolidation	10-20%	Low	Low
Delete unused PVCs/LBs	5-15%	Low	None
Remove unused namespaces/clusters	Variable	Low	None
Reserved instances for baseline	30-40% on committed spend	Low	Medium (commitment)
Graviton/Arm instances	20% on compute	Medium	Low
Cross-AZ traffic reduction	5-15%	Medium	Low

The biggest gains are almost always VPA rightsizing (people wildly overestimate what their pods need) and spot instances (trading availability SLA for 70% cost reduction). Start there, measure, then proceed to the more complex options.

Cost optimization in Kubernetes is not a one-time project — it’s a continuous practice. The infrastructure that’s correctly sized today will be over-provisioned in six months as traffic patterns change. Build the measurement tooling (Kubecost or equivalent), automate the governance (LimitRanges, ResourceQuotas, Kyverno policies), and treat the cluster like a product that requires ongoing care.