⚖️ Module 05 — Managing Applications at Scale in Amazon EKS

Pods (replicas). HPA scales the number of Pod instances horizontally — adding more identical pods under load, removing them when load drops. It does not resize CPU/memory on existing pods.

kubectl autoscale deployment myapp --cpu-percent=50 --min=3 --max=10

This creates an HPA object that watches average CPU utilisation across all pods and scales between 3 and 10 replicas to keep it near 50%.

Built-in: CPU utilisation, memory utilisation
Custom metrics: any metric from Kubernetes Metrics API (e.g. requests-per-second, queue depth)
External metrics: metrics from outside the cluster (e.g. SQS queue length via KEDA)

Pods must have CPU resource requests set in their spec. HPA measures utilisation as a percentage of the requested CPU. Without requests, the Metrics Server cannot compute utilisation and HPA will not scale.

vpa-recommender. It continuously watches historical and current pod CPU/memory usage and produces a recommendation. The recommender writes its output to the VPA object's status.recommendation field.

Off (zero disruption — read-only) or InPlaceOrRecreate (tries in-place resize first, only evicts if in-place is impossible). For production workloads, start with Off to observe recommendations before enabling any mutation.

It is a MutatingAdmissionWebhook that intercepts pod creation requests and rewrites the resource requests/limits to match VPA's current recommendation — before the pod is scheduled. This means new pods start right-sized even before the updater acts.

They fight each other: HPA adds replicas when CPU is high; VPA simultaneously increases each pod's CPU request. The Metrics Server sees changing denominators, causing oscillation. Safe combo: HPA on custom/external metrics + VPA for CPU/memory right-sizing.

Pods stuck in Pending state because no existing node has sufficient CPU/memory or matches the pod's node selector/affinity constraints. CA simulates scheduling and determines which node group to expand.

They remain Pending indefinitely. CA cannot exceed the ASG's maximum. Once 6 nodes are running, CA has hit the hard limit — the 2 extra pods cannot be scheduled until you raise the ASG max or reduce the replica count.

An ASG configuration that lets a single node group use a mix of On-Demand and Spot instances. CA can scale this group normally — it doesn't need to know which individual instance type is launched. Reduces cost while maintaining capacity.

CA tags ASGs with k8s.io/cluster-autoscaler/<cluster-name>=owned and k8s.io/cluster-autoscaler/enabled=true. The CA controller discovers all matching ASGs automatically — no need to list ASG names statically in the CA deployment config.

ASG dependency. CA requires pre-configured Auto Scaling Groups — it can only scale groups that exist. Karpenter provisions EC2 instances directly via the EC2 API, choosing instance type and size based on the pending pod's exact requirements — no ASG needed.

Choose Karpenter when:
• Spiky demand — fast node provisioning needed
• Diverse compute — need many instance families/sizes
• Cost optimisation — want Karpenter to auto-select cheapest instance that fits
• Spot workloads — Karpenter handles spot interruption and rebalancing natively

karpenter.sh/do-not-evict: "true" placed on the pod annotation. This tells Karpenter's consolidation logic to skip evicting that pod when it tries to remove underutilised nodes. Use for stateful workloads, long-running jobs, or anything that cannot tolerate rescheduling.

Karpenter applies constraints from multiple layers: cluster-level NodePool constraints → workload pod spec (node selector, affinity, tolerations). Each layer narrows the provisioning decision. For example: the NodePool allows only us-east-1a + us-east-1b, and the pod's affinity further restricts to GPU nodes.

Karpenter is an open-source project originally created and currently maintained by AWS. It was donated to the CNCF ecosystem. It works with any Kubernetes cluster on any cloud, though it has native AWS EC2 integration via the karpenter-provider-aws.

Cluster Autoscaler and Karpenter both scale EC2 nodes. The difference: CA works through existing ASGs, Karpenter provisions nodes directly. HPA and VPA operate at the pod level.

Yes — this is the recommended production pattern. HPA scales pods in response to traffic. When HPA adds pods that cannot fit on existing nodes, those pods go Pending. Karpenter detects Pending pods and provisions the right-sized node. They work at different layers and complement each other.

VPA — it needs vpa-recommender, vpa-updater, and vpa-admission-controller all running. HPA is built into the K8s controller manager. CA and Karpenter each deploy as a single controller.

Plan → Code → Build → Test → Release → Deploy → Operate → Monitor

Memory trick: "Please Can Bob Take Really Detailed Operations Metrics"

CI — automatically builds and tests code on every commit (Plan → Test stages). Ensures the codebase is always in a releasable state.

CD — automatically delivers validated builds to a staging or production environment (Release → Deploy stages). May include a manual approval gate before production.

The built and tested image is pushed to Amazon ECR (Elastic Container Registry). The Deploy stage then references the new ECR image tag in a Kubernetes Deployment manifest — either directly (imperative) or via a GitOps manifest update (declarative).

1. Developer pushes code to app Git repo
2. CI pipeline builds image, runs tests
3. CI pushes new image to ECR
4. CI updates image tag in manifest Git repo
5. Argo CD detects manifest change
6. Argo CD syncs → rolls out new pods in EKS cluster

Argo CD detects drift — it continuously compares live cluster state to Git. The direct kubectl change creates a mismatch. On the next sync cycle (or immediately if auto-sync is on), Argo CD reverts the cluster to the state declared in Git, undoing the manual change.

Declarative = you describe the desired end state (e.g. "3 replicas of v2.1.0"). The GitOps controller figures out how to get there.
Imperative = you issue commands (e.g. "kubectl set image", "kubectl scale"). GitOps eliminates one-off imperative commands because they bypass Git and get reverted.

App code repo — application source code. Changes here trigger the CI pipeline.
Manifest repo (infra/config repo) — Kubernetes YAML manifests (Deployments, Services, etc.) + infrastructure-as-code. Changes here are watched by Argo CD and applied to the cluster.

When auto-sync is enabled in Argo CD, any merge to the watched Git branch automatically triggers a sync to the cluster — no human operator needs to run kubectl or approve a deploy. The Git merge/pull request process IS the approval gate. Argo CD then ensures the cluster matches.

✓ B — HPA

HPA scales pod replicas in response to CPU/traffic. For spiky traffic on stateless apps, HPA is the primary tool. Karpenter/CA add nodes when HPA can't fit the extra pods.

✓ C — vpa-updater

vpa-recommender generates recommendations. vpa-admission-controller injects resources at pod creation. vpa-updater is the only one that evicts running pods to apply new resource settings.

The Auto Scaling Group's maximum size has been reached. CA cannot scale beyond the ASG max. Fix: increase the max capacity on the ASG directly. Also check: IAM permissions for CA to modify the ASG, and that the CA pod is running and healthy.

Argo CD detects the drift between live cluster state and the Git manifest. On the next sync cycle (or immediately with auto-sync), it reverts the cluster to the Git-declared state. The manual change is overwritten.

Generates recommendations only — no changes are applied to pods. Use it to observe what VPA would recommend before enabling mutations. Zero risk to running workloads.

karpenter.sh/do-not-evict: "true" on the pod. This signals Karpenter's consolidation loop to leave the node running even if it's underutilised.

1. Git as single source of truth
2. Declarative (not imperative)
3. Approved changes applied automatically
4. Software agents ensure consistent state (drift detection + auto-reconcile)

Karpenter. It provisions EC2 instances directly via the EC2 Fleet API, choosing the right instance type based on pending pod requirements. Cluster Autoscaler requires pre-configured ASGs.

1. vpa-recommender — generates resource recommendations
2. vpa-updater — evicts pods needing resource changes
3. vpa-admission-controller — mutates new pods at creation

A change in the manifest Git repo. Argo CD polls the repo (or uses webhooks) and detects that the desired state (Git) differs from the current cluster state. It then applies the diff to bring the cluster in line with Git.