What it stores · where · how it flows
A Secret stores sensitive data — passwords, tokens, TLS certs, SSH keys. It lives as an object in etcd (base64 encoded by default). Pods consume it as env vars or volume-mounted files.
What's inside a Secret
$ kubectl create secret generic db-creds \ --from-literal=password=p@ssw0rd \ --from-literal=username=admin -n prod secret/db-creds created $ kubectl get secret db-creds -n prod -o yaml data: password: cEBzc3cwcmQ= username: YWRtaW4= $ echo "cEBzc3cwcmQ=" | base64 -d p@ssw0rd ← decoded in 0.01 seconds!
Where it lives — storage journey
aws eks create-cluster --secrets-encryption-config + KMS key.How a Pod consumes a Secret — Method 1: Environment Variable
# In Pod spec: env: - name: DB_PASSWORD # env var name in container valueFrom: secretKeyRef: name: db-creds # Secret object name key: password # key inside the Secret $ kubectl exec -it nginx -n prod -- env | grep DB DB_PASSWORD=p@ssw0rd ← visible in process environment! # Problem: visible in kubectl describe pod, process list, crash dumps
kubectl describe pod output, ps aux on the node, application crash reports, and CI/CD logs that print all env vars.How a Pod consumes a Secret — Method 2: Volume Mount (safer)
# In Pod spec: volumes: - name: creds-vol secret: secretName: db-creds # which Secret to mount volumeMounts: - name: creds-vol mountPath: /etc/creds # directory in container readOnly: true # always set readOnly! $ kubectl exec -it nginx -n prod -- cat /etc/creds/password p@ssw0rd # File is in-memory tmpfs — never written to disk # Not visible in kubectl describe, process list, or logs # Auto-updated when Secret rotates (with ASCP)
Secret types — not all Secrets are Opaque
OpaqueGeneric secrets. Arbitrary key-value pairs. What you've been using. DB passwords, API keys, tokens.
kubernetes.io/tlsTLS cert + private key. Used by Ingress controllers. Fields: tls.crt and tls.key.
kubernetes.io/dockerconfigjsonDocker registry credentials. Used as imagePullSecrets so pods can pull from private ECR/registries.
What it stores · where · how it flows
A ConfigMap stores non-sensitive configuration: feature flags, database hostnames, log levels, entire config files. Same image, different ConfigMap = different environment. Stored in etcd in plaintext.
What's inside a ConfigMap
$ kubectl create configmap app-config \ --from-literal=DB_HOST=postgres.prod \ --from-literal=LOG_LEVEL=info \ --from-file=nginx.conf=./nginx.conf -n prod configmap/app-config created
ConfigMap vs Secret — when to use each
What's inside a Pod · how containers share it
A Pod is the smallest deployable unit. It wraps one or more containers that share a network namespace (same IP, same localhost) and optional shared volumes. The Pod spec is what you write; Kubernetes creates the actual containers from it.
What's inside a Pod spec
How containers share a Pod
Pod lifecycle — the exact state machine
What it stores · how traffic flows · 4 types
A Service is a stable virtual endpoint in front of dynamic pods. It stores: a virtual IP (ClusterIP), a label selector (which pods to route to), and port mappings. kube-proxy on every node reads this and writes iptables rules.
What's inside a Service
$ kubectl apply -f svc.yaml service/nginx-svc created $ kubectl describe svc nginx-svc -n prod Selector: app=nginx ClusterIP: 10.96.100.10 Endpoints: 10.244.0.5:80, 10.244.1.3:80 ↑ these are the actual pod IPs behind the service
How traffic flows through a Service
What it stores · rolling update mechanics · rollback
A Deployment stores the desired state for a stateless application: image, replicas, update strategy. It creates ReplicaSets which create Pods. The deployment history lets you instantly roll back to any previous version.
What's inside a Deployment
Rolling update — exactly what happens step by step
Stable identity · ordered ops · own PVC per pod
StatefulSets give each pod a permanent name, a permanent DNS hostname, and its own PersistentVolumeClaim. mysql-0 always gets PVC data-mysql-0. Even after a crash and restart, same pod name, same storage, same DNS hostname.
$ kubectl get pvc -n prod data-mysql-0 Bound 10Gi node-01 ← mysql-0 OWNS this data-mysql-1 Bound 10Gi node-02 ← mysql-1 OWNS this # Delete mysql-0 pod → new pod named mysql-0 binds to SAME PVC # Data survives. Identity survives. Replication config survives.
One pod per node · automatic · node-level agents
A DaemonSet guarantees exactly one copy of a pod runs on every node in the cluster. When a new node joins, the DaemonSet controller automatically places a pod there. When a node leaves, the pod is garbage-collected.
$ kubectl get ds -n kube-system NAME DESIRED CURRENT READY NODE-SELECTOR fluent-bit 2 2 2 <none> # DESIRED = number of nodes in cluster # Scales automatically with node count $ kubectl get pods -n kube-system -o wide | grep fluent fluent-bit-n1 Running node-01 fluent-bit-n2 Running node-02
Why each agent MUST run on every node
Fluent Bit reads log files from the node's own filesystem. If you ran it as a Deployment with 2 replicas, the scheduler might put both on node-01 — node-02's logs would be completely missed.
What a node reports · capacity vs allocatable · conditions
A Node object in Kubernetes represents a worker machine (EC2 on EKS). It reports its capacity, allocatable resources, conditions (Ready/Memory/DiskPressure), and lists all pods running on it.
$ kubectl describe node node-01 Capacity: cpu: 2 memory: 3967Mi Allocatable: cpu: 1900m memory: 3467Mi Allocated resources: cpu: 1000m/1900m (52%) ← sum of all pod requests memory: 1200Mi/3467Mi (34%) # Scheduler uses Allocatable, not Capacity # Remaining allocatable = how many more pods can fit
Capacity vs Allocatable — the 100m gap matters
api-server · etcd · scheduler · controller-manager — complete data flow
The control plane processes every kubectl command. Here is the exact sequence of what happens when you run kubectl apply -f deploy.yaml — across all four components.
Complete flow: kubectl apply → running pods
api-server — the gatekeeper
# What it checks on every request: 1. Authentication "Are you who you say?" (IAM token on EKS) 2. Authorization (RBAC) "Can this identity do this?" (kubectl apply needs create) 3. Admission Control "Does this object meet policy?" (resource limits set?) 4. Validation "Is the YAML schema correct?" (image field not empty?)
etcd — what it actually stores
# etcd key structure: /registry/pods/prod/nginx → full Pod spec + status JSON /registry/services/prod/nginx-svc → Service spec + ClusterIP /registry/deployments/prod/webapp → Deployment spec + history /registry/secrets/prod/db-creds → base64 data (add KMS!) /registry/nodes/node-01 → node capacity + conditions
scheduler — watch → decide → write
# Scheduler algorithm: 1. Watch etcd for pods where spec.nodeName == "" 2. Run predicates (filter) - ResourceFit (cpu+mem) - NodeReady condition - Taints/tolerations - Node/pod affinity 3. Score survivors - Spread (prefer empty) - Least waste - Zone balance 4. Write nodeName to etcd → kubelet sees it and acts