iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

Rollbacks

By · Reviewed & last updated · Part of Kubernetes Tutorial

Rolling a Deployment back is one command if you set things up. The mechanism: every change to the Deployment spec creates a new ReplicaSet; old ReplicaSets stay around (up to revisionHistoryLimit), and `kubectl rollout undo` reactivates the previous one. With healthchecks and a sensible budget, rollback is a 60-second operation.

Inspect, undo, and prevent the need for rollbacks

EXAMPLE
# 1) Deploy a change
kubectl -n shop set image deployment/shop-api api=example.com/shop-api:v1.4.1
# OR: edit / apply a manifest with the new image

# 2) Watch the rollout
kubectl -n shop rollout status deployment/shop-api --timeout=10m

# 3) Inspect the history
kubectl -n shop rollout history deployment/shop-api
# REVISION  CHANGE-CAUSE
# 1         kubectl set image ... v1.4.0
# 2         kubectl set image ... v1.4.1   <-- newest

kubectl -n shop rollout history deployment/shop-api --revision=2

# 4) Roll back to the previous revision
kubectl -n shop rollout undo deployment/shop-api

# 5) Roll back to a SPECIFIC revision
kubectl -n shop rollout undo deployment/shop-api --to-revision=1

# 6) Pause + resume — useful for canary / staged rollouts
kubectl -n shop rollout pause deployment/shop-api
# (apply more changes; nothing rolls yet)
kubectl -n shop rollout resume deployment/shop-api

# 7) Restart all pods of a deployment (no spec change — picks up new ConfigMaps/secrets)
kubectl -n shop rollout restart deployment/shop-api

# 8) Keep MORE history if you roll back often
# In the Deployment spec:
# spec.revisionHistoryLimit: 10
# Default is 10; set to 20 if you want a wider safety net.

# 9) Make CHANGE-CAUSE useful
kubectl -n shop annotate deployment/shop-api \
  kubernetes.io/change-cause='Release v1.4.1 - shipping calc fix' --overwrite
# Now 'rollout history' shows a meaningful row, not 'kubectl set image ...'

# 10) Why rollback is sometimes NOT enough
# - Pods are healthy AND broken (200 OK responses with wrong data).
#   Add deeper readiness probes that actually verify a known-good response.
# - The release migrated the database forward; rolling back the binary leaves
#   the schema ahead. Plan two-phase migrations (additive only).
# - The release flipped a feature flag at runtime; rollback the FLAG, not the pod.

# 11) Tooling that makes rollback feel boring
# - GitHub Actions or GitLab CI lane that runs 'kubectl rollout undo' on PR comment
# - Argo Rollouts: declarative canary/blue-green with metric-based auto-revert
# - Flagger:        works with Linkerd, Istio, App Mesh; promotes / rolls back on SLO

# 12) Pre-rollback drill
# 1. Confirm the issue is in the CURRENT revision (not config / DB / upstream)
# 2. Decide: roll back (binary stays compatible) OR roll forward (binary broke schema)
# 3. Snapshot any state you may need (DB, queue depths, traces)
# 4. Run 'kubectl rollout undo'
# 5. Watch 'kubectl rollout status' and your SLO dashboards for 10 minutes
# 6. Post-incident: how did the bad revision pass canary? Update the playbook.

# 13) Make it impossible to roll back into trouble
# - PodDisruptionBudget prevents the rollback from dropping below capacity
# - readinessProbe ensures only HEALTHY pods take traffic
# - maxUnavailable=0 + maxSurge>0 = new pods come up first, old ones go second
# - DB changes are additive across releases so old code keeps working

Why it matters

Annotate every deploy with `kubernetes.io/change-cause`. Six months later, `rollout history` becomes the human-readable changelog you wish you had — and during an incident, finding "the last green revision" goes from grep-the-CI-logs to a one-line `kubectl` command.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
kubectl rollout undo deployment/api
kubectl rollout history deployment/api
Try it Yourself »

Discussion

Loading…