Rollback

Rollback is its own workflow, not an afterthought

If your only rollback option is 'redeploy the previous commit', you'll find out during an incident that the previous commit's image is gone, the migration can't be reversed, and the rollback path nobody tested doesn't exist. Build the rollback workflow before you need it.

What rollback actually means

Different deploy targets have different rollback shapes:

Container orchestrator (ECS, Kubernetes) — re-deploy the previous task definition / deployment manifest. Fastest.
Static site (Vercel, Netlify, Pages) — promote the previous deploy. UI button or API.
Lambda — point the alias to the previous version.
Custom VPS deploy — symlink swap, git checkout <previous> and re-run the deploy script.

The rollback workflow

workflow_dispatch with input version: choice populated from the last 10 deploys.
Or: triggered automatically by if: failure() on the smoke job.
Posts a Slack/email alert with the chosen version and the operator.
Always runs the smoke after rollback — verify the rollback itself worked.

Things rollback can't fix

Schema changes already applied — you need a forward-fix migration.
Data already mutated — point-in-time recovery if available, or accept the loss.
External systems already notified (emails sent, payments charged) — those don't un-happen.

Code

Manual rollback workflow·yaml

name: rollback
on:
  workflow_dispatch:
    inputs:
      version:
        type: string
        required: true
        description: 'SHA or tag to roll back to (e.g., abc1234)'
      reason:
        type: string
        required: true
        description: 'Why are we rolling back?'

jobs:
  rollback:
    runs-on: ubuntu-latest
    environment: production
    permissions:
      id-token: write
      contents: read
    steps:
      - run: echo "Rolling back to ${{ inputs.version }} — reason: ${{ inputs.reason }}"
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123:role/gha-deploy-prod
          aws-region: us-east-1
      - run: ./deploy.sh ${{ inputs.version }}
      - run: ./tests/smoke.sh
      - name: Notify
        if: always()
        run: ./notify-slack.sh "Rollback to ${{ inputs.version }}: ${{ job.status }}"

Rollback is its own workflow, not an afterthought

What rollback actually means

The rollback workflow

Things rollback can't fix

Code

External links

Exercise

Progress

Comments 0