Monitoring CI

What to watch

CI is software too. Treat it like a service: monitor its health and respond when it degrades. The metrics that matter:

Pass rate on main — %% of main runs that go green. Target 95%+. Below 90% is a crisis.
p50 / p95 duration — feedback latency.
Flake rate — % of failed runs that pass on retry. >2% means you have flakes to fix.
Queue time — how long runs wait before starting. Self-hosted fleet too small if this grows.
Cost — minutes used per repo, per workflow, per OS.

Where to put the data

Built-in — Insights tab, Actions usage report. Good first step, limited drill-down.
Datadog / Honeycomb / Grafana — pipe Actions events via the workflow_run webhook to your observability stack.
Custom — a scheduled job that calls the GitHub API, computes metrics, posts to Slack / a dashboard / a database.

Alerting

Page when main pass rate drops below threshold. Slack on flake-rate spike. Don't alert on individual failures — those are normal noise.

Code

Scheduled job that posts CI weekly digest·yaml

name: ci-digest
on:
  schedule: [{ cron: '0 9 * * 1' }]   # Monday 09:00 UTC
  workflow_dispatch: {}

permissions:
  actions: read
  contents: read

jobs:
  digest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Compute and post digest
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SLACK_WEBHOOK: ${{ secrets.SLACK_DEPLOY_WEBHOOK }}
        run: ./scripts/ci-digest.sh

What to watch

Where to put the data

Alerting

Code

External links

Exercise

Progress

Comments 0