Cuckoo

Monitor GitLab CI Scheduled Pipelines

Updated

GitLab pipeline schedules are a handy way to run nightly tests, dependency updates, backups or report jobs. When they break, though, the signal is weak. A schedule becomes inactive when its owner leaves the project, a runner tag stops matching, or a rules: change quietly removes the job from scheduled pipelines. In each case there is no failed pipeline to notify about, because nothing ran. This guide adds a Cuckoo check to a scheduled job so a failure or a missing run triggers an alert.

How Cuckoo sees the pipeline

Each check has a ping URL. The job reports to it:

  • https://cuckoo.metastic.ch/ping/YOUR-UUID/start when the job starts,
  • https://cuckoo.metastic.ch/ping/YOUR-UUID when it succeeded,
  • https://cuckoo.metastic.ch/ping/YOUR-UUID/fail when it failed.

The check has the same schedule as the pipeline. If no success or failure arrives within the schedule plus grace time, it goes down and you are notified by email, Slack, Discord or webhook.

Step 1: create the pipeline schedule

In the project, open Build → Pipeline schedules → New schedule. Set:

  • a description, for example nightly-backup,
  • the interval pattern as a cron expression, for example 30 1 * * *,
  • the cron time zone (GitLab lets you choose one per schedule),
  • the target branch, usually the default branch.

The schedule runs as the user who owns it. If that user loses access to the project, the schedule stops running, so consider a dedicated bot or service account as the owner.

Scheduled pipelines are queued through a background worker, so on GitLab.com and many self-managed instances the actual start can be a few minutes after the cron time. Plan grace time accordingly.

Step 2: store the ping URL as a masked variable

In Settings → CI/CD → Variables → Add variable:

  • Key: CUCKOO_URL
  • Value: https://cuckoo.metastic.ch/ping/YOUR-UUID
  • Visibility: Masked (or Masked and hidden)
  • Protect variable: enable it if the schedule targets a protected branch, which keeps the value away from pipelines on other branches.
  • Clear Expand variable reference, which GitLab recommends for masked values.

A masked value must be a single line without spaces and at least 8 characters long, which a ping URL is. GitLab then replaces it with [MASKED] in job logs.

Step 3: the job

nightly-backup:
  image: alpine:3.20
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
  before_script:
    - apk add --no-cache curl
    - curl -fsS -m 10 --retry 5 -o /dev/null "$CUCKOO_URL/start" || true
  script:
    - ./scripts/backup.sh
  after_script:
    - |
      if [ "$CI_JOB_STATUS" = "success" ]; then
        curl -fsS -m 10 --retry 5 -o /dev/null "$CUCKOO_URL" || true
      else
        curl -fsS -m 10 --retry 5 -o /dev/null "$CUCKOO_URL/fail" || true
      fi

How it works:

  • rules: - if: $CI_PIPELINE_SOURCE == "schedule" runs the job only in scheduled pipelines, so pushes and merge requests do not send pings.
  • after_script runs after script, whether it passed or failed. It runs in a separate shell, so variables you export in before_script are not available there, but CI/CD variables such as CUCKOO_URL are.
  • CI_JOB_STATUS is success, failed or canceled inside after_script. Reporting anything other than success as a failure keeps cancelled runs visible.
  • The result of after_script does not change the job status, so a failing ping never breaks the pipeline.

If one project has several schedules, add a variable to each schedule (for example SCHEDULE_NAME=backup) and match it in rules: so each schedule only runs its own job, then give each job its own check and URL variable.

Several jobs and stages

When the scheduled work spans multiple stages, put the pings in dedicated jobs instead:

heartbeat-start:
  stage: .pre
  image: curlimages/curl:latest
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
  script:
    - curl -fsS -m 10 --retry 5 -o /dev/null "$CUCKOO_URL/start" || true

heartbeat-success:
  stage: .post
  image: curlimages/curl:latest
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: on_success
  script:
    - curl -fsS -m 10 --retry 5 -o /dev/null "$CUCKOO_URL" || true

heartbeat-failure:
  stage: .post
  image: curlimages/curl:latest
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: on_failure
  script:
    - curl -fsS -m 10 --retry 5 -o /dev/null "$CUCKOO_URL/fail" || true

.pre and .post are built-in stages that always run first and last.

Schedule and grace advice

Copy the cron expression and time zone from the pipeline schedule into the check. Set the grace time to the usual pipeline duration, including the time a job waits for a runner, plus a buffer for scheduling delay. Set a max runtime near the job timeout so a job killed by a timeout, where after_script may not get to report, still makes the check go down.

Troubleshooting

  • Check down, no pipeline in the list: open the schedule and check that it is active and that its owner still has access. The Last pipeline column shows whether it fired.
  • Pipeline ran but the job is missing: the rules: do not match, or a variable set on the schedule is not what the rule expects.
  • Job stuck in pending: no runner with the required tags is online. The check goes down after the grace time, which is the intended alert.
  • URL is empty in the job: a protected variable is not available on unprotected branches.

FAQ

Can I trigger a scheduled pipeline manually to test the check?

Yes. Click Run (the play button) next to the schedule. The pipeline then has CI_PIPELINE_SOURCE set to schedule, so the job and its pings run exactly as they would on the timer.

Does after_script run when a job is cancelled?

On current GitLab versions, after_script runs when a job is cancelled while before_script or script is running, and CI_JOB_STATUS is canceled. A job cancelled while still pending does not run it.

Can I send the job output with the failure ping?

Yes. Write the output to a file in script, then POST it in after_script, for example tail -c 10000 out.log | curl -fsS -m 10 --data-binary @- "$CUCKOO_URL/fail". The first 10 KB are stored.

Related guides

All guides · Compare Cuckoo with other tools

Send feedback

Ideas, bugs, questions: we read every message. Or write to hello@metastic.ch.