Cuckoo

Monitor AWS EventBridge and Lambda Cron Jobs

Updated

A Lambda function triggered by an EventBridge schedule is easy to set up and easy to lose track of. A schedule can be disabled during a deploy, an IAM change can block the invocation, the function can time out every night, and unless you have built CloudWatch alarms for each of these, nobody is told. This guide adds Cuckoo pings to the handler so a failure or a missing run sends an alert.

How Cuckoo sees the function

Your handler calls the check's ping URL:

  • https://cuckoo.metastic.ch/ping/YOUR-UUID/start at the beginning,
  • https://cuckoo.metastic.ch/ping/YOUR-UUID on success,
  • https://cuckoo.metastic.ch/ping/YOUR-UUID/fail on an exception.

If nothing arrives within the schedule plus grace time, the check goes down. That covers runs that were never invoked, which produce no Lambda error metrics at all.

EventBridge cron syntax

EventBridge uses six fields, not the five of Unix cron:

cron(minutes hours day-of-month month day-of-week year)

cron(30 2 * * ? *)        # every day at 02:30
cron(0 6 ? * MON-FRI *)   # weekdays at 06:00
cron(15 */4 * * ? *)      # every 4 hours at minute 15
  • One of day-of-month or day-of-week must be ?. You cannot set * in both.
  • Day-of-week uses 1-7 with 1 = Sunday, or names like MON.
  • EventBridge rules (the older scheduled rules) always evaluate in UTC.
  • EventBridge Scheduler accepts a time zone per schedule and handles daylight saving time.

A Scheduler schedule with a time zone:

aws scheduler create-schedule \
  --name nightly-report \
  --schedule-expression "cron(30 2 * * ? *)" \
  --schedule-expression-timezone "Europe/Zurich" \
  --flexible-time-window Mode=OFF \
  --target '{"Arn":"arn:aws:lambda:eu-central-1:123456789012:function:nightly-report","RoleArn":"arn:aws:iam::123456789012:role/scheduler-invoke","RetryPolicy":{"MaximumRetryAttempts":0}}'

Step 1: put the ping URL in an environment variable

aws lambda update-function-configuration \
  --function-name nightly-report \
  --environment '{"Variables":{"CUCKOO_URL":"https://cuckoo.metastic.ch/ping/YOUR-UUID"}}'

Note that this command replaces all existing environment variables of the function; include the others too, or set the variable in your infrastructure code.

Step 2a: Python handler

urllib is in the standard library, so no extra package is needed:

import os
import urllib.request

HB = os.environ["CUCKOO_URL"]

def ping(suffix="", body=None):
    try:
        data = body.encode("utf-8")[:10000] if body else None
        with urllib.request.urlopen(HB + suffix, data=data, timeout=5):
            pass
    except Exception as exc:  # never let a ping break the job
        print(f"heartbeat ping failed: {exc}")

def run_job(event):
    ...  # your work here

def handler(event, context):
    ping("/start")
    try:
        run_job(event)
    except Exception as exc:
        ping("/fail", body=repr(exc))
        raise
    ping()
    return {"ok": True}

Step 2b: Node.js handler

Node.js 18 and later runtimes include fetch:

const HB = process.env.CUCKOO_URL;

async function ping(suffix = "", body) {
  try {
    await fetch(HB + suffix, {
      method: body ? "POST" : "GET",
      body: body ? String(body).slice(0, 10000) : undefined,
      signal: AbortSignal.timeout(5000),
    });
  } catch (err) {
    console.log("heartbeat ping failed:", err.message);
  }
}

async function runJob(event) {
  // your work here
}

export const handler = async (event) => {
  await ping("/start");
  try {
    await runJob(event);
  } catch (err) {
    await ping("/fail", err.stack || String(err));
    throw err;
  }
  await ping();
  return { ok: true };
};

Always await the ping. Once the handler returns, Lambda can freeze the execution environment, and a request still in flight may never be sent.

Timeouts and retries

  • Function timeout: when Lambda kills a function at its timeout (up to 15 minutes), the catch block never runs, so no fail ping is sent. Configure max runtime on the check slightly above the function timeout: the /start ping arrived, nothing followed, so the check goes down.
  • Ping timeout: keep the ping timeout short (5 seconds above) so a slow network does not eat into your function time.
  • Async retries: EventBridge invokes Lambda asynchronously, and Lambda retries a failed async invocation up to two times by default. You may see fail, then start again, then success. Set the function's MaximumRetryAttempts to 0 if you do not want retries.
  • Scheduler retries: the Scheduler RetryPolicy covers failures to deliver the invocation, such as throttling, which is separate from function errors.

Schedule and grace advice

Enter the same cron expression in Cuckoo in five-field form, dropping the year and turning ? into *: cron(30 2 * * ? *) becomes 30 2 * * *. Note that Unix cron numbers weekdays from 0 = Sunday, so prefer names such as MON-FRI. Use UTC for EventBridge rules, or the Scheduler time zone. A grace time of a few minutes plus the expected runtime works for short functions; add more if retries are enabled.

Troubleshooting

  • No pings, no invocations: check that the schedule is ENABLED and the role in RoleArn may call lambda:InvokeFunction. For rules, the function needs a resource-based permission for events.amazonaws.com.
  • Invocations, but no pings: functions in a VPC need a NAT gateway or other egress path to the internet.
  • Ping errors in logs: look for DNS or TLS errors in CloudWatch Logs.

FAQ

Does Cuckoo replace CloudWatch alarms?

Not necessarily. CloudWatch is useful for duration, throttles and error rates. Cuckoo adds an outside check that alerts when the expected run does not report, including when it was never invoked.

Should I use EventBridge rules or EventBridge Scheduler?

AWS recommends Scheduler for new schedules. It supports time zones, one-time schedules and retry policies per schedule. Existing rules keep working, but they evaluate cron expressions in UTC.

How do I monitor a Step Functions workflow on a schedule?

Ping /start from a first state and success or /fail from the final states. For example, add a Catch that routes to a failure ping, or use an HTTP Task, so every path reports.

Related guides

All guides · Compare Cuckoo with other tools

Send feedback

Ideas, bugs, questions: we read every message. Or write to hello@metastic.ch.