Skip to content

feat(huey): Add cron monitoring for Huey periodic tasks - #8052

Draft
wedamija wants to merge 1 commit into
masterfrom
danf/huey-crons
Draft

wedamija wants to merge 1 commit into
masterfrom
danf/huey-crons

Conversation

@wedamija

@wedamija wedamija commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

Description

Part of a series adding automatic Crons monitoring for the most-used job schedulers in each SDK. Check-ins for a monitor that doesn't exist yet are dropped, so today each monitor has to be created by hand with its schedule copied from the code. Sending the job's schedule as the monitor config lets Sentry create the monitor and keep it in sync.

This adds opt-in check-ins for Huey periodic tasks (@huey.periodic_task(...)), modelled on CeleryIntegration(monitor_beat_tasks=True, exclude_beat_tasks=[...]):

sentry_sdk.init(
    integrations=[HueyIntegration(monitor_periodic_tasks=True, exclude_periodic_tasks=["cleanup"])],
)
  • The worker sends in_progress when a periodic task starts and ok or error when it ends. Regular tasks are not affected.
  • The slug is the task name (name= or the function name). exclude_periodic_tasks takes regexes, matched like exclude_beat_tasks.
  • crontab(...) schedules are sent as a crontab in UTC. The crontab is rebuilt from the values Huey parsed, so it matches what Huey runs.
  • Check-ins are sent without a config for custom validate_datetime functions, for crontabs that restrict both day and day_of_week (Huey requires both to match, cron either), and when utc=False, since Huey then runs in the system's local time.

Issues

Docs: getsentry/sentry-docs#20057

Send check-ins around each run of a Huey periodic task when
HueyIntegration(monitor_periodic_tasks=True) is set. Huey crontab()
schedules are sent as the monitor config when Huey runs in UTC, so
Sentry creates the monitor and keeps it in sync.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Coverage Results 📊

✅ 133758 passed | ⏭️ 7293 skipped | Total: 141051 | Pass Rate: 94.83% | Execution Time: 444m 38s

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +109
Passed Tests 📈 +92
Failed Tests —
Skipped Tests 📈 +17

All tests are passing successfully.

✅ Patch coverage is 96.00% (48 of 50 changed executable lines covered; target 80%).
Project statement coverage is 90.27% (up 0.01 percentage points from base (0f7acd5) to head (4588cdd)).

Changed files with executable lines (1)
File Patch coverage Changed executable lines
sentry_sdk/integrations/huey.py 96.00% 48/50 covered; missed: 303, 333; partial branches: 302, 332
Coverage diff
@@            Coverage Diff             @@
##        master     #8052       +/-##
==========================================
+ Coverage    90.26%    90.27%    +0.01%
==========================================
  Files          202       202         —
  Tracked lines     26660     26709       +49
  Branches      9926      9950       +24
==========================================
+ Hits         24062     24109       +47
- Misses        2598      2600        +2
- Partials      1510      1512        +2

Generated by Coverage Action

import sentry_sdk
from sentry_sdk.api import continue_trace, get_baggage, get_traceparent
from sentry_sdk.consts import OP, SPANDATA, SPANSTATUS
from sentry_sdk.crons import monitor

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huey control-flow exceptions produce error check-ins

When CancelExecution, TaskLockedException, or RetryTask propagates from a monitored periodic task, the monitor records an error check-in even though Huey’s exception handler treats these as control-flow exceptions rather than failures. Consider applying that exception handling to the check-in status as well.

Evidence
  • _monitor_periodic_task() wraps the task’s execute function with monitor(...) when periodic-task monitoring is enabled.
  • patch_execute() first wraps task.execute with _wrap_task_execute(), then applies the monitor wrapper, so propagated exceptions reach the monitor.
  • _capture_exception() recognizes Huey control-flow exceptions and returns without capturing an error event, but _wrap_task_execute() re-raises them.
  • monitor.__exit__() sets the check-in status to error whenever an exception reaches it.
Also found at 2 additional locations
  • tests/integrations/huey/test_huey_crons.py:30-39
  • sentry_sdk/integrations/huey.py:293-293

Identified by Warden · code-review · BY4-E5D

Comment on lines +63 to +71
_run_periodic_tasks(huey)

check_ins = _check_ins(envelopes)
assert [c["status"] for c in check_ins] == ["in_progress", "error"]
assert {c["monitor_slug"] for c in check_ins} == {"nightly"}

events = [envelope.get_event() for envelope in envelopes]
(error,) = [e for e in events if e is not None and e.get("exception")]
assert error["exception"]["values"][0]["type"] == "ValueError"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error-path test lets Huey's re-raised exception escape

test_periodic_task_error calls _run_periodic_tasks() without handling the task's ValueError. Huey's execution path re-raises task failures, so the test exits before checking the error check-in or captured event. Catch the expected ValueError (for example, with pytest.raises) around the helper call, then run the assertions.

Evidence
  • test_periodic_task_error defines failing() to raise ValueError and calls _run_periodic_tasks(huey) directly before its assertions.
  • _run_periodic_tasks() calls huey.execute(task) without catching exceptions.
  • _wrap_task_execute() in sentry_sdk/integrations/huey.py captures task exceptions and re-raises them; the existing execute_huey_task() test helper also catches exceptions around huey.execute(task) for failing tasks.
  • The monitor decorator records the error check-in but does not suppress the exception, so the call aborts before the new test's check-in and event assertions.

Identified by Warden · code-review · BCZ-G9S

Comment on lines +307 to +319
def _get_crontab_field(values: "List[int]", first: int, last: int) -> str:
full = list(range(first, last + 1))
if values == full:
return "*"
if len(values) > 2:
step = values[1] - values[0]
if values == list(range(values[0], values[-1] + 1, step)):
if step == 1:
return "%d-%d" % (values[0], values[-1])
if values == full[::step]:
return "*/%d" % step
return "%d-%d/%d" % (values[0], values[-1], step)
return ",".join(str(v) for v in values)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huey cron settings sets break periodic-task monitor configuration

Huey stores cron fields as sets, but _get_crontab_field treats them as ordered lists. For normal fields with more than two values, indexing a set raises TypeError; the internal-exception guard then prevents the monitor wrapper from being applied, so the periodic task runs without check-ins. Sort the values before comparing or indexing them.

Evidence
  • _get_monitor_config() passes Huey’s cron_settings fields directly to _get_crontab_field(); only weekdays are sorted first.
  • _get_crontab_field() indexes values[0] and values[1] when the field has more than two values, which fails when Huey supplies a set.
  • patch_execute() calls _monitor_periodic_task() inside capture_internal_exceptions(); if config generation fails, the monitor decorator is not applied and the task still executes without check-ins.

Identified by Warden · find-bugs · X55-2A3

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant