From aafb410ec9251769e34cbdd0bbd553c53e2a5a9c Mon Sep 17 00:00:00 2001 From: Simon Kelly Date: Thu, 30 Jul 2026 17:07:55 +0200 Subject: [PATCH 1/2] Document heartbeat_interval for Celery and Procrastinate Co-Authored-By: Claude Opus 5 (1M context) --- docs/data_model.md | 1 + docs/python-celery.md | 51 +++++++++++++++++++++++++++++++++++ docs/python-procrastinate.md | 52 ++++++++++++++++++++++++++++++++++++ 3 files changed, 104 insertions(+) diff --git a/docs/data_model.md b/docs/data_model.md index 45a6c1b..25318a0 100644 --- a/docs/data_model.md +++ b/docs/data_model.md @@ -54,6 +54,7 @@ The main attributes or a task are: : This value can be used in conjunction with [actions](actions.md) and monitors to trigger alerts if a task exceeds its expected runtime. The value is in seconds. + `stale_timeout` : This represents the maximum number of seconds allowed between task updates. If a task does not receive diff --git a/docs/python-celery.md b/docs/python-celery.md index f8b6fcf..1e81644 100644 --- a/docs/python-celery.md +++ b/docs/python-celery.md @@ -41,6 +41,15 @@ The `CelerySystemIntegration` class takes a number of optional parameters: ==Since v1.4.0== +- `heartbeat_interval`: Seconds between automatic task updates while a task is running. See + [Keeping Long-Running Tasks Fresh](#keeping-long-running-tasks-fresh). + + ==Since v2.4.0== + +- `stale_timeout`: The [`stale_timeout`](data_model.md#stale_timeout) to set on tracked tasks. + + ==Since v2.4.0== + Exclusions take precedence over inclusions so if a task name matches both an include and an exclude, it will be excluded. @@ -176,6 +185,48 @@ def my_task(self, items): creating the task, or the task is being run synchronously e.g. via `.apply()` or calling the task using `.map` or `.starmap`, `.chunk`. +## Keeping Long-Running Tasks Fresh + +==Since v2.4.0== + +A task with a [`stale_timeout`](data_model.md#stale_timeout) is marked `stale` if it goes too long +without an update, so a long-running task that doesn't report progress will trip the timeout while it +is perfectly healthy. Setting `heartbeat_interval` (seconds) makes the worker update the task for you +while it runs, instead of having to do it from the task body. + +The interval can be set on the system integration, on the task, or per call: + +```python +# for all tracked tasks +taskbadger.init( + token="YOUR_API_KEY", + systems=[CelerySystemIntegration(heartbeat_interval=60)], +) + +# on the task +@app.task(base=Task, taskbadger_heartbeat_interval=60) +def my_task(): + ... + +# per call +my_task.apply_async(taskbadger_heartbeat_interval=60) +``` + +Unless `stale_timeout` is given explicitly it is set to twice the interval, so each of the examples +above creates the task with a `stale_timeout` of 120 seconds. Pass both to control it: + +```python +my_task.apply_async(taskbadger_heartbeat_interval=60, taskbadger_stale_timeout=300) +``` + +As with the other options, values set on the task or on `apply_async` take precedence over the values +set on `CelerySystemIntegration`. + +!!! note + + All running tasks are updated from a single background thread per worker process, started the + first time a task with a heartbeat runs. Updates stop when the task finishes. + ## Canvas primitives (map / starmap / chunks) As of `v1.6.3`, Task Badger now tracks tasks created via Celery canvas primitives: `map`, `starmap`, and `chunks`. Previously these were executed as built-in `celery.map` / `celery.starmap` tasks and were filtered out; TaskBadger now creates task records for the *inner* tasks produced by these primitives. diff --git a/docs/python-procrastinate.md b/docs/python-procrastinate.md index 92f12d3..2cfc3ac 100644 --- a/docs/python-procrastinate.md +++ b/docs/python-procrastinate.md @@ -51,6 +51,10 @@ The `ProcrastinateSystemIntegration` class takes the following parameters: the patterns will not be tracked. - `record_task_args`: If `True`, the job's keyword arguments will be recorded in the Task Badger task data under `procrastinate_task_kwargs`. +- `heartbeat_interval`: Seconds between automatic task updates while a task is running. See + [Keeping Long-Running Tasks Fresh](#keeping-long-running-tasks-fresh). ==Since v2.4.0== +- `stale_timeout`: The [`stale_timeout`](data_model.md#stale_timeout) to set on tracked tasks. + ==Since v2.4.0== Patterns are matched against the full task name using `re.fullmatch`. Exclusions take precedence over inclusions, so if a task name matches both an include and an exclude, it will be excluded. @@ -112,6 +116,13 @@ task is created: - `record_task_args`: If `True`, the job's keyword arguments are recorded under `data["procrastinate_task_kwargs"]`. Defaults to inheriting the value from the `ProcrastinateSystemIntegration` if one is configured, otherwise `False`. +- `heartbeat_interval`: Seconds between automatic task updates while the task is running. See + [Keeping Long-Running Tasks Fresh](#keeping-long-running-tasks-fresh). ==Since v2.4.0== +- `stale_timeout`: The [`stale_timeout`](data_model.md#stale_timeout) to set on the task. + ==Since v2.4.0== + +`record_task_args`, `heartbeat_interval` and `stale_timeout` are inherited from the +`ProcrastinateSystemIntegration` when they are not set on the decorator. ```python @track(name="report", value_max=100, tags={"env": "prod"}, record_task_args=True) @@ -144,6 +155,47 @@ async def report(rows): `current_task()` returns `None` outside of a tracked job, if Task Badger has not been [configured](python.md#configure), or if the task could not be fetched. +## Keeping Long-Running Tasks Fresh + +==Since v2.4.0== + +A task with a [`stale_timeout`](data_model.md#stale_timeout) is marked `stale` if it goes too long +without an update, so a long-running task that doesn't report progress will trip the timeout while it +is perfectly healthy. Setting `heartbeat_interval` (seconds) makes the worker update the task for you +while it runs, instead of having to do it from the job body. + +The interval can be set on the task or on the system integration: + +```python +# on the task +@track(heartbeat_interval=60) +@app.task +async def slow_job(): + ... + + +# for all tracked tasks +taskbadger.init( + token="YOUR_API_KEY", + systems=[ProcrastinateSystemIntegration(app=app, heartbeat_interval=60)], +) +``` + +Unless `stale_timeout` is given explicitly it is set to twice the interval, so both of the examples +above create tasks with a `stale_timeout` of 120 seconds. Pass both to control it: + +```python +@track(heartbeat_interval=60, stale_timeout=300) +@app.task +async def slow_job(): + ... +``` + +!!! note + + All running tasks are updated from a single background thread per worker process, started the + first time a task with a heartbeat runs. Updates stop when the task finishes. + ## Periodic Tasks Periodic tasks scheduled with `@app.periodic` are tracked as well. Each periodic deferral creates a new From 60d1cf06baf2a8391b625f99e3ca2c3205af6099 Mon Sep 17 00:00:00 2001 From: Simon Kelly Date: Thu, 30 Jul 2026 17:07:55 +0200 Subject: [PATCH 2/2] changelog for 2.4.0 Co-Authored-By: Claude Opus 5 (1M context) --- docs/changelog.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/changelog.md b/docs/changelog.md index f373483..0108a5f 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -7,6 +7,15 @@ hide: Full release notes for the Python SDK are available on [GitHub](https://github.com/taskbadger/taskbadger-python/releases). +## v2.4.0 + +**2026-07-30** + +**Python SDK** + +* **NEW** `heartbeat_interval` option for the [Celery](python-celery.md#keeping-long-running-tasks-fresh) and [Procrastinate](python-procrastinate.md#keeping-long-running-tasks-fresh) integrations. The worker updates running tasks for you so that long-running tasks don't go [`stale`](data_model.md#stale_timeout). +* **FIX** An eager Celery task no longer closes a Task Badger session opened by its caller. + ## v2.3.1 **2026-07-27**