Skip to content

feat: add --generic-hostname-worker-task-metric flag - #388

Open
bafulton wants to merge 1 commit into
danihodovic:masterfrom
semgrep:feat/generic-hostname-worker-task-metric
Open

feat: add --generic-hostname-worker-task-metric flag#388
bafulton wants to merge 1 commit into
danihodovic:masterfrom
semgrep:feat/generic-hostname-worker-task-metric

Conversation

@bafulton

Copy link
Copy Markdown

Problem

The executing worker's hostname label drives task-metric cardinality on autoscaled deployments: celery_task_runtime and the celery_task_* counters are all labeled with the worker's hostname, which in Kubernetes/KEDA setups is an ephemeral pod name. The name × hostname × queue set therefore churns and grows with the worker fleet. --generic-hostname-task-sent-metric solves this on the client side, but there is no equivalent for the worker-executed metrics.

Solution

Add --generic-hostname-worker-task-metric, mirroring the task-sent flag: it labels the celery_task_* counters and celery_task_runtime with a generic hostname instead of the worker's. track_task_event builds a single labels dict that feeds every counter and the runtime observation, so one gated assignment collapses them together. celery_task_sent keeps its own flag so the two compose, and celery_worker_up / celery_worker_tasks_active are untouched since they do not pass through track_task_event.

The test mirrors the existing generic_hostname_task_sent_metric test and passes on the memory, redis, and rabbitmq brokers.

Refs #387

Mirror --generic-hostname-task-sent-metric for the worker-executed metrics:
label the celery_task_* counters and celery_task_runtime with a generic
hostname instead of the executing worker's, to bound label cardinality when
workers are autoscaled (for example Kubernetes/KEDA pods). track_task_event
builds a single labels dict that feeds every counter and the runtime
observation, so one gated assignment collapses them together. celery_task_sent
keeps its own flag, and celery_worker_up / celery_worker_tasks_active are
unaffected since they do not pass through track_task_event.

Refs danihodovic#387
@bafulton
bafulton force-pushed the feat/generic-hostname-worker-task-metric branch from 837eea6 to 6e5e924 Compare July 24, 2026 15:45
@bafulton
bafulton marked this pull request as ready for review July 24, 2026 15:47
@bafulton

Copy link
Copy Markdown
Author

@danihodovic, I'd love to get your thoughts when you have a moment! I'm currently running into cardinality issues in my prod environment; Celery Exporter is generating more than 90,000 independent metrics and it's breaking prometheus and datadog scrapes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant