fix(beat): make it dependant from API service by jfagoagas · Pull Request #10603 · prowler-cloud/prowler

jfagoagas · 2026-04-07T16:38:19Z

Context

Relates to #10179

On a cold start, worker-beat boots in parallel with api and queries django_celery_beat_* tables before api finishes running migrations. This produces a psycopg2.errors.UndefinedTable / django.db.utils.ProgrammingError traceback in worker-beat and a matching ERROR in postgres logs on every fresh deployment. The current mitigation in api/docker-entrypoint.sh is sleep 15 inside start_worker_beat() — a timing hack that fails on slower hosts (CI runners, cold image pulls, busy disks) where migrations exceed 15 seconds. Reproduced on cold start: tables created at T+18s, worker-beat queried DB at T+15s in the original logs.

Description

Replace the sleep with a real readiness signal: worker-beat (and worker) now wait on an api healthcheck that only goes green once gunicorn binds, which the entrypoint guarantees happens AFTER apply_migrations completes.

Checklist

Community Checklist

This feature/issue is listed in here or roadmap.prowler.com
Is it assigned to me, if not, request it via the issue/feature in here or Prowler Community Slack

Review if the code is being covered by tests.
Review if code is being documented following this specification https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings
Review if backport is needed.
Review if is needed to change the Readme.md
Ensure new entries are added to CHANGELOG.md, if applicable.

SDK/CLI

Are there new checks included in this PR? Yes / No
- If so, do we need to update permissions for the provider? Please review this carefully.

UI

All issue/task requirements work as expected on the UI
Screenshots/Video of the functionality flow (if applicable) - Mobile (X < 640px)
Screenshots/Video of the functionality flow (if applicable) - Table (640px > X < 1024px)
Screenshots/Video of the functionality flow (if applicable) - Desktop (X > 1024px)
Ensure new entries are added to CHANGELOG.md, if applicable.

API

All issue/task requirements work as expected on the API
Endpoint response output (if applicable)
EXPLAIN ANALYZE output for new/modified queries or indexes (if applicable)
Performance test results (if applicable)
Any other relevant evidence of the implementation (if applicable)
Verify if API specs need to be regenerated.
Check if version updates are required (e.g., specs, Poetry, etc.).
Ensure new entries are added to CHANGELOG.md, if applicable.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

github-actions · 2026-04-07T16:38:48Z

✅ Conflict Markers Resolved

All conflict markers have been successfully resolved in this pull request.

github-actions · 2026-04-07T16:38:49Z

✅ All necessary CHANGELOG.md files have been updated.

github-actions · 2026-04-07T16:47:10Z

🔒 Container Security Scan

Image: prowler-api:ff2cc87
Last scan: 2026-04-10 14:31:25 UTC

📊 Vulnerability Summary

Severity	Count
🔴 Critical	5
Total	5

4 package(s) affected

⚠️ Action Required

Critical severity vulnerabilities detected. These should be addressed before merging:

Review the detailed scan results
Update affected packages to patched versions
Consider using a different base image if updates are unavailable

📋 Resources:

Download full report (see artifacts)
View in Security tab
Scanned with Trivy

codecov · 2026-04-07T16:52:10Z

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.60%. Comparing base (e4b2950) to head (6718429).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #10603   +/-   ##
=======================================
  Coverage   93.60%   93.60%           
=======================================
  Files         227      227           
  Lines       31906    31906           
=======================================
  Hits        29867    29867           
  Misses       2039     2039

Flag	Coverage Δ
api	`93.60% <0.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
prowler	`∅ <ø> (∅)`
api	`93.60% <0.00%> (ø)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

jfagoagas · 2026-04-08T07:38:57Z

docker-compose.yml

      neo4j:
        condition: service_healthy
+    healthcheck:
+      test: ["CMD-SHELL", "wget -q -O /dev/null http://localhost:${DJANGO_PORT:-8080}/api/v1/ || exit 1"]


Review localhost

Changed to 127.0.0.1. localhost can resolve to ::1 (IPv6) in some containers and if gunicorn only listens on IPv4 the healthcheck would fail. Also matches the mcp-server healthcheck.

jfagoagas · 2026-04-08T07:39:02Z

docker-compose-dev.yml

      neo4j:
        condition: service_healthy
+    healthcheck:
+      test: ["CMD-SHELL", "wget -q -O /dev/null http://localhost:${DJANGO_PORT:-8080}/api/v1/ || exit 1"]


Review localhost

Changed to 127.0.0.1. localhost can resolve to ::1 (IPv6) in some containers and if gunicorn only listens on IPv4 the healthcheck would fail. Also matches the mcp-server healthcheck.

Copilot

Pull request overview

This PR aims to eliminate cold-start race conditions where worker-beat (and worker) query django_celery_beat_* tables before the API finishes running migrations, replacing a fixed sleep delay with a Compose-level readiness dependency on the API container healthcheck.

Changes:

Add an api/api-dev container healthcheck that probes /api/v1/.
Update worker and worker-beat (and dev equivalents) to depends_on the API service being healthy.
Remove the hardcoded sleep 15 from start_worker_beat() in the API image entrypoint.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
docker-compose.yml	Adds API healthcheck and gates `worker`/`worker-beat` startup on API health.
docker-compose-dev.yml	Same as production compose, for local dev stack.
api/docker-entrypoint.sh	Removes the `sleep` timing hack before starting Celery beat.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

api/docker-entrypoint.sh

docker-compose.yml

docker-compose-dev.yml

…rom workers, add changelog

josema-xyz · 2026-04-10T08:48:16Z

Removed the direct valkey/postgres dependencies from worker and worker-beat since they're now transitive through api.

…ners in Helm worker deployments

josema-xyz · 2026-04-10T08:57:28Z

Aligned gunicorn default port to 8080 (guniconf.py) to match compose, and added initContainers support to worker and worker-beat Helm deployments so K8s operators can configure an init container that waits for the API before starting.

…at-api-docker-dependency

fix(beat): make it dependant from API service

0571c26

github-actions bot added the component/api label Apr 7, 2026

jfagoagas commented Apr 8, 2026

View reviewed changes

jfagoagas marked this pull request as ready for review April 10, 2026 08:32

jfagoagas requested review from a team as code owners April 10, 2026 08:32

Copilot AI review requested due to automatic review settings April 10, 2026 08:32

Copilot started reviewing on behalf of jfagoagas April 10, 2026 08:33 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

api/docker-entrypoint.sh Show resolved Hide resolved

docker-compose.yml Show resolved Hide resolved

docker-compose-dev.yml Show resolved Hide resolved

fix(docker): use 127.0.0.1 in API healthcheck, drop transitive deps f…

8b362b4

…rom workers, add changelog

fix(api): align gunicorn default port to 8080 and support init contai…

62ec243

…ners in Helm worker deployments

josema-xyz requested a review from a team as a code owner April 10, 2026 08:56

josema-xyz added 2 commits April 10, 2026 11:38

Merge branch 'master' of github.com:prowler-cloud/prowler into fix-be…

ada2341

…at-api-docker-dependency

fix(api): fix bind-mount permissions for Linux Docker hosts (#10646)

6718429

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(beat): make it dependant from API service#10603

fix(beat): make it dependant from API service#10603
jfagoagas wants to merge 5 commits intomasterfrom
fix-beat-api-docker-dependency

jfagoagas commented Apr 7, 2026

Uh oh!

github-actions bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

codecov bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

jfagoagas Apr 8, 2026

Uh oh!

josema-xyz Apr 10, 2026

Uh oh!

jfagoagas Apr 8, 2026

Uh oh!

josema-xyz Apr 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

josema-xyz commented Apr 10, 2026

Uh oh!

josema-xyz commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jfagoagas commented Apr 7, 2026

Context

Description

Checklist

SDK/CLI

UI

API

License

Uh oh!

github-actions bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Container Security Scan

📊 Vulnerability Summary

⚠️ Action Required

Uh oh!

codecov bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jfagoagas Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

josema-xyz Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

jfagoagas Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

josema-xyz Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

josema-xyz commented Apr 10, 2026

Uh oh!

josema-xyz commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Apr 7, 2026 •

edited

Loading

github-actions bot commented Apr 7, 2026 •

edited

Loading

github-actions bot commented Apr 7, 2026 •

edited

Loading

codecov bot commented Apr 7, 2026 •

edited

Loading