Skip to content

Commit 730d0d0

Browse files
author
szachovy
committed
Fix SQL_QUERY_MUTATOR=None in Celery worker and downgrade mysql-connector-python
Two fixes: 1. SQL_QUERY_MUTATOR=None: superset.sql_lab captures SQL_QUERY_MUTATOR at module import time from current_app.config. In Celery's prefork model, import_default_modules() runs in child processes before worker_process_init (which provides the app context), so the capture can result in None. Fix: add superset_celery_app.py wrapper that explicitly sets sql_lab.SQL_QUERY_MUTATOR = flask_app.config["SQL_QUERY_MUTATOR"] after create_app() completes but before any worker processes are forked. 2. Downgrade mysql-connector-python from 9.0.0 to 8.4.0: version 9.0.0 was removed from PyPI; 8.4.0 is the latest stable 8.x release and is better aligned with SQLAlchemy 1.4 which Superset 4.0.2 targets.
1 parent 1e662c3 commit 730d0d0

3 files changed

Lines changed: 22 additions & 3 deletions

File tree

services/superset/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ RUN \
1919
install \
2020
--no-cache-dir \
2121
"redis==4.5.4" \
22-
"mysql-connector-python==9.0.0"
22+
"mysql-connector-python==8.4.0"
2323

2424
USER root
2525

services/superset/entrypoint.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@ if superset test_db \
1919
/app/set_database_uri.exp
2020
/usr/bin/run-server.sh &
2121

22-
celery \
23-
--app superset.tasks.celery_app:app worker \
22+
PYTHONPATH="/app${PYTHONPATH:+:${PYTHONPATH}}" celery \
23+
--app superset_celery_app:app worker \
2424
--pool prefork \
2525
--concurrency 4 \
2626
-O fair &
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
"""
2+
Custom Celery app wrapper for Superset.
3+
4+
Guarantees that superset.sql_lab.SQL_QUERY_MUTATOR is set to the correct
5+
function from the Flask app config before the Celery prefork pool forks
6+
worker processes.
7+
8+
Background: superset.sql_lab captures SQL_QUERY_MUTATOR at module import
9+
time via config["SQL_QUERY_MUTATOR"] (where config = current_app.config).
10+
In some timing scenarios with the prefork pool, this capture can result in
11+
None, causing 'NoneType' object is not callable during async SQL execution.
12+
"""
13+
# Import from the standard Superset celery entrypoint (runs create_app())
14+
from superset.tasks.celery_app import app, flask_app # noqa: F401
15+
16+
# Explicitly ensure superset.sql_lab has the correct SQL_QUERY_MUTATOR
17+
# before any worker processes are forked.
18+
import superset.sql_lab as _sql_lab # noqa: E402
19+
_sql_lab.SQL_QUERY_MUTATOR = flask_app.config["SQL_QUERY_MUTATOR"]

0 commit comments

Comments
 (0)