Summary
The pipeline_manager attempts to load the yolo_mask pipeline during session initialisation, but fails with Invalid pipeline ID: yolo_mask because the scope_yolo_mask plugin has not yet been installed on the worker at that point. The pipeline loads successfully on the next attempt (after plugin install completes), but the transient ERROR log is misleading.
This is a distinct class of issue from #843 (turbo4mac/bloom, which are truly invalid IDs that never resolve). Here, the pipeline ID is valid — the plugin just isn't installed yet.
cc @mjh1 @emranemran
Error Messages
2026-04-13 23:38:36,488 - scope.server.pipeline_manager - ERROR - Failed to load pipeline yolo_mask: Invalid pipeline ID: yolo_mask. If this error persists, consider removing the models directory '/data/models' and re-downloading models.
2026-04-13 23:38:36,488 - scope.server.pipeline_manager - ERROR - Failed to load pipeline: yolo_mask
Timeline (2026-04-13 ~23:36–23:40 UTC)
23:38:36 - ❌ pipeline_manager: Failed to load pipeline yolo_mask: Invalid pipeline ID
(plugin not yet installed — registry doesn't know about yolo_mask yet)
23:38:36 - pipeline_manager loads 2 pipelines: [('yolo_mask', 'yolo_mask'), ('longlive', 'longlive')]
23:38:53 - scope.server.app: Installing plugin: git+https://github.com/daydreamlive/scope_yolo_mask
23:38:54 - scope_yolo_mask plugin installed successfully
23:38:54 - scope.core.plugins.manager: Registered plugin pipeline: yolo_mask
23:39:10 - scope_yolo_mask.plugin: Registered YOLO mask pipeline (YOLO26-seg)
23:39:20 - ✅ pipeline_manager: Pipeline yolo_mask loaded successfully (second attempt, after plugin install)
Job ID: ea7914a1-6a2f-4472-ad7b-e6fc0983cde2
App: no fal_app_name (grpc-run session, scope-livepeer staging)
Frequency (last 12h): 1 session
Root Cause
pipeline_manager starts the pipeline loading phase concurrently with (or before) plugin installation. On first startup, if a required plugin (scope_yolo_mask) is not bundled and needs to be installed from a git source, the pipeline load races the install. The pipeline registry doesn't contain yolo_mask until after the install and registration steps complete, so the first load attempt returns Invalid pipeline ID.
Impact
Suggested Fix
Ensure the plugin installation phase (and registry update) completes before pipeline_manager begins its initial pipeline load pass. Or: distinguish "pipeline ID not in registry because plugin not yet installed" from "pipeline ID permanently invalid" and log at WARN rather than ERROR for the former case.
Summary
The
pipeline_managerattempts to load theyolo_maskpipeline during session initialisation, but fails withInvalid pipeline ID: yolo_maskbecause thescope_yolo_maskplugin has not yet been installed on the worker at that point. The pipeline loads successfully on the next attempt (after plugin install completes), but the transient ERROR log is misleading.This is a distinct class of issue from #843 (turbo4mac/bloom, which are truly invalid IDs that never resolve). Here, the pipeline ID is valid — the plugin just isn't installed yet.
cc @mjh1 @emranemran
Error Messages
Timeline (2026-04-13 ~23:36–23:40 UTC)
Job ID:
ea7914a1-6a2f-4472-ad7b-e6fc0983cde2App: no
fal_app_name(grpc-run session, scope-livepeer staging)Frequency (last 12h): 1 session
Root Cause
pipeline_managerstarts the pipeline loading phase concurrently with (or before) plugin installation. On first startup, if a required plugin (scope_yolo_mask) is not bundled and needs to be installed from a git source, the pipeline load races the install. The pipeline registry doesn't containyolo_maskuntil after the install and registration steps complete, so the first load attempt returnsInvalid pipeline ID.Impact
Suggested Fix
Ensure the plugin installation phase (and registry update) completes before
pipeline_managerbegins its initial pipeline load pass. Or: distinguish "pipeline ID not in registry because plugin not yet installed" from "pipeline ID permanently invalid" and log at WARN rather than ERROR for the former case.