Skip to content

feat(lfx): add capability routing pluggability - #13617

Open
ogabrielluiz wants to merge 6 commits into
feat/executor-seam-stepflow-mvpfrom
feat/mt-pluggability-core
Open

feat(lfx): add capability routing pluggability#13617
ogabrielluiz wants to merge 6 commits into
feat/executor-seam-stepflow-mvpfrom
feat/mt-pluggability-core

Conversation

@ogabrielluiz

@ogabrielluiz ogabrielluiz commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Stacks on #13063. Adds the Langflow-side capability-routing contract needed for pluggable execution policy.

This keeps the scope intentionally narrow: Langflow can ask a pluggable capability provider how to route a run and which opaque per-run metadata to attach, while the provider implementation remains outside this PR.

What changed

  • Adds lfx.services.capability:
    • CapabilityService
    • provider/classifier/resolver protocols
    • noop/default implementations
    • entry-point installation hook
  • Adds ServiceType.CAPABILITY_SERVICE and service dependency wiring.
  • Lets ExecutorService create a Coordinator with a capability service.
  • Extends Coordinator to request a routing decision before dispatch.
  • Adds Unit.executor_kind so routing can select a non-default executor per unit.
  • Adds generic opaque per-run metadata when a provider is active.

Scope boundary

This PR is pluggability only. It defines the contract and default noop behavior, but it does not ship a concrete provider implementation.

The default provider is noop, so behavior remains unchanged unless a capability provider is installed/configured.

How this fits the stack

Verification

  • Targeted Ruff passed.
  • Targeted unit suite passed:
    • src/lfx/tests/unit/services/capability/test_capability_service.py
    • src/lfx/tests/unit/execution/test_capability_routing.py
    • existing executor/coordinator routing tests
  • Service-manager probe passed.

Review focus

  • Is the capability contract generic enough for external providers?
  • Does the default noop path preserve existing behavior?
  • Is the opaque metadata handoff acceptable for this boundary?
  • Is the scope tight enough for a contract-only extension point?

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)
  • release-.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d2ad5d1d-5c7f-42b2-91d1-89e671c5769e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/mt-pluggability-core

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 10, 2026
@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Frontend Unit Test Coverage Report

Coverage Summary

Lines Statements Branches Functions
Coverage: 43%
43.39% (57785/133148) 69.06% (7842/11355) 41.64% (1299/3119)

Unit Test Results

Tests Skipped Failures Errors Time
4945 0 💤 0 ❌ 0 🔥 13m 31s ⏱️

@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.90155% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.67%. Comparing base (1340f30) to head (76e0398).

Files with missing lines Patch % Lines
src/lfx/src/lfx/services/capability/service.py 75.30% 18 Missing and 2 partials ⚠️
src/lfx/src/lfx/services/capability/protocols.py 87.17% 0 Missing and 5 partials ⚠️
src/lfx/src/lfx/execution/coordinator.py 86.66% 2 Missing and 2 partials ⚠️
src/lfx/src/lfx/services/capability/defaults.py 88.88% 2 Missing ⚠️
src/lfx/src/lfx/services/deps.py 33.33% 2 Missing ⚠️

❌ Your project check has failed because the head coverage (54.73%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@                         Coverage Diff                         @@
##           feat/executor-seam-stepflow-mvp   #13617      +/-   ##
===================================================================
- Coverage                            58.84%   58.67%   -0.18%     
===================================================================
  Files                                 2320     2317       -3     
  Lines                               221132   219749    -1383     
  Branches                             34351    32992    -1359     
===================================================================
- Hits                                130121   128932    -1189     
+ Misses                               89494    89349     -145     
+ Partials                              1517     1468      -49     
Flag Coverage Δ
backend 65.53% <100.00%> (+0.15%) ⬆️
frontend 57.91% <ø> (+0.13%) ⬆️
lfx 54.73% <82.72%> (-1.79%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/backend/base/langflow/services/utils.py 84.61% <100.00%> (+0.78%) ⬆️
src/lfx/src/lfx/execution/types.py 100.00% <100.00%> (ø)
src/lfx/src/lfx/services/capability/factory.py 100.00% <100.00%> (ø)
src/lfx/src/lfx/services/executor/factory.py 100.00% <100.00%> (ø)
src/lfx/src/lfx/services/executor/service.py 95.38% <100.00%> (+0.07%) ⬆️
src/lfx/src/lfx/services/schema.py 100.00% <100.00%> (ø)
src/lfx/src/lfx/services/capability/defaults.py 88.88% <88.88%> (ø)
src/lfx/src/lfx/services/deps.py 66.97% <33.33%> (-0.96%) ⬇️
src/lfx/src/lfx/execution/coordinator.py 88.33% <86.66%> (-2.30%) ⬇️
src/lfx/src/lfx/services/capability/protocols.py 87.17% <87.17%> (ø)
... and 1 more

... and 281 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 10, 2026

@erichare erichare left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [P1] Reserved capability metadata can be spoofed when mint() returns no token.
    src/lfx/src/lfx/execution/coordinator.py:49-54 merges caller/runtime options with decision.runtime_options, but src/lfx/src/lfx/services/capability/service.py:115-120 only writes lfx_capability_token when the provider returns a truthy token. A caller-supplied lfx_capability_token can therefore survive into the selected executor. Strip reserved lfx_* capability keys before partitioning/merging, or make the routing decision the sole source of those fields and add a test where an active provider returns None.

  • [P2] Untrusted classification falls back to the default executor when no untrusted executor is configured.
    src/lfx/src/lfx/services/capability/service.py:103-106 leaves executor_kind as the default if Trust.UNTRUSTED is returned but _untrusted_executor_kind is None. That is a risky fail-open mode for an isolation/routing boundary. I’d fail closed with a clear error unless there is an explicit “allow untrusted on default” opt-in, and cover that misconfiguration with a unit test.

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 12, 2026
@ogabrielluiz
ogabrielluiz force-pushed the feat/executor-seam-stepflow-mvp branch from fabcc03 to 8890e9d Compare June 18, 2026 18:49
@ogabrielluiz
ogabrielluiz force-pushed the feat/mt-pluggability-core branch from f9cc37e to 2789ac0 Compare June 18, 2026 19:53
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 18, 2026
@github-actions

github-actions Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

✅ Test Coverage Advisor

No source changes detected without accompanying tests. Thanks for keeping coverage up! 🎉

Advisory check only — never blocks merge.

@ogabrielluiz
ogabrielluiz force-pushed the feat/executor-seam-stepflow-mvp branch from 8890e9d to 7ab1e8b Compare June 18, 2026 20:12
@ogabrielluiz
ogabrielluiz force-pushed the feat/mt-pluggability-core branch from 2789ac0 to d82f246 Compare June 18, 2026 20:12
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 18, 2026
@carlosrcoelho
carlosrcoelho requested a review from erichare June 19, 2026 20:35

@erichare erichare left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ogabrielluiz this looks great, just two things:

[P1] Active capability routing breaks Graph.arun() on the in-process legacy path.
src/lfx/src/lfx/services/capability/service.py:121-123 adds lfx_tenant_id / lfx_trust to every active routing decision, even when the decision keeps the default in-process executor. The in-process legacy path then forwards every non-underscore runtime option into _arun_legacy at src/lfx/src/lfx/execution/backends/in_process.py:24-26, but Graph._arun_legacy does not accept those new keywords (src/lfx/src/lfx/graph/graph/base.py:897-908). So enabling any non-passthrough capability provider can make existing Graph.arun() calls fail with TypeError: ... unexpected keyword argument 'lfx_tenant_id'. I reproduced this with a minimal PR-head snippet.
Suggested fix: filter capability-only keys before calling _arun_legacy (including lfx_capability_token, lfx_tenant_id, lfx_trust, and probably request-only scope keys), and add a regression using real InProcessExecutor + active CapabilityService + _use_arun_legacy=True.

[P2] Custom primitives that subclass the defaults are silently treated as passthrough.
CapabilityService.is_passthrough uses isinstance(...) for NoopCapabilityProvider, AllTrustedClassifier, and SingleTenantResolver at src/lfx/src/lfx/services/capability/service.py:76-80. If a plugin subclasses one of those defaults to reuse behavior while overriding mint() or trust classification, the coordinator sees is_passthrough == True and skips routing entirely at src/lfx/src/lfx/execution/coordinator.py:83.
Suggested fix: use exact type checks or maintain an explicit “custom primitive installed” flag in install().

@ogabrielluiz
ogabrielluiz force-pushed the feat/mt-pluggability-core branch from d82f246 to dfb455b Compare June 22, 2026 13:06
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 22, 2026
@ogabrielluiz
ogabrielluiz requested a review from erichare June 22, 2026 14:54

@erichare erichare left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates @ogabrielluiz — the two findings from my first pass are resolved:

  • Spoofed capability metadatarun() strips reserved keys up front (coordinator.py:81) and re-introduces them only from decision.runtime_options (coordinator.py:98-99); the "provider returns no token" case has a test.
  • Untrusted fail-openservice.py:115-117 now raises when Trust.UNTRUSTED has no untrusted_executor_kind configured, with a fail-closed test.

Two items from the second round are still open:

[P1] lfx_tenant_id / lfx_trust still break Graph.arun()'s legacy path.
route() injects both keys whenever not is_passthrough (service.py:121-123), even when the decision keeps the default in-process executor (e.g. a trusted flow under a custom resolver). The coordinator merges decision.runtime_options back into the unit (coordinator.py:98-99), and InProcessExecutor forwards every non-_ key into graph._arun_legacy(**legacy_kwargs) (in_process.py:25-26). _arun_legacy has no **kwargs (graph/graph/base.py:897), so the legacy path raises TypeError: unexpected keyword argument 'lfx_tenant_id'. Please strip the reserved lfx_* keys in in_process.py before the legacy call, and add a regression with a real InProcessExecutor + active CapabilityService + _use_arun_legacy=True.

[P2] Subclassed defaults are silently treated as passthrough.
is_passthrough uses isinstance(...) (service.py:77-79), so a plugin that subclasses NoopCapabilityProvider to override only mint() reads as passthrough and route() is never called (coordinator.py:83). Use exact type(x) is ... checks, or an explicit "custom primitive installed" flag set in install().

Nit: get_capability_service() in deps.py is missing the return annotation its siblings carry.

@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 23, 2026
@ogabrielluiz
ogabrielluiz force-pushed the feat/executor-seam-stepflow-mvp branch from 43a682b to ea33e98 Compare June 25, 2026 17:50
@ogabrielluiz
ogabrielluiz force-pushed the feat/mt-pluggability-core branch from 05b36b3 to cf5ffa1 Compare June 25, 2026 17:53
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 25, 2026
ogabrielluiz and others added 6 commits June 25, 2026 15:25
Coordinator.stream() no longer takes an inputs parameter (the streaming path
reads initial_inputs from runtime_options); update the capability routing
stream-close test to the current signature.
ExecutorService declares a hard dependency on CAPABILITY_SERVICE, but the
langflow backend's register_all_service_factories() registered
ExecutorServiceFactory without CapabilityServiceFactory. The service manager
could not resolve the dependency when building ExecutorService, so every flow
run failed with NoFactoryRegisteredError / "ExecutorService is not available".

lfx's own bootstrap (get_factories) auto-discovers the capability factory, so
lfx run/serve were unaffected; only the backend's hand-maintained factory list
had drifted. Add a regression test asserting the backend registers a factory
for every ExecutorService dependency.
@ogabrielluiz
ogabrielluiz force-pushed the feat/executor-seam-stepflow-mvp branch from ea33e98 to 1340f30 Compare June 25, 2026 18:25
@ogabrielluiz
ogabrielluiz force-pushed the feat/mt-pluggability-core branch from cf5ffa1 to 76e0398 Compare June 25, 2026 18:25
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request scaling-up

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants