Skip to content

Commit 02f3cb3

Browse files
author
Marko Petzold
committed
Merge branch 'proxy_principles'
2 parents aa008c0 + ae07eb0 commit 02f3cb3

34 files changed

Lines changed: 5640 additions & 433 deletions

‎.dockerignore‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Git
2+
.git
3+
.gitignore
4+
.gitattributes
5+
6+
# Python
7+
__pycache__
8+
*.py[cod]
9+
*$py.class
10+
*.so
11+
.Python
12+
*.egg-info
13+
dist
14+
build
15+
*.egg
16+
.pytest_cache
17+
.tox
18+
.coverage
19+
.coverage.*
20+
htmlcov
21+
*.log
22+
23+
# Virtual environments
24+
venv
25+
env
26+
ENV
27+
.venv
28+
29+
# IDE
30+
.vscode
31+
.idea
32+
*.swp
33+
*.swo
34+
*~
35+
.DS_Store
36+
37+
# Documentation
38+
docs/_build
39+
docs/_static
40+
docs/_templates
41+
*.md
42+
!README.md
43+
44+
# Testing
45+
test/
46+
.crossbar/
47+
*.log
48+
49+
# CI/CD
50+
.github
51+
.travis.yml
52+
.gitlab-ci.yml
53+
54+
# Development files
55+
Makefile
56+
tox.ini
57+
pytest.ini
58+
setup.cfg
59+
*.txt
60+
!requirements*.txt
61+
62+
# Temporary files
63+
tmp
64+
temp
65+
*.tmp
66+
*.bak

‎.github/copilot-instructions.md‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# Crossbar.io AI Coding Instructions
2+
3+
## Project Overview
4+
Crossbar.io is a decentralized WAMP (Web Application Messaging Protocol) router and application server. It implements real-time messaging patterns (PubSub, RPC) and serves as middleware for distributed/microservice applications.
5+
6+
## Architecture & Key Components
7+
8+
### Core Architecture
9+
- **Node Controller** (`crossbar/node/controller.py`): Main process manager that spawns and manages workers
10+
- **Workers**: Native processes that provide specific functionality:
11+
- **Router Workers** (`crossbar/worker/router.py`): Core WAMP routing (realms, transports, components)
12+
- **Proxy Workers** (`crossbar/worker/proxy.py`): Load balancing and connection proxying
13+
- **Container Workers**: Host application components
14+
- **Realms** (`crossbar/worker/types.py`): WAMP routing domains with roles/permissions
15+
- **Transports** (`crossbar/worker/transport.py`): Network protocols (WebSocket, RawSocket, Web/HTTP)
16+
17+
### Configuration-Driven Design
18+
All functionality is configured via JSON config files (`config.json`). The schema is in `crossbar.json`. Configuration drives:
19+
- Worker creation and management
20+
- Realm setup with authentication/authorization
21+
- Transport endpoint configuration
22+
- Component hosting and routing
23+
24+
### Worker Pattern
25+
```python
26+
# Workers inherit from WorkerController -> NativeProcess -> ApplicationSession
27+
class RouterController(TransportController):
28+
WORKER_TYPE = 'router'
29+
# Registers WAMP procedures for dynamic management
30+
@wamp.register(None)
31+
def start_router_realm(self, realm_id, realm_config, details=None):
32+
```
33+
34+
## Development Workflows
35+
36+
### Testing Strategy
37+
- **Unit Tests**: Use Twisted Trial (`trial crossbar.router.test.test_authorize`)
38+
- **Functional Tests**: pytest-based integration tests (`test/functests/`)
39+
- Run with: `pytest -sv test/functests/cbtests/`
40+
- Use `--slow` for long-running tests, `--keep` to preserve temp dirs
41+
- Tests spawn actual Crossbar processes and test end-to-end behavior
42+
- **Management Tests**: Live testing with `test/management/` scripts
43+
44+
### Build & Install Commands
45+
```bash
46+
# Development install with pinned dependencies
47+
make install
48+
49+
# Testing
50+
make test # Full test suite via tox
51+
pytest -sv test/functests/cbtests/test_cb_proxy.py # Specific functional test
52+
trial crossbar.router.test.test_authorize # Unit test
53+
54+
# Start node
55+
crossbar start --cbdir=/path/to/.crossbar
56+
```
57+
58+
### Key Files for Understanding
59+
- `crossbar/__init__.py`: Entry point, monkey patches, CLI setup
60+
- `crossbar/worker/router.py`: RouterController - main router logic
61+
- `crossbar/router/router.py`: Core Router and RouterFactory classes
62+
- `crossbar/interfaces.py`: Key interfaces (IPendingAuth, IRealmContainer)
63+
- `crossbar/node/controller.py`: Node management and worker orchestration
64+
65+
## Project-Specific Patterns
66+
67+
### WAMP Procedure Registration
68+
Controllers expose management APIs via WAMP decorators:
69+
```python
70+
@wamp.register(None)
71+
def start_router_transport(self, transport_id, config, create_paths=False, details=None):
72+
# Management API for starting transports
73+
```
74+
75+
### Monkey Patching Strategy
76+
Heavy use of compatibility monkey patches in `__init__.py` for eth_abi, web3, etc. This maintains compatibility across Python/library versions.
77+
78+
### Session Factories and Authentication
79+
- `RouterSessionFactory`: Creates WAMP sessions for realms
80+
- Authentication flows through `IPendingAuth` interface
81+
- Authorization via realm roles with URI-based permissions
82+
83+
### Error Handling Convention
84+
Crossbar raises `ApplicationError` with specific error URIs:
85+
```python
86+
raise ApplicationError("crossbar.error.invalid_configuration", emsg)
87+
```
88+
89+
### Configuration Validation
90+
Uses `crossbar.common.checkconfig` module. Configuration is validated on startup and runtime changes.
91+
92+
## Critical Integration Points
93+
94+
### Worker Communication
95+
- All workers connect to node controller via WAMP
96+
- Management operations use WAMP RPC calls between controller and workers
97+
- Event propagation for state changes (`.on_router_transport_started`)
98+
99+
### Transport Factory Creation
100+
Complex factory pattern in `crossbar/worker/transport.py`:
101+
- WebSocket: `WampWebSocketServerFactory`
102+
- RawSocket: `WampRawSocketServerFactory`
103+
- Web: Creates Twisted Web resources with WAMP backing
104+
105+
### Twisted Integration
106+
- Heavy use of `@inlineCallbacks` and `returnValue()`
107+
- Reactor management for worker processes
108+
- Deferred chains for async operations
109+
110+
## Testing Notes
111+
- Functional tests require careful cleanup - single Crossbar instance shared across tests
112+
- Use `start_crossbar()` helper from `test/functests/helpers.py`
113+
- Tests must use `pytest.inlineCallbacks`, not Twisted's version
114+
- Never import `twisted.internet.reactor` directly in tests
115+
116+
## Configuration Schema
117+
Reference `crossbar.json` for full configuration schema. Key patterns:
118+
- Workers array defines all processes
119+
- Each worker has type-specific configuration
120+
- Realms define WAMP routing domains
121+
- Transports define network endpoints
122+
- Components define hosted application code

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ vers/
4343
.DS_Store
4444
_build
4545
pip-wheel-metadata
46+
.venv
4647

4748
# vscode
4849
.vscode/*

‎BUGFIX-CALLER-DISCONNECT.md‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Bug Fix: Caller Disconnect Causing Callee Exception
2+
3+
## Problem Description
4+
5+
When a WAMP client (caller) calls a remote procedure and exits/disconnects before the call returns, the callee receives an exception and may also exit. This should not happen - the callee should be able to complete the procedure execution normally, and the router should simply discard the result since the caller is no longer available.
6+
7+
## Root Cause
8+
9+
The bug is in `/crossbar/router/dealer.py` in the `detach()` method (lines 185-247).
10+
11+
When a caller disconnects during an in-flight invocation:
12+
13+
1. The router checks if the callee supports call canceling (`call_canceling` feature)
14+
2. **If the callee does NOT support call canceling:**
15+
- **OLD BEHAVIOR (BUG)**: The code just does `continue` (line 204) without cleaning up
16+
- This leaves the invocation tracked in internal data structures:
17+
- `_invocations`
18+
- `_callee_to_invocations`
19+
- `_invocations_by_call`
20+
- When the callee later returns a result via `processYield()` or `processInvocationError()`, it finds the invocation_request still exists
21+
- The code tries to send to `invocation_request.caller._transport` which is now `None`
22+
- This causes an error that propagates to the callee
23+
24+
3. **If the callee DOES support call canceling:**
25+
- The code properly cleans up the invocation tracking
26+
- Sends an INTERRUPT message to the callee
27+
- Everything works correctly
28+
29+
## The Fix
30+
31+
The fix changes the logic to **always clean up the invocation tracking**, regardless of whether the callee supports call canceling or not. The difference is only whether an INTERRUPT message is sent:
32+
33+
```python
34+
# OLD CODE (BUGGY):
35+
if not callee_supports_canceling:
36+
log("INTERRUPT not supported")
37+
continue # <-- BUG: Skip cleanup!
38+
39+
# Clean up tracking (only executed if callee supports canceling)
40+
cleanup_invocation_tracking()
41+
send_interrupt()
42+
```
43+
44+
```python
45+
# NEW CODE (FIXED):
46+
# Always clean up the invocation tracking
47+
cleanup_invocation_tracking()
48+
49+
# Only send INTERRUPT if callee supports it
50+
if callee_supports_canceling:
51+
log("INTERRUPTing in-flight INVOKE")
52+
send_interrupt()
53+
else:
54+
log("INTERRUPT not supported - invocation cleaned up but not interrupted")
55+
```
56+
57+
## Changed Code
58+
59+
**File**: `/crossbar/router/dealer.py`
60+
**Method**: `Dealer.detach()`
61+
**Lines**: 192-229
62+
63+
### Before (buggy):
64+
```python
65+
outstanding = self._caller_to_invocations.get(session, [])
66+
for invoke in outstanding:
67+
if invoke.canceled:
68+
continue
69+
if invoke.callee is invoke.caller:
70+
continue
71+
callee = invoke.callee
72+
73+
# BUG: If callee doesn't support canceling, skip cleanup entirely
74+
if not callee_supports_call_canceling(callee):
75+
self.log.debug("INTERRUPT not supported...")
76+
continue # <-- SKIPS CLEANUP!
77+
78+
# Cleanup only happens if we get past the continue
79+
cleanup_invocation()
80+
send_interrupt()
81+
```
82+
83+
### After (fixed):
84+
```python
85+
outstanding = self._caller_to_invocations.get(session, [])
86+
for invoke in outstanding:
87+
if invoke.canceled:
88+
continue
89+
if invoke.callee is invoke.caller:
90+
continue
91+
callee = invoke.callee
92+
93+
# ALWAYS clean up invocation tracking
94+
if invoke.timeout_call:
95+
invoke.timeout_call.cancel()
96+
invoke.timeout_call = None
97+
98+
invokes = self._callee_to_invocations[callee]
99+
invokes.remove(invoke)
100+
if not invokes:
101+
del self._callee_to_invocations[callee]
102+
103+
del self._invocations[invoke.id]
104+
del self._invocations_by_call[(invoke.caller_session_id, invoke.call.request)]
105+
106+
# ONLY send INTERRUPT if callee supports it
107+
if callee_supports_call_canceling(callee):
108+
self.log.debug("INTERRUPTing in-flight INVOKE...")
109+
self._router.send(invoke.callee, message.Interrupt(...))
110+
else:
111+
self.log.debug("INTERRUPT not supported - invocation cleaned up but not interrupted")
112+
```
113+
114+
## Impact
115+
116+
### What Changes:
117+
- **Callee behavior when caller disconnects without call_canceling support:**
118+
- **BEFORE**: Invocation stays tracked, callee gets exception when returning result
119+
- **AFTER**: Invocation is cleaned up silently, callee can complete normally, result is discarded
120+
121+
### What Stays the Same:
122+
- Callees that support `call_canceling` still receive INTERRUPT messages as before
123+
- All other WAMP routing behavior unchanged
124+
- Backward compatible with existing deployments
125+
126+
## Testing
127+
128+
Existing test coverage:
129+
- `test_caller_detach_interrupt_cancel_not_supported` - Tests that no INTERRUPT is sent when callee doesn't support canceling
130+
- `test_caller_detach_interrupt_cancel_supported` - Tests that INTERRUPT is sent when callee supports canceling
131+
132+
The fix preserves the behavior verified by these tests while fixing the cleanup issue.
133+
134+
## Recommendation
135+
136+
This is a critical bug fix that prevents callee crashes when callers disconnect. It should be:
137+
1. Applied to the current development branch
138+
2. Backported to any maintained release branches
139+
3. Included in the next patch release
140+
141+
## Technical Details
142+
143+
### Data Structures Involved:
144+
- `_caller_to_invocations[session]` - Maps caller session to list of in-flight invocations
145+
- `_callee_to_invocations[session]` - Maps callee session to list of in-flight invocations
146+
- `_invocations[invocation_id]` - Maps invocation ID to InvocationRequest
147+
- `_invocations_by_call[(caller_session_id, call_request_id)]` - Maps call request to invocation
148+
149+
### WAMP Protocol Compliance:
150+
According to the WAMP specification, when a caller disconnects during an RPC:
151+
- If the callee supports call canceling, it should receive an INTERRUPT
152+
- If the callee doesn't support call canceling, it continues execution normally
153+
- In both cases, any result from the callee should be discarded by the router
154+
155+
The old code violated this by leaving stale invocation tracking that would cause errors when the callee tried to return a result.

0 commit comments

Comments
 (0)