Skip to content

Commit 25aa1f2

Browse files
HimavarshaVSHimavarshaVS
committed
add md files
1 parent 9d73199 commit 25aa1f2

2 files changed

Lines changed: 190 additions & 0 deletions

File tree

‎PR_DESCRIPTION.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# Graph Execution Debugging and Event System
2+
3+
## Overview
4+
5+
This PR introduces a comprehensive event-based debugging system for Langflow graph execution, enabling detailed tracking and analysis of graph state mutations during execution. The implementation uses a pure observer pattern that provides zero overhead when not in use, making it production-safe.
6+
7+
## Key Features
8+
9+
### 🎯 Graph Mutation Event System
10+
11+
- **Event Infrastructure**: New `GraphMutationEvent` system that tracks all graph state changes with before/after snapshots
12+
- **Observer Pattern**: Pure observer pattern implementation with `register_observer()` and `unregister_observer()` methods
13+
- **Zero Overhead**: Fast path when no observers are registered, ensuring no performance impact in production
14+
- **Serializable Events**: Events can be serialized to dictionaries for replay and storage
15+
16+
### 📊 Event-Based Recording
17+
18+
- **EventRecorder**: Observer that captures all graph mutations during execution
19+
- **EventBasedRecording**: Rich recording object with analysis methods:
20+
- `get_events_by_type()` - Filter events by type
21+
- `get_events_for_vertex()` - Get all events for a specific vertex
22+
- `get_queue_evolution()` - Track how the execution queue changes over time
23+
- `get_dependency_changes()` - Monitor dependency modifications
24+
- `show_summary()`, `show_timeline()`, `show_events_for_component()` - Visualization methods
25+
- **Save/Load**: Recordings can be saved to and loaded from files for later analysis
26+
27+
### 🔧 Graph Execution Improvements
28+
29+
#### Loop Component Enhancements
30+
- **Synchronized Dependencies**: Loop component now properly updates both `run_predecessors` and `run_map` to keep dependency structures synchronized
31+
- **State Reset**: New `reset_loop_state()` method for clean loop state management between executions
32+
- **Better Documentation**: Added critical comments explaining the relationship between dependency structures
33+
34+
#### Graph Manager Refactoring
35+
- **Async Methods**: Made `remove_from_predecessors()` and `remove_vertex_from_runnables()` async for consistency
36+
- **Sync Variants**: Added `mark_branch_sync()` for synchronous contexts (used by custom components)
37+
- **Centralized Mutations**: All graph mutations now go through centralized methods that emit events
38+
39+
### 🧪 Testing Infrastructure
40+
41+
#### Execution Path Validation
42+
- **Path Equivalence Testing**: New test suite that validates both `async_start()` and `arun()` execution paths produce identical results
43+
- **Test Data Flows**: Uses test flows that don't require API keys for reliable CI testing
44+
- **Comprehensive Tracing**: `ExecutionTracer` captures complete execution traces for comparison
45+
46+
#### Event System Tests
47+
- **Mutation Event Tests**: Tests for queue operations, dependency updates, and event emission
48+
- **Event Recorder Tests**: Tests for event capture, queue evolution tracking, and recording analysis
49+
- **Graph Mutation Tests**: Tests ensuring both dependency structures stay synchronized
50+
51+
### 🛠️ Component Validation Improvements
52+
53+
- **TYPE_CHECKING Block Support**: Component validation now properly handles `TYPE_CHECKING` blocks, extracting imports needed for `get_type_hints()` to work correctly
54+
- **Better Error Handling**: Improved error handling for components defined in notebooks or REPL environments
55+
- **Source Code Extraction**: More robust source code extraction with graceful fallbacks
56+
57+
## Technical Details
58+
59+
### Event Types Tracked
60+
61+
- `queue_extended` - When vertices are added to the execution queue
62+
- `queue_dequeued` - When vertices are removed from the queue
63+
- `dependency_added` - When dynamic dependencies are added
64+
- `vertex_marked` - When vertex states change (ACTIVE/INACTIVE)
65+
66+
### Architecture
67+
68+
```
69+
Graph
70+
├── register_observer() / unregister_observer()
71+
├── _emit_event() - Emits events to all observers
72+
└── All mutations → emit before/after events
73+
├── extend_run_queue()
74+
├── add_dynamic_dependency()
75+
├── mark_branch_sync()
76+
└── remove_from_predecessors()
77+
```
78+
79+
### Usage Example
80+
81+
```python
82+
from lfx.graph.graph.base import Graph
83+
from lfx.debug.event_recorder import record_graph_with_events
84+
85+
# Record graph execution
86+
graph = Graph.from_payload(flow_data)
87+
recording = await record_graph_with_events(graph, "My Flow")
88+
89+
# Analyze the recording
90+
recording.show_summary()
91+
recording.show_timeline()
92+
93+
# Get specific insights
94+
queue_evolution = recording.get_queue_evolution()
95+
dependency_changes = recording.get_dependency_changes()
96+
97+
# Save for later analysis
98+
recording.save("flow_recording.pkl")
99+
```
100+
101+
## Files Changed
102+
103+
### New Files
104+
- `src/lfx/src/lfx/debug/__init__.py` - Debug module initialization
105+
- `src/lfx/src/lfx/debug/events.py` - GraphMutationEvent and observer types
106+
- `src/lfx/src/lfx/debug/event_recorder.py` - EventRecorder and EventBasedRecording
107+
- `src/backend/tests/unit/graph/test_execution_path_validation.py` - Execution path equivalence tests
108+
- `src/backend/tests/unit/graph/test_execution_path_equivalence.py` - Execution tracing utilities
109+
- `src/backend/tests/unit/graph/test_event_recorder.py` - Event recorder tests
110+
- `src/backend/tests/unit/graph/test_graph_mutation_events.py` - Mutation event tests
111+
112+
### Modified Files
113+
- `src/lfx/src/lfx/graph/graph/base.py` - Added observer pattern, event emission
114+
- `src/lfx/src/lfx/graph/graph/runnable_vertices_manager.py` - Made methods async
115+
- `src/lfx/src/lfx/components/logic/loop.py` - Improved dependency synchronization
116+
- `src/lfx/src/lfx/custom/custom_component/component.py` - Better error handling
117+
- `src/lfx/src/lfx/custom/custom_component/custom_component.py` - Use mark_branch_sync
118+
- `src/lfx/src/lfx/custom/validate.py` - TYPE_CHECKING block support
119+
- `pyproject.toml` - Added marimo dependency for debugging notebooks
120+
121+
## Benefits
122+
123+
1. **Debugging**: Comprehensive visibility into graph execution state changes
124+
2. **Testing**: Better test coverage with execution path validation
125+
3. **Reliability**: Synchronized dependency structures prevent bugs
126+
4. **Performance**: Zero overhead when debugging is not active
127+
5. **Extensibility**: Easy to add new event types and observers
128+
129+
## Testing
130+
131+
- ✅ All existing tests pass
132+
- ✅ New execution path validation tests pass
133+
- ✅ Event system tests pass
134+
- ✅ Loop component tests pass with improved dependency handling
135+
136+
## Breaking Changes
137+
138+
None - This is a purely additive change. The event system is opt-in and has zero overhead when not used.
139+
140+
## Future Work
141+
142+
- [ ] Add more event types (vertex execution start/end, memory updates, etc.)
143+
- [ ] Create visualization tools for event recordings
144+
- [ ] Add event filtering and querying capabilities
145+
- [ ] Integrate with Langflow UI for real-time debugging
146+

‎PR_DESCRIPTION_SHORT.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Graph Execution Debugging and Event System
2+
3+
## Summary
4+
5+
Introduces a comprehensive event-based debugging system for Langflow graph execution with zero overhead when not in use. Uses a pure observer pattern to track all graph state mutations during execution.
6+
7+
## Key Changes
8+
9+
### 🎯 Event System
10+
- New `GraphMutationEvent` system tracking all graph state changes
11+
- Observer pattern with `register_observer()` / `unregister_observer()`
12+
- Zero overhead fast path when no observers registered
13+
- Serializable events for replay and analysis
14+
15+
### 📊 Event Recording
16+
- `EventRecorder` captures all graph mutations
17+
- `EventBasedRecording` with analysis methods (queue evolution, dependency changes, etc.)
18+
- Save/load recordings for later analysis
19+
20+
### 🔧 Graph Improvements
21+
- **Loop Component**: Synchronized `run_predecessors` and `run_map` dependencies
22+
- **Graph Manager**: Made `remove_from_predecessors()` async, added `mark_branch_sync()` for sync contexts
23+
- **Component Validation**: Better `TYPE_CHECKING` block handling
24+
25+
### 🧪 Testing
26+
- Execution path validation tests (`async_start()` vs `arun()` equivalence)
27+
- Event system tests
28+
- Comprehensive test coverage
29+
30+
## Usage
31+
32+
```python
33+
from lfx.debug.event_recorder import record_graph_with_events
34+
35+
recording = await record_graph_with_events(graph, "My Flow")
36+
recording.show_summary()
37+
recording.get_queue_evolution()
38+
recording.save("recording.pkl")
39+
```
40+
41+
## Breaking Changes
42+
43+
None - purely additive, opt-in feature.
44+

0 commit comments

Comments
 (0)