feat: Graph Execution Debugging and Event System

[HimavarshaVS]

Summary

Introduces a comprehensive event-based debugging system for Langflow graph execution with zero overhead when not in use. Uses a pure observer pattern to track all graph state mutations during execution.

Key Changes

🎯 Event System

• New GraphMutationEvent system tracking all graph state changes
• Observer pattern with register_observer() / unregister_observer()
• Zero overhead fast path when no observers registered
• Serializable events for replay and analysis

📊 Event Recording

• EventRecorder captures all graph mutations
• EventBasedRecording with analysis methods (queue evolution, dependency changes, etc.)
• Save/load recordings for later analysis

🔧 Graph Improvements

• Loop Component: Synchronized run_predecessors and run_map dependencies
• Graph Manager: Made remove_from_predecessors() async, added mark_branch_sync() for sync contexts
• Component Validation: Better TYPE_CHECKING block handling

🧪 Testing

• Execution path validation tests (async_start() vs arun() equivalence)
• Event system tests
• Comprehensive test coverage

Usage

from lfx.debug.event_recorder import record_graph_with_events

recording = await record_graph_with_events(graph, "My Flow") recording.show_summary() recording.get_queue_evolution() recording.save("recording.pkl")

Summary by CodeRabbit

Release Notes

• New Features

  • Event-based debugging system for graph execution with zero overhead when unused
  • EventRecorder captures mutations; EventBasedRecording enables queue evolution and dependency analysis
  • Built-in save/load and visualization tools for recorded graphs

• Tests

  • Added comprehensive tests for event recording and execution path equivalence

• Chores

  • Added marimo development dependency

[coderabbitai[bot]]

[!IMPORTANT]

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

[!NOTE]

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

This PR introduces an event-based debugging system for graph execution using an observer pattern to track mutations. Adds GraphMutationEvent, EventRecorder, and EventBasedRecording for capturing and analyzing mutations. Converts multiple graph operations to async variants, enhances loop component synchronization, improves error handling in custom components, and extends TYPE_CHECKING support in validation. Zero overhead when no observers are registered.

Changes

Cohort / File(s)	Change Summary
Debug Module src/lfx/src/lfx/debug/__init__.py, src/lfx/src/lfx/debug/events.py, src/lfx/src/lfx/debug/event_recorder.py	Introduces event-based debugging system: GraphMutationEvent dataclass with before/after snapshots and serialization; EventRecorder observer to capture mutations; EventBasedRecording with analysis methods (queue evolution, dependency changes, filtering by type/vertex/component) and save/load capabilities.
Graph Core Event System src/lfx/src/lfx/graph/graph/base.py	Adds observer pattern registration/unregistration, event emission on mutations with before/after snapshots, fast path with zero overhead when no observers, introspection helpers (get_run_queue, get_context_dict), and converts to async: get_next_in_queue, extend_run_queue, add_dynamic_dependency, mark_vertex, _mark_branch, plus sync wrappers (mark_branch_sync).
Graph Manager src/lfx/src/lfx/graph/graph/runnable_vertices_manager.py	Converts remove_from_predecessors and remove_vertex_from_runnables to async methods for consistency with centralized mutation emission.
Component Enhancements src/lfx/src/lfx/components/logic/loop.py	Adds reset_loop_state public method for state cleanup; enhances update_dependency with synchronization logic for run_map/run_predecessors bidirectional linkage.
Custom Component Error Handling src/lfx/src/lfx/custom/custom_component/component.py	Replaces exceptions with placeholder comments when module not found or source code unavailable, improving REPL/notebook robustness.
Custom Component Updates src/lfx/src/lfx/custom/custom_component/custom_component.py	Replaces mark_branch with mark_branch_sync to use synchronous variant.
Validation Enhancements src/lfx/src/lfx/custom/validate.py	Adds _is_type_checking_block helper and extends prepare_global_scope to extract and make available imports from TYPE_CHECKING blocks for runtime type hint resolution.
Tests src/backend/tests/unit/graph/test_graph_mutation_events.py, src/backend/tests/unit/graph/test_event_recorder.py	Adds test coverage for mutation event emission, observer overhead (fast path), event recording with graph execution, and queue/dependency change tracking.
Project Configuration pyproject.toml	Adds marimo\>=0.17.0 to dev dependencies for debugging notebook support.
Starter Project src/backend/base/langflow/initial_setup/starter_projects/Research Translation Loop.json	Updates LoopComponent template code_hash and embeds updated code with sync dependency update notes.

Sequence Diagram(s)

sequenceDiagram
    participant App as Application
    participant Graph
    participant Observer as EventRecorder
    participant Queue as Run Queue
    
    Note over App,Observer: Event System Registration
    App->>Graph: register_observer(EventRecorder)
    Graph->>Graph: _enable_events = True
    
    Note over App,Observer: Mutation Emission with Event
    App->>Graph: extend_run_queue(vertices)
    Graph->>Graph: _emit_event(before snapshot)
    Note right of Graph: event_type="queue_extended"<br/>timing="before"
    Graph->>Observer: on_event(before_event)
    Observer->>Observer: collect event
    
    Graph->>Queue: append vertices
    Graph->>Graph: _emit_event(after snapshot)
    Note right of Graph: timing="after"<br/>updated state
    Graph->>Observer: on_event(after_event)
    Observer->>Observer: collect event
    
    Note over App,Observer: No Observer Path (Zero Overhead)
    App->>Graph: extend_run_queue(vertices)
    Note right of Graph: _enable_events=False<br/>fast path skips<br/>event emission
    Graph->>Queue: append vertices

sequenceDiagram
    participant LoopComponent
    participant Graph as Graph Manager
    participant RunQueue
    
    Note over LoopComponent,RunQueue: Loop Dependency Synchronization
    LoopComponent->>LoopComponent: item_output()
    LoopComponent->>LoopComponent: update_dependency(dependency)
    Note right of LoopComponent: sync path for<br/>run_map update
    
    LoopComponent->>Graph: add_dynamic_dependency(vertex, predecessor)
    Graph->>Graph: update run_predecessors
    Graph->>Graph: update run_map<br/>(bidirectional)
    Graph->>RunQueue: reflect new dependency
    
    Note over LoopComponent,RunQueue: Reset for Fresh Execution
    LoopComponent->>LoopComponent: reset_loop_state()
    LoopComponent->>LoopComponent: clear initialized/index<br/>clear aggregated/data

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Areas requiring extra attention:

• src/lfx/src/lfx/graph/graph/base.py — Extensive refactoring with async method conversions, observer pattern integration, event emission logic, and state snapshot handling; critical path for all graph mutations
• Async method signature changes — Breaking changes converting remove_from_predecessors, remove_vertex_from_runnables, mark_vertex, and branch-marking operations to async; verify all callers are updated correctly
• Event emission timing and accuracy — Verify before/after snapshots capture correct state and step counters increment properly; validate zero-overhead fast path is truly overhead-free
• Loop component synchronization — Validate that update_dependency properly maintains bidirectional linkage between run_predecessors and run_map
• Observer error handling — Verify observer errors are collected without breaking graph execution

Possibly related PRs

• langflow-ai/langflow#8091 — Touches LoopComponent dependency management and graph run/predecessor structures with similar update_dependency patterns

Suggested labels

size:XXL, lgtm

Suggested reviewers

• ogabrielluiz
• edwinjosechittilappilly
• erichare

Pre-merge checks and finishing touches

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Test Quality And Coverage	⚠️ Warning	Test coverage for the event-based debugging system is incomplete with only 5 functional tests and critical functionality lacking coverage including EventBasedRecording query methods, error handling, and edge cases.	Expand test coverage to include EventBasedRecording persistence, all query methods, observer error handling, async testing with pytest.mark.asyncio, and integration tests combining multiple features.
Excessive Mock Usage Warning	❓ Inconclusive	Test files specified in PR summary do not exist in repository; cannot assess mock usage without reviewing actual test implementations.	Verify all test files mentioned in summary are committed and available. Review test code to confirm integration tests use real graph execution and unit tests avoid excessive mocking of internal state.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically summarizes the main change: introducing an event-based debugging system with event tracking for graph execution.
Docstring Coverage	✅ Passed	Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.
Test Coverage For New Implementations	✅ Passed	Test coverage is adequate for the event-based debugging system. The PR includes substantive tests for GraphMutationEvent, EventRecorder, EventBasedRecording, and observer registration mechanisms.
Test File Naming And Structure	✅ Passed	All test files follow correct naming conventions (test_*.py), use pytest with @pytest.mark.asyncio for async tests, employ descriptive test function names, and cover both positive scenarios and edge cases including performance, event capture, and execution equivalence.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions[bot]]

Frontend Unit Test Coverage Report

Coverage Summary

Lines	Statements	Branches	Functions
	16.67% (4695/28164)	9.99% (2179/21794)	10.94% (677/6183)

Unit Test Results

Tests	Skipped	Failures	Errors	Time
1829	0 :zzz:	0 :x:	0 :fire:	23.633s :stopwatch:

[codecov[bot]]

Codecov Report

:x: Patch coverage is 65.63707% with 89 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 33.34%. Comparing base (2d02456) to head (f0cf5f5).

Files with missing lines	Patch %	Lines
src/lfx/src/lfx/debug/event_recorder.py	50.00%	49 Missing and 1 partial :warning:
src/lfx/src/lfx/graph/graph/base.py	74.57%	22 Missing and 8 partials :warning:
...c/lfx/src/lfx/custom/custom_component/component.py	0.00%	4 Missing :warning:
src/lfx/src/lfx/custom/validate.py	70.00%	1 Missing and 2 partials :warning:
src/lfx/src/lfx/debug/events.py	90.90%	2 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10545      +/-   ##
==========================================
+ Coverage   33.22%   33.34%   +0.12%     
==========================================
  Files        1391     1393       +2     
  Lines       65849    66076     +227     
  Branches     9745     9777      +32     
==========================================
+ Hits        21877    22034     +157     
- Misses      42850    42910      +60     
- Partials     1122     1132      +10     

Flag	Coverage Δ
backend	52.44% <ø> (-0.07%)	:arrow_down:
frontend	15.36% <ø> (ø)
lfx	39.79% <65.63%> (+0.35%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...rc/lfx/custom/custom_component/custom_component.py	38.10% <100.00%> (+0.57%)	:arrow_up:
...x/src/lfx/graph/graph/runnable_vertices_manager.py	96.05% <100.00%> (+1.31%)	:arrow_up:
src/lfx/src/lfx/debug/events.py	90.90% <90.90%> (ø)
src/lfx/src/lfx/custom/validate.py	41.27% <70.00%> (+0.99%)	:arrow_up:
...c/lfx/src/lfx/custom/custom_component/component.py	59.44% <0.00%> (+0.06%)	:arrow_up:
src/lfx/src/lfx/graph/graph/base.py	50.22% <74.57%> (+1.82%)	:arrow_up:
src/lfx/src/lfx/debug/event_recorder.py	50.00% <50.00%> (ø)

... and 7 files with indirect coverage changes

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.