Add pipeline breakpoint support for the internals of SuperComponent
It would be great if we could support breakpoints within SuperComponents and Agent.
This is still in an exploratory phase so it would be good to plan out what this would require and if the existing breakpointing architecture could be easily extended to work within SuperComponents and Agent.
@sjrl I thought of it in two ways.
- First way:
- Focus on Pipeline._run_component. Add a check against a simple list of component names. When a breakpoint is hit, log the component name and its inputs.
- Comprehensive Solution:
- Full interactive debugging (like pdb) within the pipeline/agent flow. It requires managing execution state, providing commands to inspect/modify data, and resuming execution. Properly handling breakpoints in AsyncPipeline and Agent (especially its iterative loop and async nature). Designing a clean API for users to set, manage, and interact with breakpoints requires careful thought. Extending this to SuperComponent to correctly map internal pipeline breakpoints to the SuperComponent context.
Hey @YassinNouh21 we already have an initial implementation of pipeline breakpoints in haystack-experimental. You can find the example colab here and the discussion on it here.
So the plan for this issue for us is to extend the current implementation there with the ability to specify a breakpoint within a SuperComponent or an Agent.