haystack icon indicating copy to clipboard operation
haystack copied to clipboard

feat: Trace pipeline run input/output data

Open vblagoje opened this issue 10 months ago • 4 comments

Why:

This change is motivated by the need to improve the pipeline's debugging and tracing capabilities by providing more detailed information about the input and output data. Most importantly this change will help us generate better pipeline level traces in langfuse and other integrations.

What:

The specific changes made include:

  • Adding two new tracing tags, haystack.pipeline.input_data and haystack.pipeline.output_data, to provide more detailed information about the input and output data.
  • Moving the declaration of final_outputs to the top of the tracing context manager with block, ensuring it is initialized before being used.

How can it be used:

The added tracing tags can be utilized to monitor and analyze the pipeline's input and output data, enabling developers to:

  • Use the tracing data to debug and optimize the pipeline's performance.
  • Without these traces we can't capture pipeline inputs/outputs and langfuse integration for pipeline run looks like this:

pipeline_run_before

  • After these changes langfuse integration for pipeline run looks like this:

pipeline_run_after

How did you test it:

  • No tests were added, CI run hopefully discovers potential issues

Notes for the reviewer:

Please review the placement of the final_outputs declaration and ensure it does not affect the pipeline's behavior. Additionally, consider adding more tracing tags to provide further insight into the pipeline's operation.

vblagoje avatar Apr 24 '24 11:04 vblagoje

cc @silvanocerza just to double check everything as well

vblagoje avatar Apr 24 '24 11:04 vblagoje

Dropping comment to watch this PR.

mrm1001 avatar Apr 25 '24 07:04 mrm1001

Pull Request Test Coverage Report for Build 8880780958

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 5 unchanged lines in 1 file lost coverage.
  • Overall coverage remained the same at 90.129%

Files with Coverage Reduction New Missed Lines %
core/pipeline/pipeline.py 5 94.06%
<!-- Total: 5
Totals Coverage Status
Change from base Build 8878916224: 0.0%
Covered Lines: 6337
Relevant Lines: 7031

💛 - Coveralls

coveralls avatar Apr 26 '24 08:04 coveralls

Dropping @anakin87 and adding @silvanocerza for this change in pipeline run logic

vblagoje avatar Apr 26 '24 09:04 vblagoje