promptflow icon indicating copy to clipboard operation
promptflow copied to clipboard

[Feature Request] Support external Tracing Tools (e.g. langfuse)

Open carlFsq opened this issue 1 year ago • 5 comments

Is your feature request related to a problem? Please describe. I want to trace my whole prompt flow application with my existing langfuse solution. This functions of course well with the python tool, but not with e.g. the LLM and the Embedding Tool.

Describe the solution you'd like It would be nice to use the existing prompt flow without just being able to use the Python Tool.

Describe alternatives you've considered Only using the Python Tool and passing the trace id to every node in the graph

Additional context

carlFsq avatar Jul 23 '24 06:07 carlFsq

Hi all, Langfuse founder/maintainer here.

We'd love to support the promptflow community in building this out. We won't have internal capacity to build/maintain this in the coming weeks but can provide guidance and answer questions (here or via discord/slack channel as things move along).

Happy to pick this up if folks are interested. Wrapping this in our Python Decorator may be a possibility.

  • Langfuse has a powerful API that includes a downloadable OpenAPI spec (at the top here which is a great starting point.

clemra avatar Jul 23 '24 17:07 clemra

Hi @carlFsq @clemra ,

Thanks for your input and the time you took to share your idea with us.

Currently, we don't have the bandwidth to implement this feature right away, so we've labeled it as a long-term task. We hope to address it in the future as our resources allow.

0mza987 avatar Jul 24 '24 03:07 0mza987

This would be wonderful.

We're currently building a custom integration, it's not ready yet, but as soon as it is we could try to add it here.

@aiancheruk @stephaneBylaw

tuanardouin avatar Sep 17 '24 16:09 tuanardouin

Hi @tuanardouin -- that sounds great. Feel free to ping me if the Langfuse team can be of help. here or clemens at langfuse.com

clemra avatar Sep 17 '24 16:09 clemra

Hi @tuanardouin, I am really interested, too. Any updates so far?

FabianSchurig avatar Feb 15 '25 17:02 FabianSchurig