deepset-ai
Add Langfuse tracing support for AmazonBedrockChatGenerator
Is your feature request related to a problem? Please describe.
Currently, our Langfuse integration does not support generation span tracing for Amazon Bedrock, which limits observability and debugging capabilities for developers using Bedrock as their LLM provider.
Describe the solution you'd like
I'd like to see native Langfuse support added for AmazonBedrockChatGenerator, similar to the existing integrations for other chat generators. This would enable automatic logging of traces, spans, and metadata for Bedrock-based LLMs.
Describe alternatives you've considered There are no alternatives except to use another Haystack/Langfuse supported LLM.
Additional context See: https://github.com/deepset-ai/haystack-core-integrations/pull/1577 https://github.com/deepset-ai/haystack-core-integrations/pull/1087
@jkruzek note that it does work already it is just that we don't have as detailed generation spans for AmazonBedrockChatGenerator Perhaps you can add your first contribution? We can review it together!
Hey @vblagoje what exactly is missing in our integration to enable this to work for AmazonBedrockChatGenerator?
Ah not much. We need to add AmazonBedrockChatGenerator to a list of supported chat generators inside langfuse tracer and we need to add a few things to ABCG as we did in https://github.com/deepset-ai/haystack-core-integrations/pull/1577 https://github.com/deepset-ai/haystack-core-integrations/pull/1087 to other generators.
The difference is this:
See this trace for an agent with OpenAIChatGenerator. You notice how the icon is this ventilator, and once you click on chat_generator node you get all the latency, cost, model info, token count, and other info
while with ABCG you get not so rich trace version - no such detailed metadata