docs/feat: usage pattern for adding metadata/userId/sessionId/... at runtime when using new llamaindex integration (instrumentation module)

[marcklingen]

Discussed in https://github.com/orgs/langfuse/discussions/3168

^{Originally posted by erik-squared August 30, 2024} sorry if i'm asking an obvious question, but trying to figure out correct usage pattern for llamaindex usage:

i see that the basic using model is:

langfuse_span_handler = LlamaIndexSpanHandler()
instrument.get_dispatcher().add_span_handler(langfuse_span_handler)

llamaindex operations then subsequently generate traces/spans to langfuse.

if i wanted to add a userid to the trace, i see that LlamaIndexSpanHandler() constructor accepts a user_id as a parameter.

langfuse_span_handler = LlamaIndexSpanHandler(user_id="someuser")
instrument.get_dispatcher().add_span_handler(langfuse_span_handler)

but if i am running this code as part of a backend and would like to capture an inbound user_id for each request, i don't see how to do that.

thanks in advance!

Options

• Wrap via langfuse decorator to set metadata via langfuse_context.
• Native llamaindex metadatafields that can be extracted. Risk of setting trace-level metadata via span metadata when used as part of larger application.

[marcklingen]

Using the decorator here is probably the best solution as it also makes this integration interoperable with other Langfuse integrations for Python and reduces complexity in the llamaindex specific integration logic.

[erik-squared]

followup question re: using decorator - using the original callback-based integration, llamaindex activity was linked to the wrapped trace via:

@observe()
def llama_index_fn(question: str):
    # Set callback manager for LlamaIndex, will apply to all LlamaIndex executions in this function
    langfuse_handler = langfuse_context.get_current_llama_index_handler()
    Settings.callback_manager = CallbackManager([langfuse_handler])

with the new integration, how would the new span be connected to the wrapped trace?

[marcklingen]

This is not yet supported. As mentioned above I agree that using the decorator is probably the best way to harmonize this new integration (still experimental) and make it interoperable with all Langfuse platform features

[hassiebp]

@erik-squared We have just shipped a major improvement to the LlamaIndex instrumentation integration that should address your root-setting issue. Please upgrade and check out our docs in how to enable the integration via the LlamaIndexInstrumentor and let us know your feedback here! 🙏🏾

• Release
• Docs
• Cookbook