langchaingo icon indicating copy to clipboard operation
langchaingo copied to clipboard

Tracing: Added Langsmith Support

Open jubeless opened this issue 3 months ago • 0 comments

This PR introduces LangSmith tracing to the langchaingo library, enhancing debugging and monitoring capabilities for applications leveraging large language models (LLMs). These changes align with our observability needs and have been successfully running in production for several months.

We have subsequent PR ready to improve Inputs and Outputs capturing https://github.com/streamingfast/langchaingo/pull/4

Key Features

LangSmith Tracing Integration:

  • Utilizes the callbacks.Handler to enable tracing capabilities.
  • Tracing is configurable via environment variables or options in the constructor.

Refactor and Enhancements:

  • Ensures complete tracing of function calls to chains.Call.
  • Addresses an issue where callbackHandler set when calling chain.Call was not passed to the llm.Model instance.
  • Introduces a solution to inject callbackHandler into the context and retrieve it within the llm.Model.
  • Demonstrates this implementation using the OpenAI llm.Model.

Backward Compatibility

No breaking changes have been introduced. Tracing is optional and does not affect existing functionality when disabled.

Future Improvements

Refactor HandleLLMGenerateContentStart and HandleLLMGenerateContentEnd into a centralized GenerateContent function in the llms package to reduce code duplication across llms models.

image

PR Checklist

  • [x] Read the Contributing documentation.
  • [x] Read the Code of conduct documentation.
  • [x] Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to Good commit messages (such as memory: add interfaces for X, Y or util: add whizzbang helpers).
  • [x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
  • [x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. Fixes #123).
  • [x] Describes the source of new concepts.
  • [x] References existing implementations as appropriate.
  • [x] Contains test coverage for new functions.
  • [x] Passes all golangci-lint checks.

jubeless avatar Nov 21 '24 21:11 jubeless