langfuse-js icon indicating copy to clipboard operation
langfuse-js copied to clipboard

fix: bedrock model and token detection

Open tomi-bigpi opened this issue 7 months ago • 2 comments

Problem

The JavaScript library is missing the latest support for AWS Bedrock models as compared to the Python library. Currently, it will not correctly detect the model name when using Bedrock and Langchain. Additionally, it will not track input/output token usage.

Changes

  • Added a shared extractModelName method to core utils
  • Added _parseUsage method to langchain callback to extract token usage
  • Use new model and token usage logic for langchain
  • Added langfuse-langchain tests to top-level package.json test scripts

All new methods were ported over from the Python library with minor adjustments and fixes.

Note: Both changes in Langchain will affect all model detection and token extraction. While all of the existing tests pass, there didn't seem to be great coverage on these aspects. I've added (simple) tests only for the Bedrock use cases since that's all I have readily available.

Release info Sub-libraries affected

Bump level

  • [ ] Major
  • [x] Minor
  • [ ] Patch

Libraries affected

  • [ ] All of them
  • [ ] langfuse
  • [ ] langfuse-node
  • [x] langfuse-core
  • [x] langfuse-langchain

Changelog notes

  • Added support for Bedrock input/output token usage statistics
  • Added improved support for model name detection

tomi-bigpi avatar Jul 16 '24 20:07 tomi-bigpi