semantic-conventions icon indicating copy to clipboard operation
semantic-conventions copied to clipboard

Change `gen_ai.response.finish_reasons` type from array to `string`, use it on metrics

Open lmolkova opened this issue 1 year ago • 0 comments
trafficstars

Fixes #1277

Changes

While OpenAI supports returning multiple completions (and therefore could have multiple finish reasons), I did not find models outside of OpenAI that return multiple choices and many SDKs (e.g. vercel) don't support returning multiple choices.

So effectively there is usually one finish reason. Having an array for it:

  • complicates querying
  • makes usage of finish reason on metrics more complicated

This PR:

  • turns finish reasons attribute into a string (comma-separated if there is more than one)
  • adds it to metrics.

Having comma-separated list has high-ish theoretical cardinality in edge cases, but results in best experience for common case. I'm suggesting to start with it and consider making it opt-in on metrics if it's proven to be problematic.

Merge requirement checklist

lmolkova avatar Jul 26 '24 21:07 lmolkova