semantic-conventions
semantic-conventions copied to clipboard
GenAI: Clarify what `gen_ai.request.top_k` means
An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.
Ensures only the top k most likely tokens are considered for generation at each step. Defaults to 0, min value of 0, max value of 500.
Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Learn more technical details here.
We should document what this parameter is, currently it's not clear
https://github.com/open-telemetry/semantic-conventions/blob/26e3779ba8367c7f4c1d5c6e23000cdb51455d86/docs/gen-ai/gen-ai-spans.md?plain=1#L62