agenta icon indicating copy to clipboard operation
agenta copied to clipboard

[AGE-163] Propagating the cost from Span to Trace

Open mmabrouk opened this issue 2 months ago • 4 comments

Right now the user needs to explicitly return in the traced function a dict that contains the cost, message, and number of tokens. However, this information is simply the sum of costs and tokens used in all the spans for this trace. So, instead we want to propagate the cost from the span to the trace.

First, we need to determine whether to do the calculation in the SDK, backend or frontend. It looks like the SDK is the right place to do that.

This issue goes hand in hand with another issue for changing the way the playground interacts with the LLM apps (Removing FuncResponse).

We need to determine therefore the schema for the output of the LLM applications. Right now it includes the message, the cost, and the number of tokens.

A first proposal is to require the user to only provide the message, the output should have the output and the trace_id (and the cost/tokens inferred from the trace?)

From SyncLinear.com | AGE-163

mmabrouk avatar May 01 '24 17:05 mmabrouk