Clickhouse query for total number of LLM spans

[areibman]

[teocns]

@Dwij1704 is this already working with the new exporter setup for metrics?

[Dwij1704]

I am coming up with a plan, give me a few hours, I will freshen up an get back to work.

Here is what I am thinking, we can solely rely on metrics for duration and token usage for each run (helpful for waterfall) and query click house for total tokens and llm calls (which I really think is possible and worth investigating). We can rely on spanAttributes for other stuff like model inputs and outputs or instructions and other fancy stuff.

Now that I am almost done with the instrumentation stuff, I can investigate further and come up with some conclusions in a few hours before I go to sleep.