gateway icon indicating copy to clipboard operation
gateway copied to clipboard

Published 20 hours ago • verified publisher icon Portkey-AI

Adding requestExecTime to requestOptions in the context

Open roh26it opened this issue 1 year ago • 1 comments

This improves response time metrics for streaming and cache use cases, without retries.

roh26it avatar May 21 '24 18:05 roh26it

We can add metrics like time to first token also here.

roh26it avatar Jul 05 '24 06:07 roh26it

Closing this in favour of another PR with a larger scope

VisargD avatar Dec 09 '24 09:12 VisargD