openlimit icon indicating copy to clipboard operation
openlimit copied to clipboard

Published 20 hours ago • verified publisher icon shobrook

Is the token limit check correct?

Open causalbody opened this issue 1 year ago • 3 comments

For example 40k TPM is converted to 666.66 TPS I've a summarization usecase where the incoming token is 800+, in which case the logic seems to wait indefinitely. Has anyone run into this or can confirm if my reading is correct or not?

causalbody avatar Aug 10 '23 03:08 causalbody