prompt2model icon indicating copy to clipboard operation
prompt2model copied to clipboard

Published 20 hours ago • verified publisher icon neulab

tiktoken usage

Open zhaochenyang20 opened this issue 1 year ago • 2 comments

https://github.com/neulab/prompt2model/pull/335#issuecomment-1711578817

Another thing is that I am not sure that all the API-based models use tiktoken to compute input tokens. Thus, the num of input tokens may slightly differ between these models. But that's minimal.

zhaochenyang20 avatar Sep 08 '23 15:09 zhaochenyang20