How to reduce token usage
The usage fee for the text-davinci-3 model is $0.0200/1K tokens, which is very expensive.On average, around 3000 tokens are required per simple request,Each simple conversation approximately costs 0.1 US dollars,this is very economically unfriendly to users.
I am trying to use a more economical text ada-1 model.
self.llm = OpenAI(temperature=0,model_name="text-ada-001")
But the maxmum token size of this model is restricted to 2048,which lead to the following Error.
openai.error.InvalidRequestError: This model's maximum context length is 2049 tokens, however you requested 2074 tokens (1818 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.
So I want to ask if there is a way to reduce the number of tokens per request or split each long request in order to use a more economical ada model.
Looking forward to hearing from you,Thanks!
If the ada model is supported, the token cost will be reduced by 50 times the original cost.
If you can specify GPT-3.5 Turbo, its significantly less expensive @ $0.002 / 1K tokens. REF
The maxmum token size of gpt-3.5-tuibo is also 2048.