Getting rate limited by OpenAI on ingestion

[jwulf]

Getting the following error when ingesting my documentation:

openai.error.RateLimitError: Rate limit reached for default-global-with-image-limits in organization org-Wge... on requests per min. Limit: 60.000000 / min. Current: 70.000000 / min. Contact [email protected] if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.

Additional context:

Number of Tokens = 538,902

I can't find any way to specify a backoff. I have a paid account, and apparently the rate limit is raised over time. However, I would like to be able to specify a rate limit.

[dartpain]

Think that vectorising data in batches will help Check this issue out #54

[jaredbradley243]

Think that vectorising data in batches will help Check this issue out #54

Are you suggesting changing the chunk size here will help? I too get the same error.

text_splitter = CharacterTextSplitter(chunk_size=1500, separator="\n")

(Line 94 of the ingest_rst_sphinx.py file)

[mefengl]

I use some method that seems too tricky: add some sleep in this function https://github.com/hwchase17/langchain/blob/b7747017d72eaeabfa65edb7eec413d3fd006ddb/langchain/embeddings/openai.py#LL72-L76C87

[dartpain]

Solved in #54 Will help avoid rate limits and save progress if llm provider has issues

[mefengl]

@dartpain probably not fix, see https://github.com/arc53/DocsGPT/pull/85

copy from that pull request:

I don't know why there needs to be a store.index = None, but I got an error when continuing the process, so I restored the index before continuing;

But that's only fixed the index issue.

Another problem is there is no error catch for the store.add_texts([i.page_content], metadatas=[i.metadata]) in the except block, so if 10 seconds is not enough, then the second error will be thrown out, and this time, no try/except for this one

[EricGao888]

I use some method that seems too tricky: add some sleep in this function https://github.com/hwchase17/langchain/blob/b7747017d72eaeabfa65edb7eec413d3fd006ddb/langchain/embeddings/openai.py#LL72-L76C87

Another workaround is to use retry: #91 , it works pretty well : )