How to configure and avoid Open AI rate limiting when ingesting files?

[emahpour]

trafficstars

Describe the problem I uploaded a json file with around 4000 entries inside. While I was monitoring the processes, I realized Open AI is enforcing rate limiting and the application was not responsive as it was keep retrying the failed calls to Open AI. What is the recommendation to avoid running into this problem?

To Reproduce Create a json file with large number of entries (eg 4000 rows).

1. Upload the file as a document
2. Initiate Graph Creation Process

Expected behavior Processing the Graph generation with consideration of API rate limits with Open AI

Screenshots