AgileCoder RateLimitIssue with 30k per minute on Azure OpenAI

I am running the demo code using Azure Open AI, tier pay as you go, the limit is 30k per minute. However, with such a large limit, my prompt was "Build a Cargo game in Python" The system keeps giving me RateLimitErrors. I believe that 30k per minute is quite large already. Not sure if the system is technically feasible if it cannot run with the above token rate limit.

Oct 02 '24 01:10 hoangvnh24

Hi hoangvnh24,

Thank you for your concern. This issue may stem from your Azure OpenAI key, which can be easily fixed by increasing the TPM. Also, you can check whether the key you're using is shared with anyone. Technically, our method will run seamlessly from phase to phase, which can lead to a noticeable number of requests within a short time.

Oct 02 '24 03:10 minhngh

Hi there, what is the new TPM you are suggesting to request from Azure? Let's say 100k TPM? Let me know about this with more details if you can. I am just afraid that if we ask for more tokens, the designed system might be too expensive/inefficient to run?

Oct 06 '24 04:10 hoangvnh24