graphrag icon indicating copy to clipboard operation
graphrag copied to clipboard

Fix/feat: Implementation of Minute-Based Rate Limiting in CommunityReportsExtractor Using asyncio and async_mode

Open glide-the opened this issue 1 year ago • 0 comments

Description

RateLimiter: The original TpmRpmLLMLimiter strategy did not account for minute-based rate limiting when scheduled. The RateLimiter was introduced to ensure that the CommunityReportsExtractor could be scheduled to adhere to rate configurations on a per-minute basis.

RateLimiter scheduled: using asyncio and async_mode

Additionally, some key loading issues for rpm = "REQUESTS_PER_MINUTE" and tpm = "TOKENS_PER_MINUTE" were fixed. Configuration loading was also enhanced to include temperature = "TEMPERATURE" and top_p = "TOP_P" settings.

Checklist

  • [x] I have tested these changes locally.
  • [x] I have reviewed the code changes.

glide-the avatar Jul 05 '24 06:07 glide-the