Add API request throttling to find-inactive-members script to respect API rate limit headers
This pull request introduces significant improvements to the find_inactive_members.rb script, focusing on robust API rate limit handling, improved error resilience, and user configurability. The most important changes are the addition of a custom throttling middleware to respect GitHub API limits, smarter request and pagination logic, and enhanced error handling for forbidden responses. Users can now opt to disable throttling for testing or use on GitHub Enterprise instances with higher limits.
API Throttling and Pagination Improvements
- Introduced a custom
ThrottleMiddlewarefor Faraday, which dynamically throttles requests based on GitHub API rate limit headers and pauses execution when limits are critically low. This helps prevent rate limit errors and provides clear warnings to users. - Added a
--no-throttleoption to the CLI and supporting logic throughout the script, allowing users to disable throttling for testing or use with GitHub Enterprise. When disabled, Octokit auto-pagination is enabled for faster, unrestricted requests. [1] [2] - Refactored all major API calls (
organization_members,organization_repositories,commits_since,list_issues,issues_comments,pull_requests_comments) to use new helper methods (smart_request,paginated_request, andretry_on_403) for unified pagination, throttling, and error handling. [1] [2] [3] [4] [5] [6]
Error Handling Enhancements
- Implemented a
retry_on_403helper that automatically retries API requests on 403 Forbidden errors, prompts the user for further action after repeated failures, and integrates with all major API interactions.
Documentation Updates
- Updated
README.mdwith instructions for installing the newfaradaydependency, clarified required OAuth scopes, and added a section explaining API rate limits and throttling behavior. [1] [2]
User Experience Improvements
- Enhanced output to show remaining API requests and reset times before analyzing each repository, using response headers for efficiency. Added clear warnings when throttling is disabled. [1] [2]
These changes make the script safer, more reliable, and easier to use for both small and large organizations.
@copilot update the readme file to ensure that the user know to install an additional gem for the throttling.
@copilot update the readme file to ensure that the user know to install an additional gem for the throttling.
Updated the README to include faraday in the gem installation instructions since the throttling middleware extends Faraday::Middleware directly. Commit a7f5548.
@copilot implement a more efficient debug check
@copilot implement a more efficient debug check
Implemented a more efficient debug check by caching the ENV['THROTTLE_DEBUG'] lookup during initialization instead of checking it on every API request. This eliminates expensive environment variable access on each of the thousands of API calls. Commit fb55e2e.