All repos for an organisation

[tinydylan]

trafficstars

My organisation has hundreds of GitHub repositories under a single GitHub Organisation. This tap only extracts data a single repository - we would like this tap to extract data for all repositories in our Organisation.

[luandy64]

https://github.com/singer-io/tap-github/blob/35ac5df8542f1c7cb1c736243f34feb252c3ec47/tap_github.py#L863

https://github.com/singer-io/tap-github/blob/35ac5df8542f1c7cb1c736243f34feb252c3ec47/tap_github.py#L868

I think according to these lines the tap is able to run for multiple repositories

[tinydylan]

Thanks, @luandy64.

So, it sounds like the tap would just need to be altered so that it can read the list of repositories from a GitHub API call (https://api.github.com/orgs/ORG_NAME/repos).

[osterman]

Btw, I went down the route but quickly ran into rate limits. Without #63, this cannot be done reliably.

[tinydylan]

In stitchdata, I have the github integration running against a hard-coded list of 350 repositories. I hit rate limits for a few hours, then it's fine after that.

Getting all repos for an org in one setting is no different to what I'm doing, it just means that I don't have to keep regenerating the list of repos every time it changes.

While the rate limit does get in the way, it gets in the way already and I would argue for going ahead with this requirement independendly of graceful rate-limit handling.