RubyGems (Gems.info) Rate limit issue

[paulmthiebauth]

Hello,

I'm hoping you can help me out with an issue we're having with Jenkins + Gemsurance. As of late we're having issues with Gemsurance due to the # of gems in our Gemfile. The issue is that Gemsurance is making too many calls to RubyGems via Gems.info(gem_name) and eventually errors out because too many calls have been made, causing our builds to fail. The error output looks like this: Psych::SyntaxError: (<unknown>): mapping values are not allowed in this context at line 13 column 15

We've been able to reproduce this issue locally by running: 188.times { |n| Gems.info('a'*n) } 188 is the number of gems in our Gemfile.

I'm creating this issue for a few reasons:

1. To let you know that this is a problem that may be worth fixing
2. To hopefully get better error output implemented so that others that may hit this issue eventually know how to handle it (Make use of the error code thats returned?)
3. To ask for your input on a potential workaround. (Fork gemsurance and add some sleep/retry/rescue?)

Were now only running Gemsurance on our primary branch which has reduced the overall # of calls we were making by more than 90% but this is still an issue.

For some added context Ive asked RubyGems.org if our IP is 'blackholed' as they call it as well just to rule that out, waiting on a reply.

[nevesenin]

I'm interested in a workaround too.

[philCryoport]

I'm having this same issue. Glad to see that I'm not the only one getting C:/Ruby253-x64/lib/ruby/2.5.0/psych.rb:402:in parse': (): mapping values are not allowed in this context at line 13 column 15 (Psych::SyntaxError)`

[philCryoport]

FYI I've for now switched to bundler-audit

[paulmthiebauth]

Thanks for the recommendation @philCryoport

[magneland]

Is this still a problem? I wonder if the right solution is to check for HTTP 429 and sleep for a bit? https://guides.rubygems.org/rubygems-org-api/#rate-limits