rollbar-gem icon indicating copy to clipboard operation
rollbar-gem copied to clipboard

2.24.0 hangs in http post, 2.23.2 is still OK

Open spjsschl opened this issue 5 years ago • 4 comments

We observe that the rollbar notifier for version 2.24.0 hangs reproducibly in http post requests set up in notifier.rb line 581.

The exact same request works with 2.32.2.

I looked through the relevant changes and could not figure out what change could have caused this; mot changes seem to relate to the write to file feature, which is off for our use case.

Please let me know if I can help with further information.

spjsschl avatar Mar 11 '20 15:03 spjsschl

@spjsschl Thank you for the report. I'd like to reproduce this, but haven't seen it in testing here. Can you let me know your ruby version, framework and version (e.g. Rails 5.2), and the method and args you're using to call Rollbar-gem. (Or is it an uncaught exception?)

waltjones avatar Mar 11 '20 16:03 waltjones

Hi Walt, when trying to get the issue reproducible and with a minimal example, I found that some of my scripts earlier were buggy due to some memoized state and confusing debug output. Matter of fact the issue was not introduced after 2.32.2 but rather after 2.16.3 and is not related to the request hanging but to rollbar losing the access_token set on the notifier when sending exceptions async (causing a 401 if no valid Rollbar.access_token is set). Here are 2 minimal testcases:


def func1
  meta =  { title: 'Foo/bar',
            fingerprint: 'Foo/bar',
            class: 'Bar',
            method_name: 'bar',
            retry_counts: 3,
            environment: 'redacted',
            git_branch: 'redacted',
            commit_hash: 'redacted',
            build_config: 'redacted',
            link_to_build: 'redacted'
  }
  notifier = Rollbar.scope(meta)
  { verify_ssl_peer: false, access_token: 'redacted', use_async: true, enabled: true  }.each do |key, value|
    notifier.configuration.send("#{key}=", value)
  end
  notifier.error(RuntimeError.new('hi'))
end

def func2
  meta =  { title: 'Foo/bar',
            fingerprint: 'Foo/bar',
            class: 'Bar',
            method_name: 'bar',
            retry_counts: 3,
            environment: 'redacted',
            git_branch: 'redacted',
            commit_hash: 'redacted',
            build_config: 'redacted',
            link_to_build: 'redacted'
  }
  notifier = Rollbar.scope(meta)
  { verify_ssl_peer: false, access_token: 'redacted', use_async: false, enabled: true  }.each do |key, value|
    notifier.configuration.send("#{key}=", value)
  end
  notifier.error(RuntimeError.new('hi'))
end

Note the only difference is the use_async flag.

in version 2.16.3, both func1 and func2 report to rollbar successfully. in version 2.24.0, func1 gets a 401, func2 works.

Sorry about the initial confusion. We are using rail 5.2.2.1 and ruby 2.6.3 but I think that's unrelated :-).

spjsschl avatar Mar 11 '20 18:03 spjsschl

my hunch is this commit could be responsible: 121a2ac13c5c2f7b60b3c2cef5c17cceaed97e7c it seems to effectively disable options for async requests

spjsschl avatar Mar 11 '20 20:03 spjsschl

That should only affect the copy of the options passed in the diagnostic part of the payload. (notifier.configured_options) But maybe there's an unintended side effect. If so, it would also fail for all ActiveSupport < 4.1. I think Travis builds would fail in that case.

waltjones avatar Mar 11 '20 20:03 waltjones