Stack level too deep error with newrelic_rpm 8.8 and elastic-apm 4.5.1

[Intrepidd]

Description

gem "elastic-apm", "~> 4.5.1"
gem "newrelic_rpm", "~> 8.8"

When both those gems are activated, I get this error when trying to start my rails server.

#<Thread:0x00000001251e1a60 /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/threading/agent_thread.rb:11 run> terminated with exception (report_on_exception is true):
/Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:182:in `unicode_compose_pair': stack level too deep (SystemStackError)
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:153:in `block in unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:149:in `unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:121:in `unicode_normalize_kc'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:583:in `normalize_component'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1144:in `normalized_host'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1267:in `normalized_authority'
	 ... 9102 levels...
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:688:in `catch_errors'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:707:in `deferred_work!'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:734:in `block in start_worker_thread'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/threading/agent_thread.rb:13:in `block in create'
[ElasticAPM] [THREAD:74320]: APM Server not responding in time, terminating request

Expected Behavior

No crash.

Troubleshooting or [NR Diag]

I tried chaining or prepending many libs but to no avail.

Your Environment

Rails 7.0.3 newrelic_rpm 8.8 elastic-apm 4.5.1 ruby 3.1.2p20

[kaylareopelle]

Hi @Intrepidd! Thank you for your issue. I'm sorry to hear the stack overflow is happening despite toggling the chain/prepend configuration settings. Our Thread instrumentation is a newer feature, and it may possible be involved in this issue. I hope we can get to the bottom of this together!

I have a few questions to help us move forward:

• Was one of the configuration values you toggled instrumentation.httprb? It looks like addressable, a library mentioned in your stacktrace, is a dependency of elastic-apm by way of its http dependency (which we refer to inside the agent as httprb. I'd like to know what happens when you set instrumentation.httprb configuration value to chain.
• How do you start your Rails server? Do you have a Procfile that spawns multiple processes?
• What application server do you use? (puma, thin, unicorn, etc.)
• When you comment out elastic-apm and try to start your server, does the problem go away?

[Intrepidd]

Thanks for coming back !

Was one of the configuration values you toggled instrumentation.httprb? It looks like addressable, a library mentioned in your stacktrace, is a dependency of elastic-apm by way of its http dependency (which we refer to inside the agent as httprb. I'd like to know what happens when you set instrumentation.httprb configuration value to chain.

Yes I did, the error seems to be the same, just in case here it is :

=> Booting Puma
=> Rails 7.0.3 application starting in development
=> Run `bin/rails server --help` for more startup options
`Redis.current=` is deprecated and will be removed in 5.0. (called from: /Users/adrien/Work/ecotable/config/initializers/redis.rb:10:in `<main>')
Exiting
#<Thread:0x0000000127f4aba0 /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/threading/agent_thread.rb:11 run> terminated with exception (report_on_exception is true):
/Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:182:in `unicode_compose_pair': stack level too deep (SystemStackError)
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:153:in `block in unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:149:in `unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:121:in `unicode_normalize_kc'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:583:in `normalize_component'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1144:in `normalized_host'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1267:in `normalized_authority'
	 ... 9102 levels...
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:688:in `catch_errors'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:707:in `deferred_work!'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:734:in `block in start_worker_thread'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/threading/agent_thread.rb:13:in `block in create'
/Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:182:in `unicode_compose_pair': stack level too deep (SystemStackError)
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:153:in `block in unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:149:in `unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:121:in `unicode_normalize_kc'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:583:in `normalize_component'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1144:in `normalized_host'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1267:in `normalized_authority'
	 ... 9102 levels...
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:688:in `catch_errors'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:707:in `deferred_work!'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/agent.rb:734:in `block in start_worker_thread'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/newrelic_rpm-8.8.0/lib/new_relic/agent/threading/agent_thread.rb:13:in `block in create'
/Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:182:in `unicode_compose_pair': stack level too deep (SystemStackError)
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:153:in `block in unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:149:in `unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:121:in `unicode_normalize_kc'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:583:in `normalize_component'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1144:in `normalized_host'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1267:in `normalized_authority'
	 ... 9104 levels...
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/railties-7.0.3/lib/rails/commands.rb:18:in `<main>'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/bootsnap-1.12.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/bootsnap-1.12.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
	from bin/rails:4:in `<main>'
/Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:182:in `unicode_compose_pair': stack level too deep (SystemStackError)
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:153:in `block in unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/activesupport-7.0.3/lib/active_support/core_ext/range/each.rb:9:in `each'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:149:in `unicode_compose'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/idna/pure.rb:121:in `unicode_normalize_kc'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:583:in `normalize_component'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1144:in `normalized_host'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/addressable-2.8.0/lib/addressable/uri.rb:1267:in `normalized_authority'
	 ... 9104 levels...
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/railties-7.0.3/lib/rails/commands.rb:18:in `<main>'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/bootsnap-1.12.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
	from /Users/adrien/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/bootsnap-1.12.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
	from bin/rails:4:in `<main>'

How do you start your Rails server? Do you have a Procfile that spawns multiple processes?

Just doing rails s in this example, also trying to launch a thor task yields a similar issue.

What application server do you use? (puma, thin, unicorn, etc.)

Puma 5.6.4

When you comment out elastic-apm and try to start your server, does the problem go away?

Yes indeed !

Unfortunately though I could not manage to reproduce the issue on a fresh rails install even with exactly the same gemfile as my app

[kaylareopelle]

Thanks for your swift responses! At this point, I believe New Relic Support would be the best place to take next steps on this issue.

This will allow us to access more secure information like Gemfiles and agent logs to help diagnose the problem.

You can engage the New Relic Support Community or open a support ticket, depending on your support level. Please provide a link to this GitHub issue when submitting your community post or support ticket and we can keep following things there.

[fallwith]

Hello @Intrepidd - just checking in from New Relic's side to see if you've had a chance to look over @kaylareopelle's guidance above and engage with our support team. Please let us know if you need any assistance. We'd very much like to help further to figure out what was causing the issues.

[Intrepidd]

Hi,

Haven't had the chance for now, I'm not a newrelic customer, I wanted to try it out but ran into this issue which discouraged me to go further at the moment. if and when I find some time to consider it again I will open a support ticket if I still see this issue

[workato-integration[bot]]

https://issues.newrelic.com/browse/NEWRELIC-3435

[kaylareopelle]

Closing this ticket as we're not actively working on it. If you find this issue experiencing something similar, please feel free to re-open.