sentry-java

sentry-java copied to clipboard

Description

From play console:

Exception java.lang.OutOfMemoryError:
  at java.lang.Thread.nativeCreate
  at java.lang.Thread.start (Thread.java:883)
  at java.util.Timer.<init> (Timer.java:183)
  at java.util.Timer.<init> (Timer.java:153)
  at io.sentry.SentryTracer.<init> (SentryTracer.java:105)
  at io.sentry.Hub.createTransaction (Hub.java:727)
  at io.sentry.Hub.startTransaction (Hub.java:677)
  at io.sentry.Sentry.startTransaction (Sentry.java:892)
  at io.sentry.HubAdapter.startTransaction (HubAdapter.java:202)
  at io.sentry.android.core.ActivityLifecycleIntegration.startTracing (ActivityLifecycleIntegration.java:227)
  at io.sentry.android.core.ActivityLifecycleIntegration.onActivityCreated (ActivityLifecycleIntegration.java:358)
  at android.app.Application.dispatchActivityCreated (Application.java:379)
  at android.app.Activity.dispatchActivityCreated (Activity.java:1263)
  at android.app.Activity.onCreate (Activity.java:1536)
  at androidx.core.app.ComponentActivity.onCreate (ComponentActivity.java:88)
  at androidx.activity.ComponentActivity.onCreate (ComponentActivity.java:378)
  at androidx.fragment.app.FragmentActivity.onCreate (FragmentActivity.java:217)
  at <private>
  at android.app.Activity.performCreate (Activity.java:7866)
  at android.app.Activity.performCreate (Activity.java:7855)
  at android.app.Instrumentation.callActivityOnCreate (Instrumentation.java:1306)
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:3273)
  at android.app.ActivityThread.handleLaunchActivity (ActivityThread.java:3437)
  at android.app.servertransaction.LaunchActivityItem.execute (LaunchActivityItem.java:83)
  at android.app.servertransaction.TransactionExecutor.executeCallbacks (TransactionExecutor.java:135)
  at android.app.servertransaction.TransactionExecutor.execute (TransactionExecutor.java:95)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:2041)
  at android.os.Handler.dispatchMessage (Handler.java:107)
  at android.os.Looper.loop (Looper.java:214)
  at android.app.ActivityThread.main (ActivityThread.java:7386)
  at java.lang.reflect.Method.invoke
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:492)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:982)

Is there a new spike of these or could they just be coming from devices with very limited resources? In other words should we investigate any newly introduced increase in memory consumption?

Let's start simple:

• Integrate our SDK into an OSS app
• check number of threads / memory consumed by SDK

A heap analysis done by a customer shows Span objects holding large stacktraces (preventing GC) for a timeframe of over 2+ hours. This definitely doesn't sound right.

After a quick test with our sentry-springboot example I can't seem to reproduce this easily. Scenario:

• Use sentry-springboot example
• Capture an exception on every request

  @PostMapping
 Person create(@RequestBody Person person) {
   Sentry.captureException(new IllegalStateException("oh no"));
   return personService.create(person);
 }

• Flood the service with 1k requests

i=0
max=1000
while [ $i -lt $max ]
do
   curl -XPOST --user user:password http://localhost:8080/person/ -H "Content-Type:application/json" -d '{"firstName":"John","lastName":"Smith"}'
   true $(( i++ ))
   sleep 2
done

• Analyze the heap using mat

Tried a few variations as well, same picture: no leaking spans.

1. Throwing an uncaught exception

@PostMapping
Person create(@RequestBody Person person) {
  throw new IllegalStateException("oh no");
  // return personService.create(person);
}

2. Attaching a stack trace to a span

@PostMapping
Person create(@RequestBody Person person) {
  final ISpan span = Sentry.getSpan().startChild("op.bad");
  span.setThrowable(new RuntimeException("bad things happened"));
  span.finish(SpanStatus.UNKNOWN_ERROR);

  return personService.create(person);
}

@lbloder we recently got a report from a sentry-java springboot customer regarding spans with stacktraces leaking and leading to OOMs. I tried a few basics (See comments ^ above). Feel free to chime in if you have any more ideas what could cause spans to be retained.

Some more details from our customer setup:

• They do not have any integrations active
• They don’t use any extra manual instrumentation
• They utilise Sentry across their spring-boot microservices which are all hosted on AKS servers
• These services all communicate via REST APIs and they receive calls from a browser-based UI.
• They noticed the usage by performing a heap-dump analysis.
• They note “Sentry spans hold the exception object and blocks the garbage collection of these exception objects even after 2+ hours.”

I'm wondering if this could be related to some background workers having long running transactions.

@markushi I'm a bit confused on They do not have any integrations active:

• How are they using Sentry?
  • My guess is, they are adding the sentry-spring-boot dependency manually, is that correct?
  • Or are they using the gradle or maven plugin for installation?
  • Or are they using the vanilla sentry-java sdk?

Also, it might be of relevance if they are using SpringBoot < 3 or not. And, one more thing that could help us narrow down the search space is whether they are using WebFlux.

I'll have a look into that as well.

@lbloder thanks for the details, let me reach out to them!

it occurred also in CpuCollector on Android