sentry

sentry copied to clipboard

Environment

SaaS (https://sentry.io/)

Version

No response

Steps to Reproduce

We've set up JavaScript source maps via "@sentry/webpack-plugin": "1.18.8". Specifically, our config looks as follows:

config.plugins.push(
      new SentryCliPlugin({
        include: './build',
        ignore: ['node_modules', 'webpack.config.js'],
        release: process.env.REACT_APP_VERSION,
        project: 'web',
        org: 'glean-gx',
        setCommits: {
          auto: true,
          ignoreEmpty: true,
          ignoreMissing: true,
        },
      })
    )

Expected Result

Sourcemapping works always

Actual Result

It appears to be working in the majority of cases - we have errors being successfully grouped together under the same issue across multiple releases/source maps.

For a handful of events, however, we are seeing 'invalid location in sourcemap' errors (see screenshots). I've followed the troubleshooting link and run the sentry-cli diagnostics for several examples - they're all of a similar flavour:

$ sentry-cli sourcemaps explain 9d553213bb6c4dfe9d18c0a275867dd1
✔ Fetched data for event: 9d553213bb6c4dfe9d18c0a275867dd1
✔ Event has release name: 1.0.37491
✔ Event has a valid exception present
✔ Event has a valid stacktrace present
⚠ Exception is already source mapped and resolves to:

    private readonly audioSourceNode: MediaElementAudioSourceNode
  
    constructor(audioContext: AudioContext, endedCallback: () => void = () => undefined) {
      this.htmlAudioElement = AudioElement.createAudioElement()
      this.htmlAudioElement.onended = endedCallback
>     this.audioSourceNode = audioContext.createMediaElementSource(this.htmlAudioElement)
    }
  
    private static createAudioElement(): HTMLAudioElement {
      const audio = document.createElement('audio')
      audio.style.display = 'none'

I don't know what this is trying to tell me, or where to go from here. I'm guessing the sourcemaps explain is a relatively new feature? It's a lovely idea - hopefully getting to the bottom of whatever our issue is can help to make it better :smile:

If you want to look at a specific example, I think all the details you need are:

Org: glean-gx Project: web Example issue IDs: 9d553213bb6c4dfe9d18c0a275867dd1, af482225b719460f9488042d83a2f4f7

Routing to @getsentry/owners-native for triage. ⏲️

Hey @alyssa-glean, thanks for the feedback. I do agree that the messaging here needs to be better, as right now, while technically correct, it doesn't provide much value.

In the case of a message like this, it means that we were able to resolve the sourcemap (as in line/col and filename was valid), but what has been uploaded is malformed, which means that the end result will not be correct.

This is usually caused by invalid webpack configuration, out-of-sync deployment process (eg. uploading artifacts to Sentry, and then deploying something slightly modified), or using out-of-date release name for previously updated files (eg. forgetting to update a release name in the codebase, producing new code release and uploading those files). Or simply a bug in the 3rd party tool (I remember Safari had issues with skipping comments while reporting line/number and terser had a bug with merging multiple functions together).

What you usually need to do is make sure that files you have locally produce a valid sourcemaps in the first place. You can use the CLI to verify it as well sentry-cli sourcemaps resolve path/to/file.js -c <col> -l <line>. If that's not working, there is nothing that Sentry could do to resolve it correctly on our servers.

I'll try to think of a better message and write something down on how to debug such issues.

Thanks for the response! I'll have a dig on our end to see if I can figure out what's wrong with our sourcemaps - the sentry-cli sourcemaps resolve command you gave me also doesn't fails with some more info, so that's given me a starting point.

In the meantime, is there a nice way for us to filter in/out issues in Sentry which encountered processing errors? It would be nice for our engineers to not stumble across these day to day whilst I work to fix it, because any information they enter into them is likely to be lost (the logs aren't aggregating properly, so we're ending up with new issues each day rather than a central issue we can keep up to date).

It would also be nice for me to be able to filter to just ones with errors, so I can see the scale of the problem. If it's only a couple of specific types of error that are encountering this problem, then we may decide to work around it via issue grouping or similar :smile:

In the meantime, is there a nice way for us to filter in/out issues in Sentry which encountered processing errors?

Sorry, unfortunately not; https://docs.sentry.io/product/sentry-basics/search/searchable-properties/

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

I also see the same exact issue. In fact, I found this on Github when searching for "Exception is already source mapped". And sentry-cli sourcemap explain gives the same result.

I'm certain there're no modifications after the build or deployment. The existing releases don't get updated, the new release is created. I tried sentry-cli sourcemap resolve and it resolves the code lines successfully.

We're using somewhat outdated webpack v4.46.0 and babel v7.4.3, could this be the reason?

@a-martynovich can you share a link to the event?

I have the same problem as you @a-martynovich. If you find something please let me know :)

Can this have something to do with this 👇 (https://docs.sentry.io/platforms/javascript/sourcemaps/generating/) @kamilogorek ?

To rely on Sentry's source map resolution, your code cannot use the source-map-support package. That package overwrites the captured stack trace in a way that prevents our processors from correctly parsing it.

In our case, we're using create-react-app, and if I execute npm ls source-map-support I get this:

[email protected] /Users/petercatalin/Projects/checkout-widget └─┬ [email protected] ├─┬ [email protected] │ └─┬ [email protected] │ └─┬ [email protected] │ └── [email protected] deduped ├─┬ [email protected] │ └─┬ [email protected] │ └── [email protected] └─┬ [email protected] └─┬ [email protected] └── [email protected] deduped

So the package is there. Could this be causing the problem?

We're also using create-react-app (also 4.0.3), so source-map-support is in our dependency tree too.

@alyssa-glean most of your frames are resolved correctly. The message from cli is correct, as the event is mapped, however your sourcemaps appear to be somehow malformed.

Notice that most frames that come from the main.2d401a2d.chunk.js file are pointing to valid places in your codebase. Sometimes its caused by overoptimizations of terser/uglify, or some other processing step in your webpack/rollup/other tool.

It's almost impossible to tell what went wrong. When you use sentry-cli sourcemaps resolve <filename> --line <line> --col <column>, for example sentry-cli sourcemaps resolve main.2d401a2d.chunk.js --line 1 --column 656166 locally, and it doesn't resolve correctly, there's nothing that our servers can do to change that. Thus, if it doesn't work locally, it wont work during processing as well.

@peter-catalin can you share link to an event which is not resolved correctly by any chance?

Of course, @kamilogorek.

The id of the event is: d9bf6bd6a2a5452abcce210dcee37523

Thanks!

So in our case we had

optimization: {
  // ...
  minify: false
}

in our webpack.babel.js. This was the only reason and changing it to true solved the problem.

@a-martynovich can you share a link to the event?

@kamilogorek I don't think I can, this is a private project.

@a-martynovich @peter-catalin I can confirm that we had multiple issues with over-optimizations in the past. They are breaking produced sourcemaps, and there's nothing that we can do to resolve them once they are malformed :/

See: https://github.com/getsentry/sentry-javascript/issues/2701#issuecomment-656592209 https://github.com/getsentry/sentry-javascript/issues/2550#issuecomment-861667668 https://github.com/getsentry/sentry-javascript/issues/3251#issuecomment-786681485

Ok @kamilogorek, thanks anyway for taking the time to look into it :)

@kamilogorek we do see the same behavior - most source maps work fine but i'd say around 20% of the issues say Invalid location in source map. However, in our case the output of sentry-cli sourcemaps outputs more strange stuff:

✔ Fetched data for event: e1bd9857e42c4876a8c9075fc9508d8a
✔ Event has release name: [email protected]
✔ Event has a valid exception present
✔ Event has a valid stacktrace present
⚠ Exception is already source mapped and resolves to:


> 

This is a self-hosted sentry instance, so i can't give you a link to the event - if you have a tip on how to get more info on that issue, i'd be happy to help.

@pulse00 if it's self-hosted, would it be maybe possible to provide just an event json file and the sourcemap file that was used for resolution? You can find both on the events page. JSON file here:

and artifacts on release page.

This has resolved itself for us - I think the most likely explanation is that we did a piece of work recently to eject from Create React App and have since been upgrading dependency versions and simplifying/removing some of the bloat that we didn't need.

So we've managed to stamp on whatever problematic combination of settings/packages was giving us problems, but unfortunately I can't pinpoint exactly when it was fixed or what the problem was still.

Thanks for the report @alyssa-glean. I'll let myself close this issue then, but feel free to ping me if you find more details :)