The INP parameter is not displayed on the dashboard if the tunnel option is active

[andreamaiolo]

Is there an existing issue for this?

• [X] I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• [X] I have reviewed the documentation https://docs.sentry.io/
• [X] I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/browser

SDK Version

7.110.1

Framework Version

React 18.2.0

Link to Sentry event

No response

SDK Setup

No response

Steps to Reproduce

When browsing a page with the tunnel option enabled, our server receives envelopes which we then POST to Sentry.

Expected Result

We expect all information that our server processes to be displayed on the dashboard, including the INP parameter.

This is, for example, an envelope that we receive on the server and POST it to Sentry, it gets lost.

{"sent_at":"2024-04-17T08:02:11.023Z","dsn":"https://xxx.ingest.sentry.io/xxx"}
{"type":"span"}
{"data":{"sentry.origin":"manual","sentry.op":"ui.interaction.click","release":"unknown","environment":"twbox","transaction":"/places"},"description":"<unknown>","op":"ui.interaction.click","span_id":"8a7171204d3da7bc","start_timestamp":1713340917.8902,"timestamp":1713340918.1701999,"trace_id":"ff475ce1d41b4f32955fb9e44e5fd749","origin":"manual","exclusive_time":280,"measurements":{"inp":{"value":280,"unit":"millisecond"}}}

Another thing is that the description is <unknown>, not sure why.

Actual Result

No INP information is shown, as in the screenshot:

[lforst]

How did you configure your tunnel? What does it look like, and to which exact url/dsn does it forward the events?

[andreamaiolo]

This is our tunnel:

import { SENTRY_HOST, SENTRY_PROJECT } from 'js/constants/sentry';
import { Request, Response } from 'express';
import { httpPost } from 'js/common/http';

class SentryDsnError extends Error {
  constructor(message: string) {
    super(message);

    this.name = 'SentryDsnError';
  }
}

const getDsn = (envelope: string) => {
  const piece = envelope.slice(0, envelope.indexOf('\n'));
  const header = JSON.parse(piece);
  const dsn = new URL(header.dsn);

  if (dsn.hostname !== SENTRY_HOST) {
    throw new SentryDsnError(`Invalid Sentry host: ${dsn.hostname}`);
  }

  return dsn;
};

const getProjectId = (dsn: URL) => {
  const projectId = dsn.pathname.substring(1);

  if (projectId !== SENTRY_PROJECT) {
    throw new SentryDsnError(`Invalid Project ID: ${projectId}`);
  }

  return projectId;
};

export const sentryTunnel = async (req: Request, res: Response) => {
  const envelope = req.body;

  try {
    const dsn = getDsn(envelope);
    const projectId = getProjectId(dsn);
    const url = `https://${SENTRY_HOST}/api/${projectId}/envelope/`;

    await httpPost(url, {
      dataRaw: envelope,
      requestHeaders: {
        'Content-Type': 'application/x-sentry-envelope',
      },
    });

    return res.sendStatus(204);
  } catch (e) {
    if (e instanceof SentryDsnError) {
      return res.status(400).send({ message: e.message });
    }

    return res.sendStatus(204);
  }
};

DSN: https://[email protected]/xxx URL: https://xxxx.ingest.sentry.io/api/xxx/envelope/

We only encounter the problem when this package is handled:

{"sent_at":"2024-04-17T08:02:11.023Z","dsn":"https://[email protected]/xxx"}
{"type":"span"}
{"data":{"sentry.origin":"manual","sentry.op":"ui.interaction.click","release":"unknown","environment":"twbox","transaction":"/places"},"description":"<unknown>","op":"ui.interaction.click","span_id":"8a7171204d3da7bc","start_timestamp":1713340917.8902,"timestamp":1713340918.1701999,"trace_id":"ff475ce1d41b4f32955fb9e44e5fd749","origin":"manual","exclusive_time":280,"measurements":{"inp":{"value":280,"unit":"millisecond"}}}

In all other cases, the tunnel works and we see data on the dashboard

[lforst]

As a last bit of information, would you mind sharing a link to the project where you would expect events to show up? Thank you! In the meanwhile I have forwarded this internally and we're taking a look.

[andreamaiolo]

Thanks for the support. I'd expect to see results here: https://xxx.sentry.io/performance/browser/pageloads/overview/?environment=twbox&project=xxx&statsPeriod=24h&transaction=%2Fplaces&type=pageloads

I did some testing this morning with @sentry/browser version 7.110.1 on a test environment.

Otherwise in production we have version 7.106.1 where no INP data has ever been received: https://xxx.sentry.io/performance/browser/pageloads/overview/?environment=prod&project=xxx&statsPeriod=24h&transaction=%2Fplaces&type=pageloads

[Dav1dde]

And the INP data shows up without a tunnel or did you never test it without a tunnel?

[andreamaiolo]

That's right, by removing the tunnel, we get INP data: https://treatwell-tech.sentry.io/performance/browser/pageloads/overview/?environment=twbox&project=6485152&statsPeriod=24h&transaction=%2F&type=interactions

[Dav1dde]

Thanks! Sorry for the pesky questions, you don't also happen to have the event payload for the working integration without the tunnel? You should be able to grab it from the browser dev console.

[andreamaiolo]

Here it is:

{"sent_at":"2024-04-17T11:04:16.872Z","dsn":"https://[email protected]/xxx"}
{"type":"span"}
{"data":{"sentry.origin":"manual","sentry.op":"ui.interaction.click","release":"unknown","environment":"twbox","transaction":"/"},"description":"input.InputField-module--input--ba477c[type=\"text\"]","op":"ui.interaction.click","span_id":"843885d36ebc5255","start_timestamp":1713351813.4854,"timestamp":1713351813.8453999,"trace_id":"a50c31e8808b46cda25213ebda37ffa8","origin":"manual","exclusive_time":360,"measurements":{"inp":{"value":360,"unit":"millisecond"}}}

https://xxx.sentry.io/performance/browser/pageloads/overview/?environment=twbox&project=xxx&statsPeriod=24h&transaction=%2F&type=interactions

[lforst]

@andreamaiolo Are you sure the latest message is without tunnel? Usually, the payload doesn't include a dsn property when the event is not tunneled.

[andreamaiolo]

Yes, it should be without the tunnel. This is the cURL copied from Chrome's devtools (without the headers that I've removed):

curl 'https://xxx.ingest.sentry.io/api/xxx/envelope/?sentry_key=xxx&sentry_version=7&sentry_client=sentry.javascript.browser%2F7.110.1' \
  --data-raw $'{"sent_at":"2024-04-17T11:04:16.872Z","dsn":"https://[email protected]/xxx"}\n{"type":"span"}\n{"data":{"sentry.origin":"manual","sentry.op":"ui.interaction.click","release":"unknown","environment":"twbox","transaction":"/"},"description":"input.InputField-module--input--ba477c[type=\\"text\\"]","op":"ui.interaction.click","span_id":"843885d36ebc5255","start_timestamp":1713351813.4854,"timestamp":1713351813.8453999,"trace_id":"a50c31e8808b46cda25213ebda37ffa8","origin":"manual","exclusive_time":360,"measurements":{"inp":{"value":360,"unit":"millisecond"}}}'

[lforst]

Ok that's weird. I can't find the place in our code that would set the dsn property, unless tunnel is configured.

Anyhow, let's wait until the team wakes up that built this feature to take a look!

[lforst]

@andreamaiolo Would you mind double checking whether your proxy actually forwards the requests? And check the response from Sentry?

[lforst]

Okay so I just set tried this with and without using a tunnel and I am getting INP events for both. Please provide a reproduction example for us to verify. Thank you!

[andreamaiolo]

@andreamaiolo Would you mind double checking whether your proxy actually forwards the requests? And check the response from Sentry?

I confirm that the request is forwarded and that Sentry replies with an empty object

[lforst]

@andreamaiolo since I could not reproduce this, would you mind sharing a reproduction example? Thank you!

[andreamaiolo]

I'll try to make a simple use case to share. But in general, how come the other events are handled and only the one related to the INP is lost? I don't exclude any implementation flaws on our side, but using the tunnel, should we have any special considerations for handling the INP? Thanks

[lforst]

Honestly, I also find this very weird. Fwiw, I think something is wrong in our infrastructure, or routing logic, but I would like to really rule out that the tunnel is working since that is a common point of failure.

I think the ideal would be for you to log exactly what you would be sending from the tunnel to Sentry, so we can "replay" it and see if it really doesn't make it through the infra. Does that make sense?

[andreamaiolo]

Apologies for the late reply!

This is what our server receives from the SDK:

{"sent_at":"2024-04-29T15:22:53.114Z","dsn":"https://[email protected]/xxx"}
{"type":"span"}
{"data":{"sentry.origin":"manual","sentry.op":"ui.interaction.click","release":"unknown","environment":"twbox","transaction":"/"},"description":"div.TabBarItem-module--item--35b4f0.TabBarItem-module--active--c530dd","op":"ui.interaction.click","span_id":"8b54d091330fc52c","start_timestamp":1714404171.7855,"timestamp":1714404172.0255,"trace_id":"d0e7c002321d41c3a863a21a6d5b65c6","origin":"manual","exclusive_time":240,"measurements":{"inp":{"value":240,"unit":"millisecond"}}}

This, is what we send to Sentry:

{"sent_at":"2024-04-29T15:22:53.114Z","dsn":"https://[email protected]/xxx"}
{"type":"span"}
{"data":{"sentry.origin":"manual","sentry.op":"ui.interaction.click","release":"unknown","environment":"twbox","transaction":"/"},"description":"div.TabBarItem-module--item--35b4f0.TabBarItem-module--active--c530dd","op":"ui.interaction.click","span_id":"8b54d091330fc52c","start_timestamp":1714404171.7855,"timestamp":1714404172.0255,"trace_id":"d0e7c002321d41c3a863a21a6d5b65c6","origin":"manual","exclusive_time":240,"measurements":{"inp":{"value":240,"unit":"millisecond"}}}

The payloads are identical! I am worried that, given the use of the tunnel, we are missing important information on Sentry, or can we exclude this scenario?

I don't know if it helps, but this is how we mount the express middleware for the /sentry-tunnel route:

app.post(
    '/sentry-tunnel',
    express.raw({ limit: '1mb', type: () => true }),
    sentryTunnel
);

Is there anything specific we can do to facilitate debugging?

[lforst]

That is super odd, I am gonna bug our ingestion folks some more.

[lforst]

Also, what response are you getting from Sentry?

[andreamaiolo]

The status code is 200 and the body is an empty object

[lforst]

Hi, sorry for the delay. We are currently very much in the weeds of a new major release (v8) and will pick this up once the update is out.

It would be really cool if you could create an isolated (minimal) reproduction example that we can use to replicate. I am sorry that this is so much back and forth.

[Roberto-Surfboard]

Hi, Im having a similar situation, the INP is being sent to our endpoint and then sent to Sentry, using the tunnel. I can see the INP value in the body when I send the value to Sentry (receive a 200, everything seems ok), but I cant see the INP value in the dashboard. There is any update on this?

[lforst]

I keep forwarding this internally but we are a bit stuck trying to figure out why some people experience this. @Roberto-Surfboard would you mind sharing a cURL copy similar to what Andrea did here: https://github.com/getsentry/sentry-javascript/issues/11646#issuecomment-2061070605 Thanks!

[Roberto-Surfboard]

I keep forwarding this internally but we are a bit stuck trying to figure out why some people experience this. @Roberto-Surfboard would you mind sharing a cURL copy similar to what Andrea did here: #11646 (comment) Thanks!

From our backend I can see the request (POST) to: https://xxxxxxx.ingest.sentry.io/api/xxxxxx/envelope/

With the body: {"data":{"environment":"staging","profile_id":"xxxxxxx","release":"xxxxxx","sentry.op":"ui.interaction.click","sentry.origin":"manual","transaction":"/schedule"},"description":"<unknown>","exclusive_time":128,"measurements":{"inp":{"unit":"millisecond","value":128}},"op":"ui.interaction.click","origin":"manual","profile_id":"xxxxxx","span_id":"xxxxxx","start_timestamp":1717170040.4727,"timestamp":1717170040.6007,"trace_id":"xxxxxx"}

I hided all the ids I see in the url/body. Let me know if you need more information @lforst Thanks!

[lforst]

@Roberto-Surfboard I think there are the first two lines missing from the body. Can you share those two?

[Roberto-Surfboard]

Hi @lforst I checked my logs and that is the full body. Im sending the body from my server using: const sentryResp = await fetch(upstreamSentryUrl, { method: 'post', body: envelope, headers: { 'Content-Type': 'application/text' }, });

where envelope is {"data":{"environment":"staging","profile_id":"xxxxxxxx","release":"xxxxx","sentry.op":"ui.interaction.click","sentry.origin":"manual","transaction":"/schedule"},"description":"<unknown>","exclusive_time":128,"measurements":{"inp":{"unit":"millisecond","value":128}},"op":"ui.interaction.click","origin":"manual","profile_id":"xxxxxxx","span_id":"xxxxxxx","start_timestamp":1717170040.4727,"timestamp":1717170040.6007,"trace_id":"xxxxxx"}

and upstreamSentryUrl is https://xxxxxxx.ingest.sentry.io/api/xxxxxx/envelope/ The envelope is the object that I receive from the front end

Here is the body from the front end

[mydea]

Actually, what version of the SDK are you running? (sorry if that was mentioned before, there are a bunch of comments already... 😅 ) I think this was fixed in 8.3.0 only - are you on a version newer than that?

See https://github.com/getsentry/sentry-javascript/pull/12096

[Roberto-Surfboard]

@mydea 7.114, I updated to the latest version inside 7, because I didnt want to break things moving to 8. But in that case Im going to update to 8.3.0 and test. Thx

[andreamaiolo]

Hey @mydea, I've updated Sentry to v8.9.2 but the problem persists. I still point out that by deactivating the tunnel the INP is recorded and is visible on the dashboard, but with the tunnel active this doesn't happen. The problem is only specific to the INP, all other metrics are reported. Do you have any idea why this is happening? For the time being, we have disabled the tunnel in order to get information on this parameter, but the idea is to re-enable it. Thank you