sentry-cli icon indicating copy to clipboard operation
sentry-cli copied to clipboard
verified publisher icon getsentry

sentry-cli >= 2.21.3 does not work with rootless docker

Open Snack-X opened this issue 1 year ago • 35 comments

Environment

sentry-cli >= 2.21.3 + rootless docker

Steps to Reproduce

  1. Install Docker in rootless mode
  2. Pull some image (eg: python:3.12) and start new container (docker run --rm -it python:3.12 bash)
  3. Install sentry-cli >= 2.21.3 (curl -sL https://sentry.io/get-cli/ | SENTRY_CLI_VERSION=2.21.3 bash)
  4. Run sentry-cli info

Expected Result

No error

Actual Result

(failure on authentication: API request failed)

Logs

$ docker version
Client:
 Version:           25.0.2
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        29cf629
 Built:             Thu Feb  1 00:22:06 2024
 OS/Arch:           linux/amd64
 Context:           rootless

$ docker pull python:3.12
$ docker run --rm -it python:3.12 bash

# curl -sL https://sentry.io/get-cli/ | SENTRY_CLI_VERSION=2.21.3 bash
(snipped)
Installed sentry-cli 2.21.3
Done!

# sentry-cli --version
sentry-cli 2.21.3

# SENTRY_LOG_LEVEL=debug sentry-cli info
  DEBUG   2024-02-06 --:--:-- sentry-cli version: 2.21.3, platform: "linux", architecture: "x86_64"
  INFO    2024-02-06 --:--:-- sentry-cli was invoked with the following command line: "sentry-cli" "info"
  DEBUG   2024-02-06 --:--:-- request GET https://sentry.io/api/0/
  DEBUG   2024-02-06 --:--:-- retry number 0, max retries: 0

Sentry Server: https://sentry.io
Default Organization: -
Default Project: -

Authentication Info:
  Method: Unauthorized
  (failure on authentication: API request failed)

  INFO    2024-02-06 --:--:-- Running update nagger update check
  DEBUG   2024-02-06 --:--:-- request GET https://release-registry.services.sentry.io/apps/sentry-cli/latest
  DEBUG   2024-02-06 --:--:-- retry number 0, max retries: 0
  • Related with: #1843

While updating our nuxt web app's sentry module(@nuxtjs/sentry), we've also updated @sentry/webpack-plugin from 1.x.x to 2.x.x.

After this change, we started to see our CI/CD pipeline failing to publish releases and sourcemaps. The only error message we could see was error: API request failed caused by: [6] Couldn't resolve host name (Could not resolve host: sentry.io).

Few hours of tests and bisection narrowed down to combination of rootless docker (used in our Actions runner) and sentry-cli >= 2.21.3. Standard docker installation works just fine, and older version of sentry-cli also has no problem -- which is the fix we chose.

We tested multiple configurations of OS and architectures, which all had same result:

  • Ubuntu 22.04 on AWS EC2 t3 (x86_64)
  • Ubuntu 22.04 on AWS EC2 t4g (arm64)
  • Amazon Linux 2023 on AWS EC2 t3 (x86_64)
  • Ubuntu 22.04 on local VM (x86_64)
  • (plus Docker 23.x, 24.x, 25.x)

I think some low level change made between 2.21.2 and 2.21.3 affected how DNS resolution works and created some kind of incompatibility.

It could be an issue of docker, it could be both, I don't know at this point. I hope this information is helpful.

Snack-X avatar Feb 06 '24 06:02 Snack-X

@Snack-X thank you for the very detailed issue report!

It appears that we did make some changes to the compiler optimizations from version 2.21.2 to 2.21.3. However, we reverted one of the changes in 2.23.0 (commit 59fcf92) because it was causing problems when Sentry CLI was run in Xcode Cloud.

Have you tried running sentry-cli ≥2.23.0 in rootless Docker yet? If not, please try again with sentry-cli ≥2.23.0, since it is possible that the fix from 59fcf92 will fix your problem, too. Otherwise, if you still have trouble with sentry-cli ≥2.23.0, please let me know, and I will investigate further.

szokeasaurusrex avatar Feb 06 '24 08:02 szokeasaurusrex

Thanks for the response.

That version was found with bisection from 2.0.0 to 2.28.0, so I think I've already tried >= 2.23.0. But I'll check it again just in case.

Snack-X avatar Feb 06 '24 11:02 Snack-X

Okay, please let me know once you have tried ≥2.23.0 again and have confirmed whether the issue is still occurring in those versions

szokeasaurusrex avatar Feb 06 '24 12:02 szokeasaurusrex

I can confirm both 2.23.0 and latest (2.28.0) still have same issue.

# curl -sL https://sentry.io/get-cli/ | SENTRY_CLI_VERSION=2.23.0 bash
Installed sentry-cli 2.23.0
Done!

# sentry-cli --version
sentry-cli 2.23.0

# sentry-cli info
Sentry Server: https://sentry.io
Default Organization: -
Default Project: -

Authentication Info:
  Method: Unauthorized
  (failure on authentication: API request failed)

# curl -sL https://sentry.io/get-cli/ | SENTRY_CLI_VERSION=2.28.0 bash
Installed sentry-cli 2.28.0
Done!

# sentry-cli --version
sentry-cli 2.28.0

# sentry-cli info
Sentry Server: https://sentry.io
Default Organization: -
Default Project: -

Authentication Info:
  Method: Unauthorized
  (failure on authentication: API request failed)

Snack-X avatar Feb 07 '24 01:02 Snack-X

  • https://github.com/clux/muslrust/issues/122
  • https://github.com/alpinelinux/docker-alpine/issues/155 (also linked in issue above)

I've also found these issues which be related to this one, since sentry-cli is built with musl.

Looks like musl has problem with DNS resolving.

Snack-X avatar Feb 07 '24 01:02 Snack-X

Thank you for the information and for linking those issues! I will investigate further to see whether we can fix this somehow, or whether this is something that needs to be fixed in musl

szokeasaurusrex avatar Feb 07 '24 11:02 szokeasaurusrex

Thanks @Snack-X for debugging and linking to issue https://github.com/getsentry/sentry-cli/issues/1843, I can confirm my issue is the same :) The error message for me is thrown in a Gitlab pipeline, using the docker:dind-rootless image to build my docker image. I have pinned my sentry-cli version to 2.21.2 currently.

darthf1 avatar Feb 12 '24 08:02 darthf1

@Snack-X, I tried to reproduce your bug using a clean install of Ubuntu 22.04 LTS running the latest Docker, installed in rootless mode (following the Docker Docs instructions). However, everything appears to be working correctly on my end with sentry-cli version 2.28.0. Below are the specific commands I ran, along with the corresponding outputs:

root@3661a4367ba7:/# sentry-cli --version
sentry-cli 2.28.0

root@3661a4367ba7:/# sentry-cli info
Sentry Server: https://sentry.io
Default Organization: -
Default Project: -

Authentication Info:
  Method: Unauthorized

root@3661a4367ba7:/# sentry-cli info --auth-token=sntrys_[redacted]
Sentry Server: https://sentry.io
Default Organization: -
Default Project: -

Authentication Info:
  Method: Auth Token
  Scopes:
    - org:ci

Perhaps, you could try passing an auth token with --auth-token to see if you observe similar output to what I observed in the last command I ran? If you still are getting an error, when passing the --auth-token, would you please be able to provide me with more information about your specific setup, so I can try to reproduce the issue?

szokeasaurusrex avatar Feb 12 '24 11:02 szokeasaurusrex

In my situation, I'm building a docker image in a Gitlab CI pipeline.

Moving from

docker:25.0.3-dind-rootless@sha256:a67a4f149cd62d8c2023778ddd65e82e3895b21c10b8a5a19fd18b3dd1c3c96a

to

docker:25.0.3-dind@sha256:915cd1624f521b6337f135075f712c8fb14c0b151595c6144d7ce05d2f257869

resolves my issue in https://github.com/getsentry/sentry-cli/issues/1843.

@szokeasaurusrex the docker:25.0.3-dind image has alpine as base and not ubuntu. Maybe thats the difference why you cannot reproduce with Ubuntu 22.04 LTS?

darthf1 avatar Feb 16 '24 12:02 darthf1

Hi @darthf1, I tried to run the docker:25.0.3-dind-rootless image, but was getting errors on my system. I was, however, able to test against the alpine Docker image, but everything appeared to work as expected (I observed the same results as in my previous comment).

the docker:25.0.3-dind image has alpine as base and not ubuntu. Maybe thats the difference why you cannot reproduce with Ubuntu 22.04 LTS?

Ubuntu 22.04 LTS is the operating system of the machine where I have installed the Docker daemon, not the operating system of the Docker containers. So, I don't think this explains the difference in behavior I observed compared with what you and @Snack-X are observing.

Unfortunately, I will only be able to help with resolving this issue if I am able to reproduce your problem, and in order to do that, I will need a detailed list of the specific commands you ran to set up a Docker container where you observe this bug, and a list of commands to run in the Docker container to observe the result.

szokeasaurusrex avatar Feb 16 '24 13:02 szokeasaurusrex

I'm affected by this issue (or a similar one), intermittently failing to resolve sentry.io while running Docker builds in a particular hosted environment:

[my-app]       my-app:build: Error: [sentry-debug-id-upload-plugin] Command failed: /usr/src/app/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli releases finalize dab5a9dd58166b08213e432181608e7e
[my-app]       my-app:build: error: API request failed
[my-app]       my-app:build:   caused by: [6] Couldn't resolve host name (Could not resolve host: sentry.io)

I'm actually using @sentry/vite-plugin, but the problem goes away when I pin the underlying @sentry/cli version like so:

  "pnpm": {
    "overrides": {
      "@sentry/cli": "2.21.2"
    }
  }

The environment is BalenaCloud's build server.

One question, as I try to create a repro: is there a good way to exercise @sentry/cli's DNS access, without supplying account credentials? It'd be really helpful if I could run something like sentry-cli --test-api-connection in a public repo without worrying about a token.

Update - I've now seen it happen with 2.21.2 as well:

[my-app]       my-app:build: Error: [sentry-debug-id-upload-plugin] Command failed: /usr/src/app/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli/sentry-cli releases finalize 7a200f797cbcd93e356691aaaf1353fc
[my-app]       my-app:build: error: API request failed
[my-app]       my-app:build:   caused by: [6] Couldn't resolve host name (Could not resolve host: sentry.io)

jrr avatar Mar 20 '24 23:03 jrr

@jrr You can run sentry-cli update. This command pings the Sentry server to determine the latest sentry-cli release, but it does not require authentication.

Please let me know if you have a reproduction; as I wrote in my previous comment, I was struggling to reproduce this issue, and will only be able to fix it once I can reproduce the problem.

szokeasaurusrex avatar Mar 22 '24 08:03 szokeasaurusrex

I switched upstream images and the problem seems to have gone away. In summary:

Using FROM balenalib/raspberrypi4-64-node:18-bookworm-run, I'd get intermittent Could not resolve host: sentry.io.

I haven't seen it since switching to FROM node:20-bookworm-slim.

I had never seen network problems with other services, and I never saw it when running the docker build locally. So I think there's something particular about the combination of 1) upstream image, 2) the hosted docker environment, and 3) sentry-cli's network access.

Since it seems to be resolved for us, I'm going to let this go. If you have a similar problem, try experimenting with your upstream image/layers.

jrr avatar Apr 05 '24 12:04 jrr

Good to hear, @jrr! I am going to leave this issue open for now, however, since I am still waiting for more information from @Snack-X on how to reproduce the original issue.

szokeasaurusrex avatar Apr 08 '24 08:04 szokeasaurusrex

Perhaps, you could try passing an auth token with --auth-token

# curl -sL https://sentry.io/get-cli/ | SENTRY_CLI_VERSION=2.21.2 bash
(snipped)
Installed sentry-cli 2.21.2
Done!

# sentry-cli info --auth-token=(redacted)
Sentry Server: https://sentry.io
Default Organization: -
Default Project: -

Authentication Info:
  Method: Auth Token
  User: (redacted)
  (snipped)

# curl -sL https://sentry.io/get-cli/ | SENTRY_CLI_VERSION=2.21.3 bash
(snipped)
Installed sentry-cli 2.21.3
Done!

# sentry-cli info --auth-token=(redacted)
Sentry Server: https://sentry.io
Default Organization: -
Default Project: -

Authentication Info:
  Method: Unauthorized
  (failure on authentication: API request failed)

Finally I've got around to create reproducible environment under AWS with Terraform.

  • https://gist.github.com/Snack-X/799a0096ace09fddd4ba93af2c953a86

My original assumption was it has something to do with IPv6. That is the reason why Terraform configuration above contains two separate VPC for both IPv4 only and dualstack VPC. It does not matter as it turns out.

Hope this helps.

Snack-X avatar Apr 12 '24 03:04 Snack-X

Thank you @Snack-X, I will look into this as soon as possible!

szokeasaurusrex avatar Apr 15 '24 13:04 szokeasaurusrex

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

github-actions[bot] avatar May 07 '24 00:05 github-actions[bot]

Don't close it, bot! This is still a thing.

@szokeasaurusrex any progress with @Snack-X 's repro?

For my project, I thought they were gone, but I'm still getting intermittent network failures during my docker builds. I have the following in my Dockerfile:

RUN nslookup sentry.io
RUN pnpx @sentry/cli update
RUN pnpm turbo --filter my-app build

nslookup always works, but the other two may fail. Update's failure looks like this:

[my-app]       error: Could not get the latest release version.

the build failure looks like this:

my-app:build: Error: [sentry-debug-id-upload-plugin] Command failed: /usr/src/app/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli releases new c7a298d8a186788b044c3272e4bca8bb
my-app:build: error: API request failed
my-app:build:   caused by: [6] Couldn't resolve host name (Could not resolve host: sentry.io)
my-app:build: 
my-app:build: Add --log-level=[info|debug] or export SENTRY_LOG_LEVEL=[info|debug] to see more output.
my-app:build: Please attach the full debug log to all bug reports.
my-app:build: 
my-app:build:     at genericNodeError (node:internal/errors:984:15)
my-app:build:     at wrappedFn (node:internal/errors:538:14)
my-app:build:     at ChildProcess.exithandler (node:child_process:422:12)
my-app:build:     at ChildProcess.emit (node:events:519:28)
my-app:build:     at maybeClose (node:internal/child_process:1105:16)
my-app:build:     at Socket.<anonymous> (node:internal/child_process:457:11)
my-app:build:     at Socket.emit (node:events:519:28)
my-app:build:     at Pipe.<anonymous> (node:net:338:12) {
my-app:build:   code: 'PLUGIN_ERROR',
my-app:build:   killed: false,
my-app:build:   signal: null,
my-app:build:   cmd: '/usr/src/app/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli releases new c7a298d8a186788b044c3272e4bca8bb',
my-app:build:   pluginCode: 1,
my-app:build:   plugin: 'sentry-debug-id-upload-plugin',
my-app:build:   hook: 'writeBundle'
my-app:build: }

I've added SENTRY_LOG_LEVEL=debug and will report back if that reveals anything interesting from future failures.

jrr avatar May 13 '24 16:05 jrr

@jrr I have been busy with other tasks and so I still have yet to be able to try @Snack-X's reproduction.

However, the issue you are experiencing is likely separate. I can see you are running RUN pnpx @sentry/cli update. However, since you have installed sentry-cli via a package manager, you should only update sentry-cli via the package manager, and not be running the RUN pnpx @sentry/cli update command. The update command is only intended for users who manually download and install the CLI, e.g. via the curl -sL https://sentry.io/get-cli/ | bash script.

szokeasaurusrex avatar May 14 '24 08:05 szokeasaurusrex

@szokeasaurusrex I don't actually need to update it. That line exists solely for testing connectivity, as you advised earlier in the thread: https://github.com/getsentry/sentry-cli/issues/1929#issuecomment-2014642070

I'm hoping it fails for the same reason as the vite plugin, and may shed more light on what's goin on.

jrr avatar May 14 '24 14:05 jrr

I have a possibly-interesting log from a failure, with SENTRY_LOG_LEVEL=debug on:

nslookup (OK)

[foo-app]       Step 35/52 : RUN nslookup sentry.io
[foo-app]        ---> Running in 0606272e1dfc
[foo-app]       Server:		1.1.1.1
[foo-app]       Address:	1.1.1.1#53
[foo-app]       Non-authoritative answer:
[foo-app]       Name:	sentry.io
[foo-app]       Address: 35.186.247.156
[foo-app]       Removing intermediate container 0606272e1dfc
[foo-app]        ---> f205de02a1d9

pnpx @sentry/cli update (OK)

Looks like this host is actually release-registry.services.sentry.io

(expand) 
[foo-app]       Step 36/52 : RUN pnpx @sentry/cli update
[foo-app]        ---> Running in 12b8d1c69500
[foo-app]       Progress: resolved 1, reused 0, downloaded 0, added 0
[foo-app]
[foo-app]       Progress: resolved 6, reused 0, downloaded 6, added 0
[foo-app]
[foo-app]       Packages: +14
[foo-app]       ++++++++++++++
[foo-app]
[foo-app]       Progress: resolved 20, reused 0, downloaded 13, added 13
[foo-app]
[foo-app]       Progress: resolved 20, reused 0, downloaded 14, added 14, done
[foo-app]
[foo-app]       .../node_modules/@sentry/cli postinstall$ node ./scripts/install.js
[foo-app]
[foo-app]       .../node_modules/@sentry/cli postinstall: Done
[foo-app]
[foo-app]         DEBUG   2024-05-14 17:15:39.224002784 +00:00 sentry-cli version: 2.31.2, platform: "linux", architecture: "aarch64"
[foo-app]         INFO    2024-05-14 17:15:39.224072264 +00:00 sentry-cli was invoked with the following command line: "/root/.cache/pnpm/dlx/avk6v3gae35im2qqboblo6lhp4/18f781a3d63-9/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli" "update"
[foo-app]
[foo-app]         DEBUG   2024-05-14 17:15:39.225037108 +00:00 request GET https://release-registry.services.sentry.io/apps/sentry-cli/latest
[foo-app]
[foo-app]         DEBUG   2024-05-14 17:15:39.225123628 +00:00 retry number 0, max retries: 0
[foo-app]
[foo-app]         DEBUG   2024-05-14 17:15:39.301998435 +00:00 > GET /apps/sentry-cli/latest HTTP/1.1
[foo-app]         DEBUG   2024-05-14 17:15:39.302026155 +00:00 > Host: release-registry.services.sentry.io
[foo-app]         DEBUG   2024-05-14 17:15:39.302035755 +00:00 > Accept: */*
[foo-app]         DEBUG   2024-05-14 17:15:39.302043875 +00:00 > Connection: TE
[foo-app]         DEBUG   2024-05-14 17:15:39.302051555 +00:00 > TE: gzip
[foo-app]         DEBUG   2024-05-14 17:15:39.302059555 +00:00 > User-Agent: sentry-cli/2.31.2
[foo-app]
[foo-app]         DEBUG   2024-05-14 17:15:39.440144031 +00:00 < HTTP/1.1 200 OK
[foo-app]         DEBUG   2024-05-14 17:15:39.440181511 +00:00 < content-type: application/json
[foo-app]         DEBUG   2024-05-14 17:15:39.440198431 +00:00 < content-length: 10441
[foo-app]         DEBUG   2024-05-14 17:15:39.440213231 +00:00 < access-control-allow-origin: *
[foo-app]         DEBUG   2024-05-14 17:15:39.440227351 +00:00 < x-from-cache: 1
[foo-app]         DEBUG   2024-05-14 17:15:39.440241511 +00:00 < x-envoy-upstream-service-time: 1
[foo-app]         DEBUG   2024-05-14 17:15:39.440255111 +00:00 < date: Tue, 14 May 2024 17:15:38 GMT
[foo-app]         DEBUG   2024-05-14 17:15:39.440268712 +00:00 < server: envoy
[foo-app]         DEBUG   2024-05-14 17:15:39.440282872 +00:00 < Via: 1.1 google
[foo-app]         DEBUG   2024-05-14 17:15:39.440298552 +00:00 < Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
[foo-app]
[foo-app]         DEBUG   2024-05-14 17:15:39.440980394 +00:00 response status: 200
[foo-app]
[foo-app]         INFO    2024-05-14 17:15:39.441215955 +00:00 Looking for file named: sentry-cli-Linux-aarch64
[foo-app]
[foo-app]         DEBUG   2024-05-14 17:15:39.441312915 +00:00 body: {"canonical":"app:sentry-cli","createdAt":"2024-03-27T15:57:26.000Z","created_at":"2024-05-02T15:17:39.286Z","file_urls":{"sentry-cli-2.31.2.tgz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-2.31.2.tgz","sentry-cli-Darwin-arm64":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Darwin-arm64","sentry-cli-Darwin-universal":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Darwin-universal","sentry-cli-Darwin-x86_64":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Darwin-x86_64","sentry-cli-Linux-aarch64":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Linux-aarch64","sentry-cli-Linux-armv7":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Linux-armv7","sentry-cli-Linux-i686":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Linux-i686","sentry-cli-Linux-x86_64":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Linux-x86_64","sentry-cli-Windows-i686.exe":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Windows-i686.exe","sentry-cli-Windows-x86_64.exe":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Windows-x86_64.exe","sentry-cli-darwin-2.31.2.tgz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-darwin-2.31.2.tgz","sentry-cli-linux-arm-2.31.2.tgz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-linux-arm-2.31.2.tgz","sentry-cli-linux-arm64-2.31.2.tgz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-linux-arm64-2.31.2.tgz","sentry-cli-linux-i686-2.31.2.tgz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-linux-i686-2.31.2.tgz","sentry-cli-linux-x64-2.31.2.tgz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-linux-x64-2.31.2.tgz","sentry-cli-win32-i686-2.31.2.tgz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-win32-i686-2.31.2.tgz","sentry-cli-win32-x64-2.31.2.tgz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-win32-x64-2.31.2.tgz","sentry_cli-2.31.2-py3-none-macosx_10_15_x86_64.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-macosx_10_15_x86_64.whl","sentry_cli-2.31.2-py3-none-macosx_11_0_arm64.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-macosx_11_0_arm64.whl","sentry_cli-2.31.2-py3-none-macosx_11_0_universal2.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-macosx_11_0_universal2.whl","sentry_cli-2.31.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_2_aarch64.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_2_aarch64.whl","sentry_cli-2.31.2-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.musllinux_1_2_armv7l.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.musllinux_1_2_armv7l.whl","sentry_cli-2.31.2-py3-none-manylinux_2_17_i686.manylinux2014_i686.musllinux_1_2_i686.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-manylinux_2_17_i686.manylinux2014_i686.musllinux_1_2_i686.whl","sentry_cli-2.31.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_2_x86_64.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_2_x86_64.whl","sentry_cli-2.31.2-py3-none-win32.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-win32.whl","sentry_cli-2.31.2-py3-none-win_amd64.whl":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-win_amd64.whl","sentry_cli-2.31.2.tar.gz":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2.tar.gz"},"files":{"sentry-cli-2.31.2.tgz":{"checksums":{"sha256-hex":"917856444544a05e2de78a5a64e55886f41c065ae83c0573b0f53bf6d02971b0"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-2.31.2.tgz"},"sentry-cli-Darwin-arm64":{"checksums":{"sha256-hex":"6de80a5944be669fb9fa934ec3602d493e6de7ebd1ce8507c5c3f6d35ab0eca5"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Darwin-arm64"},"sentry-cli-Darwin-universal":{"checksums":{"sha256-hex":"a8aac1d1821fcd42c75f9e8a4d9abd443d32dafa213f1c515d744de343d1f474"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Darwin-universal"},"sentry-cli-Darwin-x86_64":{"checksums":{"sha256-hex":"2ad229fd4f4cfd64dee7436be1e8c22e2a6c242fc606bc516973aaa69f815ad2"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Darwin-x86_64"},"sentry-cli-Linux-aarch64":{"checksums":{"sha256-hex":"17f2af78965d67571eb22b49934c4c7f2620a7b8e52c9db4116e73eb9877af88"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Linux-aarch64"},"sentry-cli-Linux-armv7":{"checksums":{"sha256-hex":"1a0bb1d6a8b79a07ac2c839d26d696d59c19b80b236ed85cc5ee118e8047e5b0"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Linux-armv7"},"sentry-cli-Linux-i686":{"checksums":{"sha256-hex":"2d5ebac8cf8abfdc08b59cb69ba1c444787a9ddf201a819d201844d73cb386c6"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Linux-i686"},"sentry-cli-Linux-x86_64":{"checksums":{"sha256-hex":"a6e82e7c4eac8d4c358bf261badf97f0d2fc7469093082397345cfe99f6caf18"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Linux-x86_64"},"sentry-cli-Windows-i686.exe":{"checksums":{"sha256-hex":"2e5eed2ccf985259503ed8bcb694a258d1b82dfc9ed5b8eca1bbaf0257b95930"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Windows-i686.exe"},"sentry-cli-Windows-x86_64.exe":{"checksums":{"sha256-hex":"82a395375f4cf732706f5b8030a9394ab57753f76334d2ae480f5d4f6961a723"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-Windows-x86_64.exe"},"sentry-cli-darwin-2.31.2.tgz":{"checksums":{"sha256-hex":"48b310f8d7ae853653fbf66e25c477ae983390de2efbd529be150d4c0f8b12b1"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-darwin-2.31.2.tgz"},"sentry-cli-linux-arm-2.31.2.tgz":{"checksums":{"sha256-hex":"c26552e72693936b41b80755807ed93bbc17f340d0e6e1908d7e235dce2ff9e7"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-linux-arm-2.31.2.tgz"},"sentry-cli-linux-arm64-2.31.2.tgz":{"checksums":{"sha256-hex":"d6b37c81d138ca24092c02caed6571476c5b0c225287bce56e99dfcdd2a5ee17"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-linux-arm64-2.31.2.tgz"},"sentry-cli-linux-i686-2.31.2.tgz":{"checksums":{"sha256-hex":"53b1df6d70caa1c370ef8eea57fb09bed83903358749e6f2735ad7fa3da70138"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-linux-i686-2.31.2.tgz"},"sentry-cli-linux-x64-2.31.2.tgz":{"checksums":{"sha256-hex":"803ded064b492b9bbe5656b6f7670b35ddfb9be83629e226bd7e9e50127201ce"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-linux-x64-2.31.2.tgz"},"sentry-cli-win32-i686-2.31.2.tgz":{"checksums":{"sha256-hex":"b67b906abc3a03c22e323164de52383d6203174ab64c6a30c6bdf2896a42a1c2"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-win32-i686-2.31.2.tgz"},"sentry-cli-win32-x64-2.31.2.tgz":{"checksums":{"sha256-hex":"d548ccc778ac1dded061e9df0234d23a0bbf7ae3d08f8e52629229b3e9030264"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry-cli-win32-x64-2.31.2.tgz"},"sentry_cli-2.31.2-py3-none-macosx_10_15_x86_64.whl":{"checksums":{"sha256-hex":"fbcd2fbe80c9ae0627f53f10f9941881c253930db85a0db98ba71104145c3677"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-macosx_10_15_x86_64.whl"},"sentry_cli-2.31.2-py3-none-macosx_11_0_arm64.whl":{"checksums":{"sha256-hex":"f54958e3638344403a9abdf7521b353712181554e46a76bfb9df6d4719f5664c"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-macosx_11_0_arm64.whl"},"sentry_cli-2.31.2-py3-none-macosx_11_0_universal2.whl":{"checksums":{"sha256-hex":"fc692969fc6e34084f5e8f851937d8e36085311a7adbf9e397df7513a9be923e"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-macosx_11_0_universal2.whl"},"sentry_cli-2.31.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_2_aarch64.whl":{"checksums":{"sha256-hex":"cef1ec3acf17fdbe65995e6542381d534f66c630ac8f51c34617f47122cb77fb"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_2_aarch64.whl"},"sentry_cli-2.31.2-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.musllinux_1_2_armv7l.whl":{"checksums":{"sha256-hex":"9b2bebef2bbe9dcb9637f5cfeeb9c889758a5f6f6792ea562e797252ae7a92a6"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.musllinux_1_2_armv7l.whl"},"sentry_cli-2.31.2-py3-none-manylinux_2_17_i686.manylinux2014_i686.musllinux_1_2_i686.whl":{"checksums":{"sha256-hex":"fd048a1dbcd6c5ac233e2aa95c537e565017857b03b07be423c1735f43b0a651"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-manylinux_2_17_i686.manylinux2014_i686.musllinux_1_2_i686.whl"},"sentry_cli-2.31.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_2_x86_64.whl":{"checksums":{"sha256-hex":"4f1450613af65eb96e8c69bcb8ba8e6dae73e614c7c2a2063cae3994c3e6093e"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_2_x86_64.whl"},"sentry_cli-2.31.2-py3-none-win32.whl":{"checksums":{"sha256-hex":"ff5585354ac5069036528a9a81d5aca640d75d52fab193601f26a7455a22f518"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-win32.whl"},"sentry_cli-2.31.2-py3-none-win_amd64.whl":{"checksums":{"sha256-hex":"b047bcf1c0c73e154760bd1040e0136b481962f85610020238c0e049898223df"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2-py3-none-win_amd64.whl"},"sentry_cli-2.31.2.tar.gz":{"checksums":{"sha256-hex":"5c8b9d3e56233f5c4c15dfcf43bb30aa3038f676bb4d1a7374f14f2f685505f4"},"url":"https://downloads.sentry-cdn.com/sentry-cli/2.31.2/sentry_cli-2.31.2.tar.gz"}},"main_docs_url":"https://docs.sentry.io/learn/cli","name":"Sentry CLI","repo_url":"https://github.com/getsentry/sentry-cli","version":"2.31.2"}
[foo-app]
[foo-app]         INFO    2024-05-14 17:15:39.442307159 +00:00 Found asset sentry-cli-2.31.2.tgz
[foo-app]         INFO    2024-05-14 17:15:39.442321039 +00:00 Found asset sentry_cli-2.31.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_2_aarch64.whl
[foo-app]         INFO    2024-05-14 17:15:39.442329519 +00:00 Found asset sentry_cli-2.31.2-py3-none-manylinux_2_17_i686.manylinux2014_i686.musllinux_1_2_i686.whl
[foo-app]         INFO    2024-05-14 17:15:39.442339199 +00:00 Found asset sentry-cli-linux-x64-2.31.2.tgz
[foo-app]         INFO    2024-05-14 17:15:39.442348399 +00:00 Found asset sentry-cli-Linux-x86_64
[foo-app]         INFO    2024-05-14 17:15:39.442357759 +00:00 Found asset sentry-cli-Linux-aarch64
[foo-app]
[foo-app]       Latest release is 2.31.2
[foo-app]       Already up to date!
[foo-app]       Removing intermediate container 12b8d1c69500
[foo-app]        ---> 7bbcf28f2def

Build (error)

With vite-plugin/2.16.1 and sentry-cli/2.31.1

(expand) 
[foo-app]       Step 37/52 : RUN pnpm turbo --filter foo-app build
[foo-app]        ---> Running in 7134e5432667
[foo-app][foo-app]         INFO    2024-05-14 17:15:39.442321039 +00:00 Found asset sentry_cli-2.31.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_2_aarch64.whl
( some lines removed )
[foo-app]       foo-app:build: build/client/assets/index-BA2eUWpk.js                               108.69 kB │ gzip:  35.29 kB │ map:   258.90 kB
[foo-app]       foo-app:build: build/client/assets/index-4onpzXex.js                               134.76 kB │ gzip:  43.50 kB │ map:   328.71 kB
[foo-app]       foo-app:build: build/client/assets/CodeHighlight-CmhsAnAh.js                       916.38 kB │ gzip: 304.98 kB │ map: 2,046.22 kB
[foo-app]       foo-app:build:
[foo-app]       foo-app:build: (!) Some chunks are larger than 500 kB after minification. Consider:
[foo-app]       foo-app:build: - Using dynamic import() to code-split the application
[foo-app]       foo-app:build: - Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
[foo-app]       foo-app:build: - Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.165081076 +00:00 sentry-cli version: 2.31.1, platform: "linux", architecture: "aarch64"
[foo-app]       foo-app:build:   INFO    2024-05-14 17:15:54.165143276 +00:00 sentry-cli was invoked with the following command line: "/usr/src/app/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli" "--header" "sentry-trace:c7a64dc033bf47068e449d5becf41644-a4c4ba6ee23a1319-1" "--header" "baggage:undefined" "releases" "files" "c5ab9b5e21dc6ef09dfe0e48a880f572" "upload-sourcemaps" "/tmp/sentry-bundler-plugin-upload-XXXXXXT7ncfT" "--ignore" "node_modules" "--no-rewrite" "--use-artifact-bundle"
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.165755798 +00:00 request GET https://sentry.io/api/0/organizations/my-org/chunk-upload/
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.165772558 +00:00 pipeline: vite-plugin/2.16.1
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.165778318 +00:00 using token authentication
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.165789558 +00:00 retry number 0, max retries: 0
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.206841952 +00:00 > GET /api/0/organizations/my-org/chunk-upload/ HTTP/1.1
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.206866872 +00:00 > Host: sentry.io
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.206874672 +00:00 > Accept: */*
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.206881312 +00:00 > Connection: TE
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.206887952 +00:00 > TE: gzip
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.206895072 +00:00 > sentry-trace:c7a64dc033bf47068e449d5becf41644-a4c4ba6ee23a1319-1
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.206902552 +00:00 > baggage:undefined
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.206908952 +00:00 > User-Agent: sentry-cli/2.31.1 vite-plugin/2.16.1
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.208688119 +00:00 > Authorization: Bearer sntrys_e***
[foo-app]       foo-app:build: x Build failed in 9.85s
[foo-app]       foo-app:build: Error: [sentry-debug-id-upload-plugin] Command failed: /usr/src/app/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli releases finalize c5ab9b5e21dc6ef09dfe0e48a880f572
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.273175799 +00:00 sentry-cli version: 2.31.1, platform: "linux", architecture: "aarch64"
[foo-app]       foo-app:build:   INFO    2024-05-14 17:15:54.273249680 +00:00 sentry-cli was invoked with the following command line: "/usr/src/app/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli" "releases" "finalize" "c5ab9b5e21dc6ef09dfe0e48a880f572"
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.274011563 +00:00 request PUT https://sentry.io/api/0/projects/my-org/foo-local/releases/c5ab9b5e21dc6ef09dfe0e48a880f572/
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.274031683 +00:00 pipeline: vite-plugin/2.16.1
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.274042563 +00:00 using token authentication
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.274064883 +00:00 json body: {"projects":["foo-local"],"dateReleased":"2024-05-14T17:15:54.273975162Z"}
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.274079723 +00:00 retry number 0, max retries: 0
[foo-app]       foo-app:build: error: API request failed
[foo-app]       foo-app:build:   caused by: [6] Couldn't resolve host name (Could not resolve host: sentry.io)
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.276773933 +00:00 skipping update nagger because session is not attended
[foo-app]       foo-app:build:
[foo-app]       foo-app:build:     at genericNodeError (node:internal/errors:984:15)
[foo-app]       foo-app:build:     at wrappedFn (node:internal/errors:538:14)
[foo-app]       foo-app:build:     at ChildProcess.exithandler (node:child_process:422:12)
[foo-app]       foo-app:build:     at ChildProcess.emit (node:events:519:28)
[foo-app]       foo-app:build:     at maybeClose (node:internal/child_process:1105:16)
[foo-app]       foo-app:build:     at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
[foo-app]       foo-app:build:   code: 'PLUGIN_ERROR',
[foo-app]       foo-app:build:   killed: false,
[foo-app]       foo-app:build:   signal: null,
[foo-app]       foo-app:build:   cmd: '/usr/src/app/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli releases finalize c5ab9b5e21dc6ef09dfe0e48a880f572',
[foo-app]       foo-app:build:   pluginCode: 1,
[foo-app]       foo-app:build:   plugin: 'sentry-debug-id-upload-plugin',
[foo-app]       foo-app:build:   hook: 'writeBundle'
[foo-app]       foo-app:build: }
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403209045 +00:00 < HTTP/1.1 200 OK
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403260125 +00:00 < server: nginx
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403277645 +00:00 < date: Tue, 14 May 2024 17:15:54 GMT
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403294366 +00:00 < content-type: application/json
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403313286 +00:00 < vary: Accept-Encoding,Accept-Language, Cookie
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403328406 +00:00 < allow: GET, POST, HEAD, OPTIONS
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403342366 +00:00 < access-control-allow-methods: GET, POST, HEAD, OPTIONS
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403369766 +00:00 < access-control-allow-headers: X-Sentry-Auth, X-Requested-With, Origin, Accept, Content-Type, Authentication, Authorization, Content-Encoding, sentry-trace, baggage, X-CSRFToken
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403386446 +00:00 < access-control-expose-headers: X-Sentry-Error, X-Sentry-Direct-Hit, X-Hits, X-Max-Hits, Endpoint, Retry-After, Link
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403402086 +00:00 < access-control-allow-origin: *
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403416246 +00:00 < x-sentry-rate-limit-remaining: 39
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403429526 +00:00 < x-sentry-rate-limit-limit: 40
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403443206 +00:00 < x-sentry-rate-limit-reset: 1715706955
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403458526 +00:00 < x-sentry-rate-limit-concurrentremaining: 24
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403472406 +00:00 < x-sentry-rate-limit-concurrentlimit: 25
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403486566 +00:00 < content-language: en
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403500366 +00:00 < x-frame-options: deny
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403515566 +00:00 < x-content-type-options: nosniff
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403528606 +00:00 < x-xss-protection: 1; mode=block
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403568727 +00:00 < content-security-policy: img-src * blob: data:; base-uri 'none'; object-src 'none'; script-src 'self' 'unsafe-inline' 'report-sample' s1.sentry-cdn.com js.sentry-cdn.com browser.sentry-cdn.com statuspage-production.s3.amazonaws.com static.zdassets.com aui-cdn.atlassian.com connect-cdn.atl-paas.net js.stripe.com 'strict-dynamic' cdn.pendo.io data.pendo.io pendo-io-static.storage.googleapis.com pendo-static-5634074999128064.storage.googleapis.com; font-src * data:; media-src *; frame-ancestors 'self' *.sentry.io; worker-src blob:; connect-src 'self' *.algolia.net *.algolianet.com *.algolia.io sentry.io *.sentry.io s1.sentry-cdn.com o1.ingest.sentry.io api2.amplitude.com app.pendo.io data.pendo.io reload.getsentry.net t687h3m0nh65.statuspage.io sentry.zendesk.com ekr.zdassets.com maps.googleapis.com; default-src 'none'; frame-src app.pendo.io demo.arcade.software js.stripe.com sentry.io; style-src * 'unsafe-inline'; report-uri https://o1.ingest.sentry.io/api/54785/security/?sentry_key=f724a8a027db45f5b21507e7142ff78e&sentry_release=1c44c7846f33414472b24e2b9fe4598b9ae09dd0
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403588127 +00:00 < x-envoy-attempt-count: 1
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403602327 +00:00 < x-envoy-upstream-service-time: 51
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403615527 +00:00 < x-served-by: getsentry-web-rpc-production-65578bfd67-bbhhv
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403629487 +00:00 < x-sentry-proxy-url: http://10.2.0.67:8999/api/0/organizations/my-org/chunk-upload/
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403642927 +00:00 < x-served-by: getsentry-control-web-default-common-production-5d4449bc7-jkd4s
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403657007 +00:00 < x-served-by: frontend-default-747768fd88-mgqxg
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403672207 +00:00 < strict-transport-security: max-age=31536000; includeSubDomains; preload
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403685647 +00:00 < via: 1.1 google
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403698927 +00:00 < Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.403713247 +00:00 < Transfer-Encoding: chunked
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.406311417 +00:00 response status: 200
[foo-app]       foo-app:build:   DEBUG   2024-05-14 17:15:54.406329937 +00:00 body: {"url":"https://us.sentry.io/api/0/organizations/my-org/chunk-upload/","chunkSize":8388608,"chunksPerRequest":64,"maxFileSize":2147483648,"maxRequestSize":33554432,"concurrency":8,"hashAlgorithm":"sha1","compression":["gzip"],"accept":["debug_files","release_files","pdbs","sources","bcsymbolmaps","il2cpp","portablepdbs","artifact_bundles","artifact_bundles_v2"]}
[foo-app]       foo-app:build: > Found 192 files
[foo-app]       foo-app:build: > Analyzing 192 sources
[foo-app]       foo-app:build:  ELIFECYCLE  Command failed with exit code 1.
[foo-app]       foo-app:build: ERROR: command finished with error: command (/usr/src/app/apps/foo) /usr/local/bin/pnpm run build exited (1)
[foo-app]
[foo-app]       foo-app#build: command (/usr/src/app/apps/foo) /usr/local/bin/pnpm run build exited (1)

I'm not fully understanding what's going on here. I see evidence of two requests (GET /chunk-upload/ and PUT /releases/). Is the second of those succeeding? ("HTTP/1.1 200 OK" in the log).

I also noticed "max retries: 0" in there. Can the vite plugin or CLI be configured to allow retries? (or perhaps some env configuration for the underlying HTTP client?)

jrr avatar May 14 '24 17:05 jrr

And here's one where @sentry/cli update fails, though it doesn't log much detail:

expand 
[my-app]       Step 36/52 : RUN pnpx @sentry/cli update
[my-app]        ---> Running in 51fc72424cf6
[my-app]       Progress: resolved 1, reused 0, downloaded 0, added 0
[my-app]
[my-app]       Progress: resolved 7, reused 0, downloaded 6, added 0
[my-app]
[my-app]       Packages: +14
[my-app]       ++++++++++++++
[my-app]
[my-app]       Progress: resolved 20, reused 0, downloaded 13, added 0
[my-app]
[my-app]       Progress: resolved 20, reused 0, downloaded 14, added 14, done
[my-app]
[my-app]       .../node_modules/@sentry/cli postinstall$ node ./scripts/install.js
[my-app]
[my-app]       .../node_modules/@sentry/cli postinstall: Done
[my-app]
[my-app]         DEBUG   2024-05-14 20:08:35.314105723 +00:00 sentry-cli version: 2.31.2, platform: "linux", architecture: "aarch64"
[my-app]         INFO    2024-05-14 20:08:35.314200763 +00:00 sentry-cli was invoked with the following command line: "/root/.cache/pnpm/dlx/avk6v3gae35im2qqboblo6lhp4/18f78b890a2-9/node_modules/.pnpm/@[email protected]/node_modules/@sentry/cli-linux-arm64/bin/sentry-cli" "update"
[my-app]
[my-app]         DEBUG   2024-05-14 20:08:35.315397647 +00:00 request GET https://release-registry.services.sentry.io/apps/sentry-cli/latest
[my-app]
[my-app]         DEBUG   2024-05-14 20:08:35.315476807 +00:00 retry number 0, max retries: 0
[my-app]
[my-app]       error: Could not get the latest release version.
[my-app]         DEBUG   2024-05-14 20:08:35.334034661 +00:00 skipping update nagger because session is not attended
[my-app]
[my-app]       Removing intermediate container 51fc72424cf6
[my-app]       The command '/bin/sh -c pnpx @sentry/cli update' returned a non-zero code: 1

jrr avatar May 14 '24 20:05 jrr

@jrr Taking a look, it seems like max retries is always hardcoded to zero, so I am not sure why this option even exists.

Is the CLI always failing in the same way for you? Based on the error messages (e.g. Couldn't resolve host name (Could not resolve host: sentry.io)), this behavior seems like it could be most easily explained by some kind of (possibly transient) network issue with your setup.

szokeasaurusrex avatar May 15 '24 13:05 szokeasaurusrex

Digging into some of the related issues and their links, it sounds like there are some significant issues with musl's DNS resolver. In particular, it uses DNS over UDP, not TCP (?!) (comment, link)

As a background refresher, TCP has reliability and retries built-in as part of the protocol. UDP does not. So when you use UDP, it's on you to implement reliability/retries at a higher layer. And it seems like neither musl nor sentry-cli is doing this.

In my opinion, long-term, sentry-cli ought to use a network stack with DNS over TCP. In the short term, can you put some retries around it? (@szokeasaurusrex)

jrr avatar Jun 10 '24 15:06 jrr

Hi @jrr, thank you for providing this information, as well as the comment and the link – this information is very helpful!

The only thing I wonder is, assuming that the underlying problem is with musl, what could have changed between 2.21.2 and 2.21.3 that caused this issue to appear?

szokeasaurusrex avatar Jun 10 '24 15:06 szokeasaurusrex

Here's another reference with some more details on musl's DNS behavior: https://wiki.musl-libc.org/functional-differences-from-glibc.html#Name_Resolver/DNS

So it looks like there's more than just the UDP thing: there are some other behavior differences between musl and others that may come into play.

Also I see that DNS over TCP was added to musl recently. @szokeasaurusrex do you know how to track down what version of musl goes into a given sentry-cli version?

jrr avatar Jun 10 '24 17:06 jrr

Also I see that DNS over TCP was added to musl recently. @szokeasaurusrex do you know how to track down what version of musl goes into a given sentry-cli version?

@jrr I am unsure; a codebase-wide search for "musl" does not yield any results containing a version number

szokeasaurusrex avatar Jun 11 '24 06:06 szokeasaurusrex

Other than f226a6e4506699b1bcd624e09c1507844003b515, I don't see any other commits related to musl. Although it doesn't make sense to me, but I'm not an expert in this area.

Previous docker image, getsentry/rust-musl-cross (repo) appears to be built in 2023-04-something with latest musl version, which would be 1.2.3 (released 2022-04-07).

2.21.3 was released in 2023-11-09, and I think implies it was built with latest build of new docker image, messense/rust-musl-cross (repo), at that time.

Although latest musl version at that time is 1.2.4 (released in 2023-05-01) which could have caused this issue, that build also appears to use 1.2.3, what?

I have no idea.

Snack-X avatar Jun 11 '24 07:06 Snack-X

Good finds, @Snack-X . Summarizing my understanding:

  • sentry-cli Linux binaries are built with a Docker container (CI build.yml), which bakes musl into the resulting sentry-cli binary.
  • Sentry used to maintain their own getsentry/rust-musl-cross Docker image for this (docker hub, git repo)
  • In Nov. 2023, the project switched to using a third-party Docker image (sentry-cli PR, messense/rust-musl-cross docker hub and git repo)
  • As of today, that third-party docker image is a couple versions behind on musl:
    • messense/rust-musl-cross is on musl 1.2.3 from 2022 (ref)
    • The latest musl is 1.2.5 from Feb 2024, which notably includes improvements to DNS resolution.

A couple ideas for how Sentry can get get musl's DNS improvements into sentry-cli binaries:

  • contribute to the upstream image, updating their musl
  • switch back to your own sentry-maintained docker image, and this time keep up on maintenance :)

jrr avatar Jun 11 '24 16:06 jrr

Yes, thank you both for this information! I wonder whether simply reverting https://github.com/getsentry/sentry-cli/commit/f226a6e4506699b1bcd624e09c1507844003b515 could solve this problem – perhaps we could try this before we implement one of the other solutions you proposed, @jrr, so that we can verify that this commit was indeed the cause?

I am unaware of the reasoning behind the change in https://github.com/getsentry/sentry-cli/commit/f226a6e4506699b1bcd624e09c1507844003b515 though, so I will first confirm with @loewenheim (who authored the change) whether there is any reason we should not revert that commit.

szokeasaurusrex avatar Jun 12 '24 07:06 szokeasaurusrex