getsentry
Latest @sentry/profiling-node (9.28.1) not working with Node 20 Azure Function: "version `GLIBCXX_3.4.29' not found"
Is there an existing issue for this?
- [x] I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
- [x] I have reviewed the documentation https://docs.sentry.io/
- [x] I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases
How do you use Sentry?
Sentry Saas (sentry.io)
Which SDK are you using?
@sentry/node
SDK Version
9.28.1
Framework Version
Azure Function with Node v20
Link to Sentry event
No response
Reproduction Example/SDK Setup
import * as Sentry from "@sentry/node";
import { nodeProfilingIntegration } from "@sentry/profiling-node";
import { version } from "./version";
// When DSN is undefined, Sentry just ignores all calls
const SENTRY_DSN = process.env.SENTRY_DSN || undefined;
export function getReleaseSemver(): string {
return `OURAPP-functions@${version}`;
}
Sentry.init({
dsn: SENTRY_DSN,
integrations: [nodeProfilingIntegration()],
// Tracing
tracesSampleRate: 1.0, // Capture 100% of the transactions
// Set sampling rate for profiling - this is relative to tracesSampleRate
profilesSampleRate: 1.0,
// Set the release version
release: getReleaseSemver(),
});
Steps to Reproduce
package.json:
{
"name": "OURAPP-functionapp",
"version": "1.0.0",
"description": "",
"main": "dist/src/functions/*.js",
"private": true,
"scripts": {
"##### postinstall #####": "# We use 'npx' so we can use 'pnpm install --prod' to install only production dependencies.",
"postinstall": "npx --yes [email protected] && pnpm run genversion",
"genversion": "(echo \"// Generated file: pnpm run genversion\" && echo \"export const version = \\\"$(jq -r .version < package.json)+$(git rev-parse --short HEAD)\\\";\") > ./src/version.ts",
"prebuild": "pnpm run clean && pnpm run genversion",
"build": "tsc && rm -r dist/tests",
"clean": "rm -rf dist",
"watch": "tsc -w",
"check-updates": "npx npm-check-updates",
"prestart": "pnpm run build",
"start": "func start --typescript --cors \"*\"",
"test": "jest --runInBand",
"test:watch": "jest --runInBand --watch",
"test:ci": "jest --runInBand --ci --coverage --coverageReporters=cobertura --reporters=default --reporters=jest-junit",
"lint": "eslint --fix --max-warnings 0 .",
"lint:ci": "eslint --max-warnings 0 .",
"install-azure-cli-ubuntu": "echo 'Installing Azure CLI. This should work at least for Ubuntu >= 20.04.' && curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash",
"all": "pnpm run lint && pnpm run build && pnpm run test",
"pack-zip": "rm -rf node_modules/ && pnpm install --prod --frozen-lockfile --node-linker=hoisted && zip -r -9 OURAPP-functionapp.zip dist node_modules package.json host.json",
"sentry:sourcemaps": "SENTRY_RELEASE=$(node -e 'const { getReleaseSemver } = require(\"./dist/src/sentry.js\"); console.log(getReleaseSemver());' 2>/dev/null) && sentry-cli sourcemaps inject --org OURCORP --project OURAPP-functions ./dist && sentry-cli sourcemaps upload --org OURCORP --project OURAPP-functions --release $SENTRY_RELEASE ./dist"
},
"dependencies": {
"@azure/functions": "^4.7.3",
"@sentry/node": "^9.28.1",
"@sentry/profiling-node": "^9.28.1",
"dotenv": "^16.5.0",
"jsonwebtoken": "^9.0.2",
"jwks-rsa": "^3.2.0",
"openpgp": "6.1.1",
"three": "^0.177.0"
},
"devDependencies": {
"@eslint/eslintrc": "^3.3.1",
"@eslint/js": "^9.28.0",
"@jest/globals": "^30.0.0",
"@openpgp/web-stream-tools": "^0.1.3",
"@sentry/cli": "^2.46.0",
"@types/jsonwebtoken": "^9.0.9",
"@types/node": "^24.0.0",
"@types/three": "^0.177.0",
"@typescript-eslint/eslint-plugin": "^8.34.0",
"@typescript-eslint/parser": "^8.34.0",
"eslint": "^9.28.0",
"globals": "^16.2.0",
"jest": "^29.7.0",
"jest-junit": "^16.0.0",
"nock": "^14.0.5",
"ts-jest": "^29.3.4",
"typescript": "^5.8.3"
},
"overrides": {},
"engines": {
"node": ">=20.0.0"
},
"packageManager": "[email protected]"
}
azure-pipelines.yml:
# Azure Functions Deploy v2
# Update a function app with .NET, Python, JavaScript, PowerShell, Java based web applications.
- task: AzureFunctionApp@2
inputs:
connectedServiceNameARM: '${{ parameters.connectedServiceNameARM }}' # string. Alias: azureSubscription. Required. Azure Resource Manager connection.
appType: 'functionAppLinux' # 'functionApp' | 'functionAppLinux'. Required. App type.
appName: '${{ parameters.appName }}' # string. Required. Azure Functions App name.
package: '$(Pipeline.Workspace)/OURAPP-functionapp.zip' # string. Required. Package or folder. Default: $(System.DefaultWorkingDirectory)/**/*.zip.
runtimeStack: 'NODE|20' # 'DOTNET|6.0' | 'DOTNET-ISOLATED|6.0' | 'DOTNET-ISOLATED|7.0' | 'DOTNET-ISOLATED|8.0' | 'JAVA|8' | 'JAVA|11' | 'JAVA|17' | 'JAVA|21' | 'NODE|14' | 'NODE|16' | 'NODE|18' | 'NODE|20' | 'PYTHON|3.8' | 'PYTHON|3.9' | 'PYTHON|3.10' | 'PYTHON|3.11'. Optional. Use when appType = functionAppLinux. Runtime stack.
Expected result (working fine with Sentry 9.14.0): Function app is deployed and running. Since 9.28.1 differs in minor version only, I would not expectthe following breaking change:
Actual result (broken with Sentry 9.28.1): Function app is deployed, but returns header-only 404 responses. SSH into the function reveals this error:
root@HOSTNAME:~/site/wwwroot/dist/src# node functions/health.js
node:internal/modules/cjs/loader:1651
return process.dlopen(module, path.toNamespacedPath(filename));
^
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/site/wwwroot/node_modules/@sentry-internal/node-cpu-profiler/lib/sentry_cpu_profiler-linux-x64-glibc-115.node)
at Module._extensions..node (node:internal/modules/cjs/loader:1651:18)
at Module.load (node:internal/modules/cjs/loader:1275:32)
at Module._load (node:internal/modules/cjs/loader:1096:12)
at Module.require (node:internal/modules/cjs/loader:1298:19)
at require (node:internal/modules/helpers:182:18)
at importCppBindingsModule (/home/site/wwwroot/node_modules/@sentry-internal/node-cpu-profiler/lib/index.js:124:28)
at Object.<anonymous> (/home/site/wwwroot/node_modules/@sentry-internal/node-cpu-profiler/lib/index.js:167:38)
at Module._compile (node:internal/modules/cjs/loader:1529:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1613:10)
at Module.load (node:internal/modules/cjs/loader:1275:32) {
code: 'ERR_DLOPEN_FAILED'
}
Node.js v20.19.1
Expected Result
Function app is deployed and running with Node v20 Azure Function.
Actual Result
See error above - it seems that the pre-built binaries are not compatible with Node v20 Azure Functions.
Could it be that this upgrade caused the incompatibility? https://github.com/getsentry/sentry-javascript/commit/c6760c47f63df51c386e100348a7132c2c34f11c "ci: Update runners to ubuntu-24.04 (https://github.com/getsentry/sentry-javascript/pull/15501)"
See also:
- https://stackoverflow.com/questions/56781513/understanding-the-gcc-version-and-the-glibc-glibcxx-versions-in-more-detail
- https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html
Hey, thanks for reporting!
Looking at our commit history, the only change that stands out since 9.14.0 is https://github.com/getsentry/sentry-javascript/pull/16236, or https://github.com/getsentry/sentry-javascript-profiling-node-binaries/pull/12 respectively, which was released in 9.18.0.
@heidemn-faro could you try 9.17.0 and 9.18.0 and check if the problem is related to this version specifically? Would be much appreciated to narrow this down.
Could it be that this upgrade caused the incompatibility? https://github.com/getsentry/sentry-javascript/commit/c6760c47f63df51c386e100348a7132c2c34f11c "ci: Update runners to ubuntu-24.04 (https://github.com/getsentry/sentry-javascript/pull/15501)"
I don't think #15501 is related, given that we build the binaries outside the monorepo (see https://github.com/getsentry/sentry-javascript-profiling-node-binaries). Also, 9.14.0 was released in April and this was merged in February.
Other than that, I don't have much context on our binaries though, so I'll tag @AbhiPrasad and @timfish (also cc @JonasBa) -- could you take a look at this please?
Thanks for your quick reply :-)
I can confirm already that it's working like this. I'll try other combinations too.
# package.json
"packageManager": "[email protected]",
"dependencies": {
"@azure/functions": "^4.7.3",
"@sentry/node": "~9.17.0",
"@sentry/profiling-node": "~9.17.0",
# pnpm-workspace.yaml
overrides:
"@sentry-internal/node-cpu-profiler": "2.1.0"
- WORKS:
9.17.0 + [email protected] - WORKS:
9.18.0 + [email protected](this violates the specified dependency version range"@sentry-internal/node-cpu-profiler": "^2.2.0") - WORKS:
9.28.1 + [email protected](also against specified version range) - BROKEN:
9.17.0 + [email protected] - BROKEN:
9.18.0 + [email protected]
GitHub retired the Ubuntu 20.04 runners so @sentry-internal/[email protected] effectively bumped the minimum libc version supported. Because @sentry/profiling-node depends on ^2.0.0 this change would have impacted anyone who updated their Sentry dependencies.
This is the specific change that would have caused this:
https://github.com/getsentry/sentry-javascript-profiling-node-binaries/pull/13
What Linux distribution and version does Azure functions use? Sounds like it uses Debian but not sure what version.
Azure apparently uses Debian GNU/Linux 11 (bullseye) with libstdc++.so.6.0.28 (obtained by connecting to Azure Function via SSH)
It seems like the latest Sentry packages work with "@sentry-internal/node-cpu-profiler": "2.1.0", even though ^2.2.0 is specified as dependency.
Overriding to 2.1.0 seems somehow "out of spec", so not sure if it's safe to do?
This is a glibc incompatibility.
Debian 11 (bullseye) | 2.31 Debian 12 (bookworm) | 2.36
The minimum supported by @sentry-internal/[email protected] is 2.35.
Overriding to 2.1.0 seems somehow "out of spec", so not sure if it's safe to do?
That is safe to do. The only difference between 2.1.0 and 2.2.0 is Node v24 support and the bump in glibc support.
"dependencies": {
"@sentry/profiling-node": "^9.28.1",
"@sentry-internal/node-cpu-profiler": "2.1.0",
@timfish would it be an option that you build "node-cpu-profiler" inside a Docker container, so you can build against an older libc? (of course it would add some complexity and make the build slower probably...)
I'll also try to test with Node 22, apparently Azure supports that now. Maybe they use a newer Debian version there.
would it be an option that you build "node-cpu-profiler" inside a Docker container,
Possibly.
Does npm install scripts get called in the Azure container and does the container have c++ build tools installed? If so, maybe it should be picking up on the incompatibility and be building from source.
@timfish what I meant is: Inside your pipeline, you could use Docker to build the binaries with an older Ubuntu + libc version, so they should work fine also on older versions.
Anyways I'll keep you updated if switching the Azure Function runtime to Node 22 solves the issue.
Does npm install scripts get called in the Azure container
No, you have to upload a pre-packaged ZIP file incl. node_modules to Azure for the deployment.
Results with Node 22 Azure Function runtime:
# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
# node --version
v22.15.0
# ls /usr/lib/x86_64-linux-gnu/libstdc++.so.*
/usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30
- WORKS: 9.28.1 + [email protected]
- WORKS: 9.28.1 + [email protected]
Nice :-) However Azure Functions does not yet advertise Node 22 as "LTS". So I'm a bit reluctant to use it already, but will hope that they advertise it as "LTS" soon.
So to summarize, upgrading the Azure Function runtime fixes the issue, but requires Node 22 which is not yet advertised as "LTS" by Azure, so people may not be 100% willing to go that route yet.
Also if you upgrade your build agents next time, the same issue may happen again. So it would be nice if you could build the pre-built native libraries with the lowest possible gcc version, to improve compatibility.
requires Node 22 which is not yet advertised as "LTS" by Azure
Node.js themselves consider 22 LTS: https://github.com/nodejs/Release. 20 is in maintenance mode (so it only gets security changes), but any performance improvements or small bug fixes only apply to 22. I would really recommend using Node 22.
Also if you upgrade your build agents next time, the same issue may happen again. So it would be nice if you could build the pre-built native libraries with the lowest possible gcc version, to improve compatibility.
Yes for sure. I think we were in a hard place with the github actions deprecations, but we'll take steps in the future to prevent situations like this. For the time being sorry for trouble.
This issue has gone three weeks without activity. In another week, I will close it.
But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!
"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀
![getsantry[bot] avatar](https://avatars.githubusercontent.com/in/66573?v=4 "Published by a pub.dev verified publisher"){kind=small}
we'll take steps in the future to prevent situations like this
@AbhiPrasad Should the ticket stay open until then? For us, the immediate issue is resolved, or at least worked around. So feel free to close.
@timfish Sorry to revive this but GLIBCXX_3.4.29 does not exist in Debian 11 which is currently the only LTS version, this is an issue as it puts everyone who uses the official debian node image in a choice to either not update sentry or use a non-lts version of debian (yes even for node 22/24)
