Keytar dependency on GLIBCXX_3.4.21 causes startup failures on Oracle Linux/RedHat/CentOS 7.x

[robertpatrick]

Prerequisites

• [X ] Put an X between the brackets on this line if you have done all of the following:
  • Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
  • Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
  • Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
  • Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description

We are building an electron-based application and trying to use keytar to load/store confidential data. This works fine on Windows and MacOS but the application fails to start on Oracle Linux 7 (and RedHat and CentOS 7).

Steps to Reproduce

1. Take an app that uses keytar in electron.
2. Use electron-builder to create a Linux executable.
3. Try to use the executable on Oracle/RedHat/CentOS 7.

Expected behavior:

The application starts up and allows access to the credential store.

Actual behavior:

[rpatrick@rpatrick-1 squashfs-root]$ ./wktui
A JavaScript error occurred in the main process
Uncaught Exception:
Error: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /tmp/.org.chromium.Chromium.gEV0BQ)
    at process.func [as dlopen] (electron/js2c/asar_bundle.js:5:1846)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1138:18)
    at Object.func [as .node] (electron/js2c/asar_bundle.js:5:2073)
    at Module.load (internal/modules/cjs/loader.js:935:32)
    at Module._load (internal/modules/cjs/loader.js:776:14)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12684)
    at Module.require (internal/modules/cjs/loader.js:959:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/home/rpatrick/squashfs-root/resources/app.asar/node_modules/keytar/lib/keytar.js:1:14)
    at Module._compile (internal/modules/cjs/loader.js:1078:30)

Reproduces how often:

100%

Versions

OS Version: Oracle Linux 7.9 Electron Version: 12.0.5 Keytar version (from package.json): 7.7.0

Additional Information

[xmedeko]

keytar require may fail for various reasons. As a workaround, do keytar require like this:

let keytar;
try {
    keytar = require('keytar');
} catch (err) {
    console.err('keytar cannot be loaded', err.message);
}
const isKeytarAvailable = !!keytar;

And design your app to work even if !isKeytarAvailable

[robertpatrick]

Yes, already planning to do that. It’s unfortunate that keytar is not supporting these very popular Linux platforms. Seems like it would be simple enough to correct this…

[vladimiry]

Did you try not to use prebuilds but rebuilding it on your system?

As a matter of curiosity, what is the output of the below sh script (replace /opt/name-of-your-app/resources/app.asar.unpacked/node_modules with specific to your app value):

for i in $( find /opt/name-of-your-app/resources/app.asar.unpacked/node_modules -type f ( -name ".so" -o -name ".node" ) ) ; do echo "$i"; objdump -T "$i" | grep GLIBC_ | sed 's/.GLIBC_([.0-9])./\1/g' | sort -u ; done

[robertpatrick]

@vladimiry No, I haven’t tried that. The latest version on these platforms is GLIBCXX_3.4.19. I will try the script and report back…

[vladimiry]

It also won't hurt to add the output of the following command (run it on the system you were starting the app): ldd --version (it should simply print the glibc version information).

[vladimiry]

No, I haven’t tried that.

There are at least few ways if you want to try it (ideally you do that on the system you are going to run the app):

• Go to the node_modules/keytar folder and run the alike command HOME=~/.electron-gyp node-gyp rebuild --target=13.0.0-beta.27 --arch=x64 --dist-url=https://electronjs.org/headers (notice electron version value).
• The way I used to rebuild the native dependencies for @electron -based project:
  • remove ./node_modules/prebuild-install folder
  • remove the following folders (just in case): ./node_modules/keytar/{bin,build,prebuilds}
  • run npx electron-builder install-app-deps or just package the app using electron-builder the way you used to do that (the step of compiling the native dependencies will be automatically triggered by electron-builder)

[vladimiry]

I messed up glibcxx with just glibc so no need to run the above-posted commands. The for i in $( find . -type f \( -name "*.so*" -o -name "*.node" \) ) ; do echo "$i"; objdump -T "$i" | grep GLIBCXX ; done command indeed has the GLIBCXX_3-like lines for the https://github.com/atom/node-keytar/releases/download/v7.7.0/keytar-v7.7.0-napi-v3-linux-x64.tar.gz prebuild:

./Release/keytar.node
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4 _Znam
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4 _ZSt9terminatev
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE6resizeEmc
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED2Ev
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4 _ZSt20__throw_length_errorPKc
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_assignERKS4_
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4 _ZSt19__throw_logic_errorPKc
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4 _ZdlPv
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4 _Znwm
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4 _ZdaPv
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE10_M_replaceEmmPKcm
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7reserveEm
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4.21 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERmm
0000000000000000      DF *UND*	0000000000000000  GLIBCXX_3.4 _ZNSt9exceptionD2Ev

So it looks like you will need to find a way to update the libstdc++ on your system (you can run strings /lib64/libstdc++.so.6 | grep GLIBCXX_ to see GLIBCXX you got) or rebuild the keytar on that system. Another possible option is to try downgrading the libstdc++ on the system keytar prepares the prebuilds but I doubt it's going to happen.

[robertpatrick]

Yeah, the problem is that a lot of our users use RedHat/CentOS/Oracle Linux 7. The latest versions of the 7.x OSes only have libstdc++.so that support up to 3.4.19 (version 8.x of these OSes work fine). We cannot require our users to upgrade the OS just to use our application features.

I would really prefer not to have to build/distribute custom keytar builds if I don’t have to do so. I guess my question to you is why do you need to link against this newer version when, for example, linking against 3.4.19 would allow the binary to run on both 7.x and 8.x versions of these popular Linux distributions?

[vladimiry]

I guess my question to you is why do you need to link against this newer version

Not really to me, I'm not a keytar project maintainer but sometimes I read through some issues to collect possibly useful information. I guess you could propose a PR if you are familiar with how to make node-gyp/compiler link against GLIBCXX 3.4.19 or just apply the change for your need internally and then recompile keytar (so you won't use prebuilds).

[robertpatrick]

Sorry, my mistake. Without understanding the current build process, my feeling is that the easiest way to fix this is simply to downgrade the OS on the CI servers on which the Linux binaries are being built...

[vladimiry]

the easiest way to fix this is simply to downgrade the OS on the CI servers on which the Linux binaries are being built...

Yep, my guess in above message was the same. But I'm sure there should be a way to make node-gyp use specific gcc version (although installing addition to default gcc version might also be a time consuming trick). According to https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html you need 4.8.3 which gives you GLIBCXX_3.4.19.

Another possible option is to try downgrading the libstdc++ on the system keytar prepares the prebuilds but I doubt it's going to happen.

[vladimiry]

simply to downgrade the OS on the CI servers on which the Linux binaries are being built

So if you need the solution ASAP you could try this approach on your own (making the system use gcc 4.8.3 and compile the keytar then instead of using prebuilds). I doubt maintainers will jump into the issue as fast as you need since this looks like a rare need (old system).

[robertpatrick]

Maybe rare in the open source developer world, definitely not rare in corporate IT world...which is why I filed the issue.

[vladimiry]

Well, then corporation could consider making a PR to the humble open-source library :smiley:

[shiftkey]

We're using Ubuntu 16.04 to build the native modules because that's the oldest runner that's available with Actions:

https://github.com/atom/node-keytar/blob/12484088ce0980a903627b6fc7ddaaf02188e982/.github/workflows/ci.yml#L25

With how much that's Ubuntu-specific in the CI scripts to setup and test things, this is not a small change to move to a different distro (even upgrading to 18.04 is also a challenge currently).

[vladimiry]

even upgrading to 18.04 is also a challenge currently

Can you share more info on this regard (just a curiosity though)? I think it's better to make prebuilds on the oldest system available since otherwise, it's more likely that users will face an issue similar to this one or https://github.com/vladimiry/ElectronMail/issues/389.

[shiftkey]

Can you share more info on this regard (just a curiosity though)?

It's not just upgrading the OS that needs to happen here, based on when I tried this in https://github.com/atom/node-keytar/pull/378:

• the version of Python included changes (Python 3 is now the de-facto version that Node requires to enable the GYP toolchain for native modules)
• there's a package for interacting with the keychain that we use in CI that's been replaced
• we also have prebuilds for Alpine (to support musl) that had it's own set of issues