dynamic linking error when package with native module ffi-napi

[DanielChen93]

What version of pkg are you using?

5.8.0

What version of Node.js are you using?

12.22.11

What operating system are you using?

Windows

What CPU architecture are you using?

x86_64

What Node versions, OSs and CPU architectures are you building for?

node12-win-x64

Describe the Bug

After package a node program, which includes native modules, cannot start the exe file. Can anyone help me out? Thanks.

Expected Behavior

can successfully run exe file

To Reproduce

I use rticonnextedds-connector as dependency in my node program, rti uses some native modules, including ref-napi ref-struct-napi ffi-napi . while not put anything in assets, pkg setting is like below

  "pkg": {
    "assets": [ ],
    "targets": [
      "node12-win-x64"
    ]
  }

run pkg . and start the exe file will get error like

pkg/prelude/bootstrap.js:1876
      throw error;
      ^

Error: No native build was found for platform=win32 arch=x64 runtime=node abi=72 uv=1 libc=glibc node=12.22.11
    loaded from: C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\ref-napi

    at Function.path (C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\node-gyp-build\index.js:60:9)
    at load (C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\node-gyp-build\index.js:22:30)
    at Object.<anonymous> (C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\ref-napi\lib\ref.js:8:53)
    at Module._compile (pkg/prelude/bootstrap.js:1930:22)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at Module.require (pkg/prelude/bootstrap.js:1855:31)
    at require (internal/modules/cjs/helpers.js:74:18)

after putting something into assets, pkg setting is like below

  "pkg": {
    "assets": [
      "./node_modules/ffi-napi/**/*",
      "./node_modules/ref-struct-napi/**/*",
      "./node_modules/ref-napi/**/*"
    ],
    "targets": [
      "node12-win-x64"
    ]
  }

run pkg . and start the exe file will get error like

pkg/prelude/bootstrap.js:1876
      throw error;
      ^

Error: Dynamic Linking Error: Win32 error 126
    at new DynamicLibrary (C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\ffi-napi\lib\dynamic_library.js:76:11)
    at Object.Library (C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\ffi-napi\lib\library.js:47:10)
    at new _ConnectorBinding (C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\rticonnextdds-connector\rticonnextdds-connector.js:98:11)
    at Object.<anonymous> (C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\rticonnextdds-connector\rticonnextdds-connector.js:151:26)
    at Module._compile (pkg/prelude/bootstrap.js:1930:22)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at Module.require (pkg/prelude/bootstrap.js:1855:31)

[robertsLando]

Try with https://github.com/vercel/pkg/issues/1744

[DanielChen93]

Try with #1744

Hi Roberts, thanks for your reply! I tried to put below code into pkg setting

    "patches": {
      "node_modules/ffi-napi/lib/dynamic_library.js": [
        "this._path = path;",
        "if (path.startsWith('/snapshot/')) { const Fs = require('fs'); const moduleContent = Fs.readFileSync(path); const hash = require('crypto').createHash('sha256').update(moduleContent).digest('hex'); const Path = require('path'); const tmpFolder = Path.join(require('os').tmpdir(), 'pkg', hash); const newPath = Path.join(tmpFolder, Path.basename(path)); if (!Fs.existsSync(tmpFolder)) { Fs.mkdirSync(tmpFolder, { recursive: true }); Fs.copyFileSync(path, newPath); } path = newPath; } this._path = path;"
      ]
    }

But I still got same error :(

[DanielChen93]

@robertsLando I also printed the path, and got

msvcrt.dll
C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\rticonnextdds-connector\rticonnextdds-connector\lib\win-x64\msvcr120.dll
C:\snapshot\dc\Desktop\AOS\iba-server\node_modules\rticonnextdds-connector\rticonnextdds-connector\lib\win-x64\nddscore.dll

the last two can be found in that path, while the first one is missing. Do you think it is the problem?

[robertsLando]

the last two can be found in that path, while the first one is missing. Do you think it is the problem?

Yes

[DanielChen93]

@robertsLando do you know how to fix it? :)

[robertsLando]

I think you need another patch in order to make this working. Give it a look at the output here: https://github.com/node-ffi/node-ffi/blob/master/lib/dynamic_library.js#L35

IMO the path to the dll is wrong and that's the reason why it fails

[DanielChen93]

@robertsLando actually I found I never enter into the if (path.startsWith("/snapshot/")) in below....

if (path.startsWith("/snapshot/")) {
  const Fs = require("fs");
  const moduleContent = Fs.readFileSync(path);
  const hash = require("crypto").createHash("sha256").update(moduleContent).digest("hex");
  const Path = require("path");
  const tmpFolder = Path.join(require("os").tmpdir(), "pkg", hash);
  const newPath = Path.join(tmpFolder, Path.basename(path));
  if (!Fs.existsSync(tmpFolder)) {
    Fs.mkdirSync(tmpFolder, { recursive: true });
    Fs.copyFileSync(path, newPath);
  }
  path = newPath;
}
this._path = path;

[DanielChen93]

@robertsLando by removing if (path.startsWith("/snapshot/")) {}, it works...

[robertsLando]

So fixed?

[DanielChen93]

So fixed?

not 100%, instead of doing if(path.startsWith("/snapshot/")), I do if(path.includes("snapshot")). I know it is not strict, but it works for my case... still don't know why startWith not working, tried C:\\snapshot C:/snapshot and /snapshot/...

[robertsLando]

It may be because you are on windows so paths looks a bit different. BTW includes should work. To be sure you could add a consol.log to print the path so you can do a more strict check

[DanielChen93]

@robertsLando thanks for your help! there is one more issue. After I package it for linux from my windows pc, I run this file from ubuntu and I got another error. Below is the error message

daniel@ubuntu:~/Desktop/iba-server-dist$ ./iba-server-linux 
/snapshot/dc/Desktop/AOS/iba-server/node_modules/rticonnextdds-connector/rticonnextdds-connector/lib/linux-x64/librtiddsconnector.so
pkg/prelude/bootstrap.js:1876
      throw error;
      ^

Error: Dynamic Linking Error: /snapshot/dc/Desktop/AOS/iba-server/node_modules/rticonnextdds-connector/rticonnextdds-connector/lib/linux-x64/librtiddsconnector.so: cannot open shared object file: No such file or directory
    at new DynamicLibrary (/snapshot/dc/Desktop/AOS/iba-server/node_modules/ffi-napi/lib/dynamic_library.js:76:11)
    at Object.Library (/snapshot/dc/Desktop/AOS/iba-server/node_modules/ffi-napi/lib/library.js:47:10)
    at new _ConnectorBinding (/snapshot/dc/Desktop/AOS/iba-server/node_modules/rticonnextdds-connector/rticonnextdds-connector.js:106:20)
    at Object.<anonymous> (/snapshot/dc/Desktop/AOS/iba-server/node_modules/rticonnextdds-connector/rticonnextdds-connector.js:151:26)
    at Module._compile (pkg/prelude/bootstrap.js:1930:22)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at Module.require (pkg/prelude/bootstrap.js:1855:31)

Below is my pkg setting

  "pkg": {
    "assets": [
      "./node_modules/ffi-napi/node_modules/ref-napi/prebuilds",
      "./node_modules/ref-struct-napi/node_modules/ref-napi/prebuilds",
      "./node_modules/ffi-napi/prebuilds",
      "./node_modules/ref-napi/prebuilds",
      "./node_modules/rticonnextdds-connector/rticonnextdds-connector/lib/linux-x64/*",
      "./node_modules/rticonnextdds-connector/rticonnextdds-connector/lib/win-x64/*",
      "./node_modules/node-expat/build/Release/*"
    ],
    "outputPath": "dist",
    "patches": {
      "node_modules/ffi-napi/lib/dynamic_library.js": [
        "this._path = path;",
        "console.log(path);",
        "if(path.includes('snapshot')) {const Fs = require('fs'); const moduleContent = Fs.readFileSync(path); const hash = require('crypto').createHash('sha256').update(moduleContent).digest('hex'); const Path = require('path'); const tmpFolder = Path.join(require('os').tmpdir(), 'pkg', hash); const newPath = Path.join(tmpFolder, Path.basename(path)); if (!Fs.existsSync(tmpFolder)) { Fs.mkdirSync(tmpFolder, { recursive: true }); Fs.copyFileSync(path, newPath); } path = newPath; } this._path = path;"
      ]
    },
    "targets": [
      "node12-win-x64",
      "node12-linux-x64"
    ]
  },

[robertsLando]

"node_modules/ffi-napi/lib/dynamic_library.js": [
        "this._path = path;",
        "if(path.includes('snapshot')) {const Fs = require('fs'); const moduleContent = Fs.readFileSync(path); const hash = require('crypto').createHash('sha256').update(moduleContent).digest('hex'); const Path = require('path'); const tmpFolder = Path.join(require('os').tmpdir(), 'pkg', hash); const newPath = Path.join(tmpFolder, Path.basename(path)); if (!Fs.existsSync(tmpFolder)) { Fs.mkdirSync(tmpFolder, { recursive: true }); Fs.copyFileSync(path, newPath); } path = newPath; } this._path = path;"
      ]

Just remove "console.log(path);", second array element as it's wrong

[DanielChen93]

"node_modules/ffi-napi/lib/dynamic_library.js": [
        "this._path = path;",
        "if(path.includes('snapshot')) {const Fs = require('fs'); const moduleContent = Fs.readFileSync(path); const hash = require('crypto').createHash('sha256').update(moduleContent).digest('hex'); const Path = require('path'); const tmpFolder = Path.join(require('os').tmpdir(), 'pkg', hash); const newPath = Path.join(tmpFolder, Path.basename(path)); if (!Fs.existsSync(tmpFolder)) { Fs.mkdirSync(tmpFolder, { recursive: true }); Fs.copyFileSync(path, newPath); } path = newPath; } this._path = path;"
      ]

Just remove "console.log(path);", second array element as it's wrong

get error like below

daniel@ubuntu:~/Desktop/iba-server-dist$ ./iba-server-linux 
pkg/prelude/bootstrap.js:1876
      throw error;
      ^

Error: ENOENT: no such file or directory, open 'libnddsc.so'
1) If you want to compile the package/file into executable, please pay attention to compilation warnings and specify a literal in 'require' call. 2) If you don't want to compile the package/file into executable and want to 'require' it from filesystem (likely plugin), specify an absolute path in 'require' call using process.cwd() or process.execPath.
    at Object.openSync (fs.js:462:3)
    at Object.openSync (pkg/prelude/bootstrap.js:796:32)
    at Object.readFileSync (fs.js:364:35)
    at readFileSync (pkg/prelude/bootstrap.js:1082:36)
    at new DynamicLibrary (/snapshot/dc/Desktop/AOS/iba-server/node_modules/ffi-napi/lib/dynamic_library.js:69:23)
    at Object.Library (/snapshot/dc/Desktop/AOS/iba-server/node_modules/ffi-napi/lib/library.js:47:10)
    at new _ConnectorBinding (/snapshot/dc/Desktop/AOS/iba-server/node_modules/rticonnextdds-connector/rticonnextdds-connector.js:106:20)
    at Object.<anonymous> (/snapshot/dc/Desktop/AOS/iba-server/node_modules/rticonnextdds-connector/rticonnextdds-connector.js:151:26)
    at Module._compile (pkg/prelude/bootstrap.js:1930:22)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: 'libnddsc.so',
  pkg: true
}

[DanielChen93]

@robertsLando hello there, would you please help me have a look? many thanks. :)

[robertsLando]

No clue I'm sorry you should debug this on your side to see where that is required and try to patch the files according

[DanielChen93]

@robertsLando Hi, I made some progress on linux build. I found after run pkg . , I did create tmp folder for those native addons. However, two files are missing, which are libnddsc.so and librtiddscore.so. I doubt if those two files have been packaged, below is my assets file. Is there any way I can see a list of native addons in the packaged bundle?

     "assets": [
      "./node_modules/ffi-napi/node_modules/ref-napi/prebuilds",
      "./node_modules/ref-struct-napi/node_modules/ref-napi/prebuilds",
      "./node_modules/ffi-napi/prebuilds",
      "./node_modules/ref-napi/prebuilds",
      "./node_modules/rticonnextdds-connector/rticonnextdds-connector/lib",
      "./node_modules/node-expat/build/Release/*"
    ],

[DanielChen93]

@robertsLando I console.log all the files, and I found libnddsc.so and librtiddscore.so. The reason they are missing is because the patch code in package.json only executed once rather than three times for these three .so file. Don't know why, it is working under windows

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[github-actions[bot]]

This issue is now closed due to inactivity, you can of course reopen or reference this issue if you see fit.