Packaging with zeit/pkg doesn't seem to work properly

[felixbrucker]

Hi,

just tried to package a simple app which uses threads.js with https://github.com/zeit/pkg and it didn't seem to work that nicely. There is no error displayed on packaging, but when running the app i'm greeted with:

pkg/prelude/bootstrap.js:1176
      throw error;
      ^

Error: Cannot find module '../config'
1) If you want to compile the package/file into executable, please pay attention to compilation warnings and specify a literal in 'require' call.
2) If you don't want to compile the package/file into executable and want to 'require' it from filesystem (likely plugin), specify an absolute path in 'require' call using process.cwd() or process.execPath.
    at Function.Module._resolveFilename (module.js:545:15)
    at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1278:46)
    at Function.Module._load (module.js:472:25)
    at Module.require (module.js:594:17)
    at Module.require (pkg/prelude/bootstrap.js:1157:31)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (C:\snapshot\WebstormProjects\some-project\node_modules\threads\lib\worker.node\worker.js:18:15)
    at Module._compile (pkg/prelude/bootstrap.js:1252:22)
    at Object.Module._extensions..js (module.js:661:10)
    at Module.load (module.js:563:32)

The line in question requireing is in lib\worker.node\worker.js:18:15.

I use the threads.js as follows:

const threads = require('threads');

const thread = threads.spawn(`${__dirname}/worker.js`);

Anybody has solved this yet or got pointers on what might be the culprit?

[andywer]

Hi @felixbrucker! Interesting...

Pretty sure this is the issue: zeit/pkg#251

Maybe you have got some time to look at the solution proposed in the solution and try it locally?

[felixbrucker]

Thanks for the link, i got it working with some small modifications:

I changed the line in question (worker.js) from

var _config = require('../config');

to

var _config = require(__dirname + '/../config');

similarly as noted in the issue you linked. However i then also needed to explicitly allow/package the js files in the worker.node dir as it seems the native node module detection filtered them out. Furthermore i had to include my worker.js worker script as the call is done via a variable (require(data.script);) which pkg also can't detect. Those last changes are solely for documentation purposes though as there is nothing you can do about it in this lib i believe. For reference, here is the package.json part for pkg if you got an app which uses threads.js:

  "pkg": {
    "assets": [
      "node_modules/threads/lib/worker.node/*.js",
      "lib/worker.js"
    ],
    "targets": [
      "node8-win-x64",
      "node8-linux-x64",
      "node8-macos-x64"
    ]
  }

Whats the best way to get this small fix with __dirname included in the next release?

[andywer]

I suppose the error is rather related to pkg not being able to follow the require() in src/worker.js and src/defaults.js.

Maybe you really only need to add "pkg": { "assets": ["lib/defaults*", "lib/worker.node/*"] } to thread's package.json to make it work?

[felixbrucker]

Yes indeed, when adding your suggested pkg config to thread.js package.json i only needed "assets": ["lib/worker.js"], in my projects package.json

[andywer]

Sounds like a straight-forward fix then :)

Mind opening a PR? I can do it, too, later.

[felixbrucker]

Though i still had to use the __dirname fix for the ..config thingy.

Probably related to this line: https://github.com/andywer/threads.js/blob/master/src/worker.node/worker.js#L5 ?

[andywer]

Ahh, I think it must be "pkg": { "scripts": ["lib/defaults*", "lib/worker.node/*"] } (scripts instead of assets). Maybe that fixes it?

The __dirname really shouldn't be necessary for the ../config.

(See https://github.com/zeit/pkg#scripts)

[felixbrucker]

Using scripts only produced some warnings and finally an error about the .map files. I changed the globs to only use .js files but it still can't find ../config.

I did a directory listing on . a line above the config require and it showed my users homedir. Not sure if that helps.

[andywer]

Can you share your pkg config in package.json in threads?

[felixbrucker]

Sure, last i tried with the js globs was

  "pkg": {
    "scripts": [
      "lib/defaults*.js",
      "lib/worker.node/*.js"
    ]
  }

before it was

  "pkg": {
    "assets": [
      "lib/defaults*",
      "lib/worker.node/*"
    ]
  }

[felixbrucker]

Hence, in order to make use of a file collected at packaging time (require a javascript file or serve an asset) you should take __filename, __dirname, process.pkg.defaultEntrypoint or require.main.filename as a base for your path calculations. For javascript files you can just require or require.resolve because they use current __dirname by default. For assets use path.join(__dirname, '../path/to/asset'). Learn more about path.join in Detecting assets in source code.

If i got this right the files are detected as assets and thus don't work without dirname, but they should be detected as regular scripts where require would work, right?

[andywer]

Another try... 😅

Let's try adding "lib/*.js" to scripts, so the config.js is definitely included.

[felixbrucker]

Just did that (infact i added some more stuff just to be sure: "scripts": ["lib/config.js","lib/defaults*.js","lib/worker.node/*.js","lib/*.js","lib/**/*.js"]) but it didn't work as well.

[andywer]

Shared our troubles in an issue in the pkg repo... (see reference above)

[felixbrucker]

Sure, thanks!

[zaxharperdb]

I was getting an error that the application could not find the module worker. This thread helped me get rid of that error.. But when I require('threads').Pool in one particular file.. the application does not even start.. When I do the require in a few different places it seems to work.. Would be nice if pkg would ever respond to their issues,, but they are getting more scarce.. I have added most of you suggestions.. and done dozens of iterations around those suggestions.. Will update if i get it working.

[venkateshva]

I am encountering the same problem. Tried different combination, but no luck. I got a bunch of JS files and not sure what to define under "Assets and/or Scripts" tag

Many thanks, if someone helps me out