node icon indicating copy to clipboard operation
node copied to clipboard

TypeError: Missing internal module 'node:crypto'

Open bncdemo opened this issue 1 year ago • 16 comments

Version

v18.20.2

Platform

win10

Subsystem

No response

What steps will reproduce the bug?

I am building node with a vcbuild.bat release x64 static dll command. node.exe <myapp.js> run as expected, but when I try to do this with the built libnode.dll, I receive:

TypeError: Missing internal module 'node:crypto'

I don't understand. In shared lib mode the internal modules not loaded automatically? (I can run 'regular' scripts with the shared lib without problem)

How often does it reproduce? Is there a required condition?

It happens every time.

What is the expected behavior? Why is that the expected behavior?

No response

What do you see instead?

The code drops an unhandled exception and stop:

process.on('uncaughtException', function(err) {
  console.log(err);
});

->

TypeError: Missing internal module 'node:crypto'

Additional information

No response

bncdemo avatar May 26 '24 12:05 bncdemo

Thanks for the issue! Could you possibly provide some example code to cause this to occur?

avivkeller avatar May 26 '24 12:05 avivkeller

Thanks for the issue! Could you possibly provide some example code to cause this to occur?

This single line:

var nc = require("node:crypto");

create the error above.

bncdemo avatar May 26 '24 15:05 bncdemo

I can't reproduce with my default installation, but it might be a build thing that I haven't done (v22).

> Object.keys(require("node:crypto"))
[
  'checkPrime',
  'checkPrimeSync',
  'createCipheriv',
  'createDecipheriv',
  'createDiffieHellman',
  'createDiffieHellmanGroup',
  'createECDH',
  'createHash',
  'createHmac',
  'createPrivateKey',
  'createPublicKey',
  'createSecretKey',
  'createSign',
  'createVerify',
  'diffieHellman',
  'generatePrime',
  'generatePrimeSync',
  'getCiphers',
  'getCipherInfo',
  'getCurves',
  'getDiffieHellman',
  'getHashes',
  'hkdf',
  'hkdfSync',
  'pbkdf2',
  'pbkdf2Sync',
  'generateKeyPair',
  'generateKeyPairSync',
  'generateKey',
  'generateKeySync',
  'privateDecrypt',
  'privateEncrypt',
  'publicDecrypt',
  'publicEncrypt',
  'randomBytes',
  'randomFill',
  'randomFillSync',
  'randomInt',
  'randomUUID',
  'scrypt',
  'scryptSync',
  'sign',
  'setEngine',
  'timingSafeEqual',
  'getFips',
  'setFips',
  'verify',
  'hash',
  'Certificate',
  'Cipher',
  'Cipheriv',
  'Decipher',
  'Decipheriv',
  'DiffieHellman',
  'DiffieHellmanGroup',
  'ECDH',
  'Hash',
  'Hmac',
  'KeyObject',
  'Sign',
  'Verify',
  'X509Certificate',
  'secureHeapUsed',
  'constants',
  'webcrypto',
  'subtle',
  'getRandomValues'
]

avivkeller avatar May 26 '24 16:05 avivkeller

I can't reproduce with my default installation, but it might be a build thing that I haven't done (v22).

> Object.keys(require("node:crypto"))
[
  'checkPrime',
  'checkPrimeSync',
  'createCipheriv',
  'createDecipheriv',
  'createDiffieHellman',
  'createDiffieHellmanGroup',
  'createECDH',
  'createHash',
  'createHmac',
  'createPrivateKey',
  'createPublicKey',
  'createSecretKey',
  'createSign',
  'createVerify',
  'diffieHellman',
  'generatePrime',
  'generatePrimeSync',
  'getCiphers',
  'getCipherInfo',
  'getCurves',
  'getDiffieHellman',
  'getHashes',
  'hkdf',
  'hkdfSync',
  'pbkdf2',
  'pbkdf2Sync',
  'generateKeyPair',
  'generateKeyPairSync',
  'generateKey',
  'generateKeySync',
  'privateDecrypt',
  'privateEncrypt',
  'publicDecrypt',
  'publicEncrypt',
  'randomBytes',
  'randomFill',
  'randomFillSync',
  'randomInt',
  'randomUUID',
  'scrypt',
  'scryptSync',
  'sign',
  'setEngine',
  'timingSafeEqual',
  'getFips',
  'setFips',
  'verify',
  'hash',
  'Certificate',
  'Cipher',
  'Cipheriv',
  'Decipher',
  'Decipheriv',
  'DiffieHellman',
  'DiffieHellmanGroup',
  'ECDH',
  'Hash',
  'Hmac',
  'KeyObject',
  'Sign',
  'Verify',
  'X509Certificate',
  'secureHeapUsed',
  'constants',
  'webcrypto',
  'subtle',
  'getRandomValues'
]

Where are you running this exactly? 🤔 Are you using the embedder code

https://github.com/nodejs/node/blob/dc609f5a81044a37cbcaff92c7fc05fcef58c52a/test/embedding/embedtest.cc#L189

RunNodeInstance --> node::LoadEnvironment() ?

bncdemo avatar May 26 '24 16:05 bncdemo

I'm running this without the embedder, just the normal REPL. I know it's not the same, but it just verifies that this isn't an issue with the entire Node.js requires.

@nodejs/core I don't think there is an embedders team (the @nodejs/embedders team is for embedding into Electron, etc, right?), if someone knows who to ping that would be helpful!

avivkeller avatar May 26 '24 16:05 avivkeller

There is some progress. Removing the 'node:' tag make the code running, but I don't want to remove this on long term in every single code:

the test shows the following:

node.exe require('crypto'); -> working node.exe require('node:crypto'); -> working

libnode.dll require('crypto'); -> working libnode.dll require('node:crypto'); -> FAIL

Can anybody comment on this? Is there a default namespace 'node' missing from the embedder code? How or where can I set that?

bncdemo avatar May 27 '24 14:05 bncdemo

Any chance the require you're using is not the CJS one but the internal one? What is the value of require.resovle in the context you're using?

aduh95 avatar May 27 '24 14:05 aduh95

(Note that @nodejs/core is not that smaller than the entire collaborators team and is only used for organizational purposes, we don't usually ping it to avoid spamming uninterested people).

If you are using the embedder API, the require you get is the internal one. It doesn't recognize the node: prefix because there is no need, since it has only access to built-ins and internal modules. If you want to use it as the user-land require, do a require('module').createRequire() and use the user-land require it builds instead.

joyeecheung avatar May 27 '24 15:05 joyeecheung

require.resovle

node.exe require.resolve: function resolve(request, options) { validateString(request, 'request'); return Module._resolveFilename(request, mod, false, options); }

libnode.dll require.resolve: undefined

bncdemo avatar May 27 '24 15:05 bncdemo

(Note that @nodejs/core is not that smaller than the entire collaborators team and is only used for organizational purposes, we don't usually ping it to avoid spamming uninterested people).

Yikes sorry! Is there a team for this API?

avivkeller avatar May 27 '24 15:05 avivkeller

(Note that @nodejs/core is not that smaller than the entire collaborators team and is only used for organizational purposes, we don't usually ping it to avoid spamming uninterested people).

If you are using the embedder API, the require you get is the internal one. It doesn't recognize the node: prefix because there is no need, since it has only access to built-ins and internal modules. If you want to use it as the user-land require, do a require('module').createRequire() and use the user-land require it builds instead.

Is there no way to match the regular node.exe context? (from the c++ / embedder side, leaving the nodejs script intact)

bncdemo avatar May 27 '24 15:05 bncdemo

It's expected that embedders take the internal require() and do some other stuff of their own before they go on executing a user script, creating a user-land require with require('module').createRequire() and then piping that into the actual user script can be one of those. It's generally not a good idea to directly use the LoadEnvironment API to execute user script because the require() it has access to can access Node.js internals, which is something embedders may need, but users shouldn't do.

joyeecheung avatar May 27 '24 15:05 joyeecheung

Yikes sorry! Is there a team for this API?

---@nodejs/embedders--- actually no that's users

benjamingr avatar May 27 '24 18:05 benjamingr

__dirname is not available because the script that gets executed by LoadEnvironment is not treated as user-land modules. __dirname is only a concept available in CJS modules. It doesn't make too much sense to predefine a __dirname for example for the main script (what should it even be? The current working directory? The path of the executable? Or something else that the embedder want to customize based on how the embedding application gets run?)

For example if you want to simply load user scripts from disk and you want to execute scripts as user-land modules, you can to use the V8 APIs to get a user-land CJS loader using require('module').createRequire(base_file_name) (it's up to the embedders to decide what base_file_name is) then invoke the returned require() function with the user script.

If you want to run a script that's already read from your native side, you can declare the __dirname etc. yourself in the context, convert the C++ string into a V8 string, pass it into the context somehow (for example as a hidden property in the global object using the V8 API), and then use require('vm').runInContext(str).

There is no readily available way to run a C++ string as a user-land CJS module as the embedder is expected to provide extra information about e.g. what __dirname is supposed to be. For example Node.js itself doesn't really know what __dirname is supposed to be when /usr/bin/embedding_app is run from /users/foo/project loading a file that /usr/bin/embedding_app loaded previously from https://example.com/test/test.js or any other end point only known to the embedder, and it's not very appropriate for Node.js to just randomly guess what it's supposed to be. The embedder is expected to customize it according to their own use case.

joyeecheung avatar May 29 '24 16:05 joyeecheung

@RedYetiDev I believe this is what happening for X509Certificate module as well under the crypto.

Version: v20.14.0 Environment: win11

just do Object.keys(require("node:crypto").X509Certificate) and the result would be []

Can anyone help me over here, would like to make use of X509Certificate module.

rajnikantsethiya-sage avatar Jun 27 '24 12:06 rajnikantsethiya-sage

Hi! If you're experiencing an issue with a specific part of crypto, I don't think this issue is the right place to post it.

Please open an issue in nodejs/help.

If I'm misunderstanding the concern, let me know

avivkeller avatar Jun 27 '24 13:06 avivkeller