C++ embedder API basic use cases

[Reon90]

• Node.js Version: 15.2
• OS: Mac OS
• Scope (install, code, runtime, meta, other?): C++ embedder API

Could you please provide basic examples with C++ embedder API:

• Pass object from C++ to JS
• Pass object from JS to C++

MaybeLocal<Value> loadenv_ret = node::LoadEnvironment(
        env,
        "function x() { return { x: 1 } }");
MaybeLocal<Value> loadenv_ret2 = node::LoadEnvironment(
        env,
        "x()"); // get { x: 1 } on C++ side

[addaleax]

To clarify, LoadEnvironment() is only expected to be called once, and calling it multiple times may lead to surprising behavior of Node.js. We probably should be clearer about that.

If your goal is passing objects between C++ and JS, then that’s all V8 territory, not Node.js. https://v8.dev/docs/embed and https://v8docs.nodesource.com/node-14.1/ might be more helpful than anything on the Node.js side here. If you want to run a script from C++ and get its result, then v8::Script::Compile(...)->Run(...) is probably what you want, and you’ll get the result of that script as a C++ Local<Value> handle. You can modify that result object from C++, or if it is a function, call it with arguments you specify in C++.

[Reon90]

@addaleax thank you for the advice.

I expect such node.js features as FS, HTTP, etc, how is it possible with v8::Script::Compile(...)->Run(...)? From my point of view, I think about something like N-API to communicate with JS/C++.

[addaleax]

@Reon90 There are multiple ways:

• The easiest is to store the require function that is available inside the LoadEnvironment script somewhere, e.g. on the global object, then access it from there. That makes it available to all scripts.
• You can also use the overload of LoadEnvironment that takes a C++ callback, get access to the raw handle for require, and then store that somewhere (you could e.g. pass it to functions that are the result of script evaluations).
• If you want to use N-API for communication, that’s just fine, but unfortunately not a lot of easy interactions between N-API and the regular V8 API are currently available on the C++ side. You can use AddLinkedBinding() to register an N-API module that is then available through process._linkedBinding().
• Shameless plug: If you want to take a JS file, and a N-API addon, https://github.com/mongodb-js/boxednode might be a good choice. It’s not intended for general embedder API usage, but if all you need is JS + N-API, it should be fine.

[Reon90]

Does it mean that I can't use N-API modules by require() in embedder? What should I do if I have a huge JS application with hundreds of js/native modules? Do I have to import them manually by AddLinkedBinding?

[addaleax]

Does it mean that I can't use N-API modules by require() in embedder?

Oh, sure, you can use that. I had assumed that it might be desirable to ship all of the native code as a single file, but I see that I might have been wrong about that :slightly_smiling_face: require('module').createRequire(...) gives you a require() function just like the ones you’re used to, you can load any JS module or native addon with it.

What should I do if I have a huge JS application with hundreds of js/native modules? Do I have to import them manually by AddLinkedBinding?

No, you don’t have to. If it’s easier for you, then just using require() is fine.

[Reon90]

@addaleax I have asked, because require of native module fails with dlopen error.

[addaleax]

@Reon90 Can you be a bit more specific about what you did and how it failed?

[savearray2]

@addaleax Sorry for resurrecting such an old issue. I don't have much experience working directly with v8, and am looking at embedding the Node.js runtime within a C++ (or ultimately Rust) application. It's a bit difficult to figure out where to start.

My use case is to process data within C++ (DB operations) and upon certain events to call functions written in Javascript/Node.js to handle how to transform said data. I'd like to possibly call several different scripts, passing data to Javascript, and reading the result back into C++. (Essentially using Node.js as the scripting environment to control a database.)

As mentioned, I believe node::LoadEnvironment has an overload which runs a callback upon completion:

NODE_EXTERN v8::MaybeLocal<v8::Value> LoadEnvironment(
    Environment* env,
    StartExecutionCallback cb);

struct StartExecutionCallbackInfo {
  v8::Local<v8::Object> process_object;
  v8::Local<v8::Function> native_require;
};

using StartExecutionCallback =
    std::function<v8::MaybeLocal<v8::Value>(const StartExecutionCallbackInfo&)>;

• Would it be correct to store the native_require function here and then pass it to external scripts (using v8::Script::Compile() for example)?
• What is the process_object that is returned here?
• How would es6 modules/imports be handled under this model? (Is there a better way of trying to do this for my use case?)

Sorry for the questions, and thank you for any help you can provide.

[savearray2]

As a side note, I tried the following code, but it fails when trying to call global->Set and include process_object (the same occurs for native_require):

    MaybeLocal<Value> loadenv_ret = node::LoadEnvironment(
      env,
      [&](const node::StartExecutionCallbackInfo& info) -> v8::MaybeLocal<v8::Value> {
        std::cout << "i was called" << std::endl;
        auto s =
          "require('module').createRequire(process.cwd() + '/');"
          "'Hello' + ', World! '"
          ;
        v8::Isolate* isolate = v8::Isolate::GetCurrent();
        v8::Local<v8::ObjectTemplate> global = v8::ObjectTemplate::New(isolate);
        global->Set(v8::String::NewFromUtf8(isolate, "test").ToLocalChecked(), v8::String::NewFromUtf8(isolate, "data").ToLocalChecked());
        global->Set(v8::String::NewFromUtf8(isolate, "require").ToLocalChecked(), info.process_object);
        auto context =
          v8::Context::New(isolate, nullptr, global);
        Context::Scope context_scope(context);
        v8::Local<v8::String> source =
          v8::String::NewFromUtf8(isolate, s, v8::NewStringType::kNormal).ToLocalChecked();
        v8::Local<v8::Script> script =
          v8::Script::Compile(context, source).ToLocalChecked();
        v8::Local<v8::Value> result = script->Run(context).ToLocalChecked();
        v8::String::Utf8Value utf8(isolate, result);
        std::cout << "result: " << *utf8 << std::endl;
        return v8::Null(isolate);
      }
    );

# Fatal error in , line 0
# Check failed: !value_obj->IsJSReceiver() || value_obj->IsTemplateInfo().

This is probably not the right way to do this.

[addaleax]

Sorry for resurrecting such an old issue.

It’s always okay to open new ones :)

• Would it be correct to store the native_require function here and then pass it to external scripts (using v8::Script::Compile() for example)?

For user-facing APIs, you’ll probably want to use native_require to use module.createRequire() in order to create a public require function that behaves like ones you’d also get from Node.js in a userland script.

What is the process_object that is returned here?

The same process object as you’d have as a global in Node.js scripts: https://nodejs.org/api/process.html

How would es6 modules/imports be handled under this model? (Is there a better way of trying to do this for my use case?)

There isn’t really any explicit integration for this in the embedder API at this point. I think the modules team has mostly been ignoring this, tbh. You might be able to put something together in JS land using vm.SourceTextModule and friends.

This is probably not the right way to do this

This is just a guess based off the error message, but I don’t think you can set actual JS objects on ObjectTemplate instances – if you want to store info.process_object on the global object, you may need to do that after the context/global object has already been created.

[savearray2]

Thank you! That was incredibly helpful! :)

It's unfortunate that there isn't support for es6 modules right out of the box, but I think the vm.SourceTextModule solution will work.

To anyone else who is banging their heads on this, I got the code to work by doing the following:

    MaybeLocal<Value> loadenv_ret = node::LoadEnvironment(
      env,
      [&](const node::StartExecutionCallbackInfo& info) -> v8::MaybeLocal<v8::Value> {
        std::cout << "i was called" << std::endl;
        auto s =
          "console.log(`Using cwd: ${process.cwd()}`);"
          "const publicRequire = require('module').createRequire(process.cwd() + '/');"
          "globalThis.require = publicRequire;"
          "globalThis.embedVars = { nön_ascıı: '🏳️‍🌈' };"
          "require('./test.js')"
          ;
        v8::Isolate* isolate = v8::Isolate::GetCurrent();
        v8::HandleScope scope(isolate);
        v8::Local<v8::Context> context = isolate->GetCurrentContext();
        v8::Local<v8::Object> global_object = context->Global();
        global_object->Set(context, v8::String::NewFromUtf8(isolate, "require").ToLocalChecked(), info.native_require).ToChecked();
        global_object->Set(context, v8::String::NewFromUtf8(isolate, "process").ToLocalChecked(), info.process_object).ToChecked();

        v8::Local<v8::String> source =
          v8::String::NewFromUtf8(isolate, s, v8::NewStringType::kNormal).ToLocalChecked();
        v8::Local<v8::Script> script =
          v8::Script::Compile(context, source).ToLocalChecked();
        v8::Local<v8::Value> result = script->Run(context).ToLocalChecked();
        v8::String::Utf8Value utf8(isolate, result);
        std::cout << "result: " << *utf8 << std::endl;
        return v8::Null(isolate);
      }

With test.js being the following simple script:

module.exports = `Hello world from ${require('os').platform}!`

[s-a]

@savearray2 ty for sharing this. I have a pretty hard time setting this up in VS2019. VS cannot find v8 and node header files even if I add additional path infos to linker settings in c++. Do you have working VS c++ project example any where on github?

How would be the strategy to call a specific function within test.js with parameters and vise versa? So to speak call a ++ function from within test.js. I try to achive a scriptable eventhandler similar to nodejs event emitter to handle event loops within my c++ application.

[brandonros]

Thank you! That was incredibly helpful! :)

It's unfortunate that there isn't support for es6 modules right out of the box, but I think the vm.SourceTextModule solution will work.

To anyone else who is banging their heads on this, I got the code to work by doing the following:

    MaybeLocal<Value> loadenv_ret = node::LoadEnvironment(
      env,
      [&](const node::StartExecutionCallbackInfo& info) -> v8::MaybeLocal<v8::Value> {
        std::cout << "i was called" << std::endl;
        auto s =
          "console.log(`Using cwd: ${process.cwd()}`);"
          "const publicRequire = require('module').createRequire(process.cwd() + '/');"
          "globalThis.require = publicRequire;"
          "globalThis.embedVars = { nön_ascıı: '🏳️‍🌈' };"
          "require('./test.js')"
          ;
        v8::Isolate* isolate = v8::Isolate::GetCurrent();
        v8::HandleScope scope(isolate);
        v8::Local<v8::Context> context = isolate->GetCurrentContext();
        v8::Local<v8::Object> global_object = context->Global();
        global_object->Set(context, v8::String::NewFromUtf8(isolate, "require").ToLocalChecked(), info.native_require).ToChecked();
        global_object->Set(context, v8::String::NewFromUtf8(isolate, "process").ToLocalChecked(), info.process_object).ToChecked();

        v8::Local<v8::String> source =
          v8::String::NewFromUtf8(isolate, s, v8::NewStringType::kNormal).ToLocalChecked();
        v8::Local<v8::Script> script =
          v8::Script::Compile(context, source).ToLocalChecked();
        v8::Local<v8::Value> result = script->Run(context).ToLocalChecked();
        v8::String::Utf8Value utf8(isolate, result);
        std::cout << "result: " << *utf8 << std::endl;
        return v8::Null(isolate);
      }

With test.js being the following simple script:

module.exports = `Hello world from ${require('os').platform}!`

@savearray2

I've got a bit of a dumb idea, maybe you can help me.

I'd like to compile a DLL, and in the DllMain, it kicks off basically exactly what you have (node::LoadEnvironment, script->Run()) but I'd like to be able to speak back and forth between the DLL (native code) and the running node environment/isolate/handle/whatever it should be called.

Is there a way to emit an event from native code, into the node.js runtime? That's the direction that seems hardest to me. nodejs -> native does not seem hard. native -> nodejs seems hard. Maybe I'm just not thinking cleverly enough in terms of existing patterns for FFI/interop.

If event emitting/callbacks aren't possible from JS into native, maybe some kind of shared memory message queue?

[softwareCobbler]

Hi, I was trying to follow along with the instructions https://nodejs.org/api/embedding.html. ctrl+f on that page didn't hit anything with "link" in it -- are we supposed to statically link against node.lib (on windows at least)?

[david-topham-precisely]

Hi, I was trying to follow along with the instructions https://nodejs.org/api/embedding.html. ctrl+f on that page didn't hit anything with "link" in it -- are we supposed to statically link against node.lib (on windows at least)?

I'm going through the same process, and I agree there appears to be no information about what to link against. Did you get anywhere?

[DavidPeicho]

@david-topham-precisely You can link with whatever you want. Simple enough is to build node as a shared library:

./configure --shared
make -j4

And then link against the generated shared library.

[github-actions[bot]]

It seems there has been no activity on this issue for a while, and it is being closed in 30 days. If you believe this issue should remain open, please leave a comment. If you need further assistance or have questions, you can also search for similar issues on Stack Overflow. Make sure to look at the README file for the most updated links.

[github-actions[bot]]

It seems there has been no activity on this issue for a while, and it is being closed. If you believe this issue should remain open, please leave a comment. If you need further assistance or have questions, you can also search for similar issues on Stack Overflow. Make sure to look at the README file for the most updated links.