node-api: use c-based api for libnode embedding

[vmoroz]

Note: this is an active work in progress and there are still a lot of code churning. You are welcome to comment on the code and share your thoughts, but please be aware that the code is not final yet.

This is a temporary spin off from the PR #43542. This separate PR is created to simplify merging and rebasing with the latest code while we discuss the new API design. When the code is ready it should be merged back to PR #43542.

The goal of the original PR is to enable C API and the Node-API for the embedded scenarios. The C API allows using the shared libnode from runtimes that do not interop with C++ such as WASM, C#, Java, etc. This PR works towards the same goal with some changes to the original code.

This is the related issue #23265.

The API design principles

• Follow the best practices of the Node-API design and provide a way to interop with it.
• Prefix the new API constructs with node_embedding_.
• Design the API for ABI safety and being future proof for new requirements.
  • Follow the Builder pattern for the API design.
  • The typical use is to create an object, configure it, initialize it based on the configuration, use it, and then delete it. The configuration changes are prohibited after the object is initialized.
  • What if the initialization sequence must be customized? It means that we add a new configuration function and insert a customization hook into the initialization sequence. Thus, we can evolve the API by adding new configuration functions, and occasionally deprecating the old functions.
  • All behavior changes must be associated with a new API version number.

The API usage

• To use the C embedding API, we must create, configure, and initialize the global node_embedding_platform. It initializes Node and V8 JS engine once per process and parses the CLI arguments.
• Then, we create, configure, and initialize one or more node_embedding_runtimes. A runtime is responsible for running JavaScript code.
• The runtime CLI arguments are initialized by default with the args and exec_args from the result of the platform initialization. They can be overridden while configuring the runtime.
• A runtime can run in its own thread, several runtimes can share the same thread, or the same runtime can be run from multiple threads.
• The runtime event loop APIs provide control over the runtime execution. These functions can be called many times because they do not destroy the runtime in the end.
• The runtime offers to specify version of Node-API and to retrieve the associated napi_api instance. Any Node-API code that uses the napi_env must be run in the runtime scope controlled by node_embedding_runtime_node_api_scope_open and node_embedding_runtime_node_api_scope_close functions.

The new API deployments

The new Microsoft.JavaScript.LibNode NuGet package compiles and packages the libnode with this new embedding API for win-x64, win-x86, win-arm64, osx-x64, osx-arm64, linux-x64, and linux-arm64 runtimes. (It is currently based on Node.js v20.18.2)

The package is being used from the node-api-dotnet project.

The API overview

Based on the use scenarios, the API can be split up into six groups.

Error handling API

• node_embedding_last_error_message_get
• node_embedding_last_error_message_set

Global platform API

• node_embedding_main_run
• node_embedding_platform_create
• node_embedding_platform_delete
• node_embedding_platform_config_set_flags
• node_embedding_platform_get_parsed_args

Runtime API

• node_embedding_runtime_run
• node_embedding_runtime_create
• node_embedding_runtime_delete
• node_embedding_runtime_config_set_node_api_version
• node_embedding_runtime_config_set_flags
• node_embedding_runtime_config_set_args
• node_embedding_runtime_config_on_preload
• node_embedding_runtime_config_on_loading
• node_embedding_runtime_config_on_loaded
• node_embedding_runtime_config_add_module
• node_embedding_runtime_user_data_get
• node_embedding_runtime_user_data_set
• [ ] add API to handle unhandled exceptions

Runtime API to run event loops

• node_embedding_runtime_config_set_task_runner
• node_embedding_runtime_event_loop_run
• node_embedding_runtime_event_loop_terminate
• node_embedding_runtime_event_loop_run_once
• node_embedding_runtime_event_loop_run_no_wait
• [ ] add API for emitting beforeExit event
• [ ] add API for emitting exit event

Runtime API to interop with Node-API

• node_embedding_runtime_node_api_run
• node_embedding_runtime_node_api_scope_open
• node_embedding_runtime_node_api_scope_close

Documentation

• The new C embedding API is added to the existing embedding.md file after the C++ embedding API description.
• The index.md is changed to indicate that the embedding.md has docs for C++ and C APIs.
• [ ] TODO: complete the examples section.

Tests

• The new C embedding API tests pass the same scenarios as the C++ embedding API tests.
• The embedtest executable can be run in several modes controlled by the first CLI argument. It effectively contains several main functions for different test scenarios.
• The JS test code is changed to provide the test mode argument based on the scenario.
• Added several new test scenarios:
  • run several Node.js runtimes each in its own thread;
  • run several Node.js runtimes all in the same thread;
  • run Node.js runtime from different threads.
  • test that preload callback is called for the main and worker threads.

The PR status

The code is not 100% complete yet. There are still a few TODO items, but I would like to start a discussion with the Node-API team about the new API.

• [ ] Address outstanding TODOs
  • [ ] Allow running Node.js uv_loop from UI loop. Follow the Electron implementation. - Complete implementation for non-Windows.
  • [ ] Can we use some kind of waiter concept instead of the observer thread?
  • [ ] Generate the main script based on the runtime settings.
  • [ ] Set the global Inspector for he main runtime.
  • [ ] Start workers from C++.
  • [ ] Worker to inherit parent Inspector.
  • [ ] Cancel pending event loop tasks on runtime deletion.
  • [ ] Can we initialize platform again if it returns early?
  • [ ] Test passing the V8 thread pool size.
  • [ ] Add a way to terminate the runtime.
  • [ ] Allow to provide custom thread pool from the app.
  • [ ] Consider adding a v-table for the API functions to simplify binding with other languages.
  • [ ] We must not exit the process on node::Environment errors.
  • [ ] Be explicit about the recoverable errors.
  • [ ] Store IsolateScope in TLS.
• [ ] Review the API design
• [ ] Write docs

[nodejs-github-bot]

Review requested:

• [ ] @nodejs/gyp
• [ ] @nodejs/node-api

[vmoroz]

This PR was discussed today 9/13/2024 in the Node-API meeting. This is the summary as I recall it. @mhdawson , @legendecas , @KevinEady , @gabrielschulhof , feel free to augment this comment in case if I missed or misunderstood something.

• The global error handling callback.
  • The initial suggestion from the team was to use the "get-the-last-error" approach as in the Node-API.
  • The counter argument was that while the last error approach works great in the single threaded case it may not work in the multi-thread environment.
  • Since the detailed error info is mostly used for logging, implementing it in the single place is much simpler.
  • The default C embedded API error handler prints the error message to the stderr and exits the process. It is intended to handle "non-recoverable" errors such as wrong argument value passed to the API, or wrong CLI arguments.
  • The related question was what to do if a V8 Isolate runs out of memory. It needs to be investigated, but I guess the answer is that it will be handled by Node.js as it is handled today. The C embedding API does not currently participate in the process. If Node.js typically recovers from that condition, then it must continue doing it.
• Does the new C-based embedding API has a goal to do the same as the C++ embedding API?
  • The answer is "yes" and "no", or better to say "it depends".
  • While we want to have the same functionality, there is no goal to wrap up all existing C++ embedding APIs.
  • The new C embedding API is going to grow based on the scenarios, and we hope that the Builder pattern let us evolve the API without ABI-breaking changes.
  • The C embedding API is going to be implemented on the top of the existing C++ embedded API.
• The API growth based on Builder pattern aims to inject various callbacks in the different parts of the initialization process when needed. E.g. if the Electron needs to do some extra work between the CLI args parsing and V8 platform initialization, then we can add a callback that can be called between these steps.
  • The concern is that such hooks may bloat the C embedding API. Would it be better to use the V8 API instead such as rusty_v8?
  • The answer is that hopefully we are not going to have too many hooks.
  • Providing the C wrappers around the whole V8 API seems to be outside of scope of this PR. One of the goals is to see if we can implement the API in a way that it might be useful for other JS runtimes and engines. Though it is not strictly necessary.
  • Another approach is to see if the whole initialization process can be represented as a pipeline connecting various tasks, and then the embedder can configure the sequence of the tasks in the pipeline.
• Why to create the new C based embedder API if the C++ embedder API provides much more freedom?
  • The main goal is to provide access to shared libnode from languages that do not support C++ interop. E.g. C#.
• Will the new API make it it be more difficult to support and change the C++ embedding API?
  • In many cases the C API is just a thin wrapper on top of the C++ API. Hopefully it will not introduce too many issues.
• It is worth to focus on specific use cases.
  • It is a good point and it should help us to introduce only a bare minimal API to start with. Then, we can grow it based on the new scenarios.
  • We discussed if we should start with a single threaded cases.
  • For one of my use cases it is not enough: we want to use libnode from ASP.NET where we must run multiple threads.
  • Should we have one primary Runtime and others are just the worker threads?
    • The node::Environment was introduced in Node.js to implement worker threads in Node.js.
    • Unlike the worker thread created from JS, the embedder has a control over the thread where the node::Environment is executed.
    • It maybe makes sense to have a single "root" node::Environment and others to be dependent upon it. It must address the issue with the Inspector that currently can be only attached to a single node::Environment or its child worker threads.
• Should we support Node.js experimental features such as the snapshots and the ES6 modules.
  • Since the C embedding API is also an experimental feature, I do not see big drawbacks against it as long as the C API experimental status will be aligned with the features experimental status.
• We have discussed the node_embedding_runtime_add_module function.
  • The function allows to add native modules that can be implemented in the same executable that embeds the libnode.
  • The implementation simply wraps the existing linked modules implementation available in the C++ embedders API.
  • We should consider to rename it to reduce confusion.
• Why do we need to invoke the Node-API code inside of a callback for the node_embedding_runtime_invoke_node_api?
  • Unlike use of the Node-API inside of the native modules, embedders must explicitly establish the V8 Isolate context, etc and then handle the Node-API and JS errors. This function is responsible for taking case of these tasks.
  • The callback for node_embedding_runtime_on_preload and node_embedding_runtime_add_module functions use the same Node-API CallIntoModule internal function.
  • As an alternative we can return back the node_embedding_runtime_open_scope and node_embedding_runtime_close_scope functions.

[vmoroz]

We have discussed the API today 09/20/2024 with @mhdawson. The key take aways:

• It is not clear how to use the new node_embedding_on_wake_up_event_loop. Its goal is to enable running UV loop tasks in app's UI event loop. It is not obvious how to use it. After the discussion and replying to @legendecas feedback, I started to consider replacing it with a V8-like "foreground task runner" concept. It is being currently used for the V8 ABI safe API based on Node-API.
• It would be great to provide the key scenarios which this API targets to address.
• An API function that supposed to aggregate other functions must use them for its implementation rather than calling existing Node.js aggregating implementations. E.g. the node_embedding_complete_event_loop must use node_embedding_run_event_loop and other currently missing functions to raise Node.js beforeExit and exit events. This way we can validate that we expose the right APIs and developers can use the low level functions without hitting a wall.
• We also discussed an idea to replace the "callback+data" pairs with small structs. Hopefully it can make the API easier to use from C++ and other languages, and reduce the number of parameters in some cases.
• The API is still churning. It is probably worth to get another review pass in a couple of weeks.

[vmoroz]

Does the new C-based embedding API has a goal to do the same as the C++ embedding API?

I think this question could be better addressed with an approach for embedders to opt into the "bleeding-edge" C++ API, like mentioned in #43542 (comment). An embedder can highly customize the behavior of V8/Node.js, e.g. Inspectors. If such advanced needs arise in an embedder that already adopted the C embedding API, I believe it would not be trivial for them to migrate to C++ based APIs. Allowing conversion between C/C++ API types would reduce the gaps for embedders using the two variant interfaces.

I just do not see how it can be done in practice. The C API is targeting languages that cannot do the C++ interop. E.g. C#, Python, or a C++ compiler that does not understand the libnode C++ mangled/decorated names. If they cannot interop with C++, then converting between C and C++ cannot help. From another hand, if the embedder code can work with C++ API, then I do not see a point to use the C API.

The only real "escape hatch" is to add the missing functionality to the C API and compile the libnode privately until the PR is accepted by Node.js. Thus, the C API is designed to be extensible from the beginning.

[alshdavid]

Hey, thanks for this PR, it's exactly what we need for a tool my team has been working on. I've created automatically prebuilt binaries from this branch to experiment with writing Rust bindings against. Anything I can do to help with this PR?

Once this is in, it would be amazing if the Nodejs team distributed official prebuilt binaries for libnode. My use case requires no customization and it would make consuming it much more reliable (and trustworthy)

[vmoroz]

@alshdavid , thank you for sharing the project that prebuild the binaries.

This PR have been discussed several times in Node-API meetings. The key areas that I must address based on these discussions:

• Submit changes to Node.js code as small PRs. The PR #57834 was the start.
• Create a separate repo with the C++ wrappers. It was previously part of this PR and then removed.
• Have a better story for the linked modules. Current activation through the process object does not look good.
• Make sure that the new API can be used with other runtimes except Node.js.
• Provide documentation and examples.

On my personal TODO list the "big rocks" are:

• How to run precompiled Node.js modules against the custom app that uses libnode. Currently it does not work at least for Windows. E.g. an ASP.NET web app that uses libnode cannot use any pre-built modules.
• Get a better threading story.
  • Support running JS in UI thread and existing UI run loop. Currently I copied that code from Electron, but there are some implementation "gaps". E.g. the initial JS execution must happen in UI thread, and to support multiple JS runtimes in the same UI thread. Also, I need to understand if the Electron approach is good/universal enough.
  • Implement providing support for application thread pool instead of the thread pools created by V8 and UV. E.g. in applications like Microsoft Word having multiple thread pools created by each library can be quite expensive.
  • Change approach to the event loop support. Currently it is unnecessary opened. Some APIs must be removed to streamline the overall approach.
• Figure out the Inspector (JS debugger) story.
• Have a better story for the flags and options. Passing them as argc/argv does not seem to be convenient. We should have a better approach.

The best way to help at this point is:

• Evaluate the existing APIs in regard to how good it is mapped to other languages. I already did it for C# and C++. It sounds like that you use Rust. I wonder if there are any issues with the API projections to Rust.
• Evaluate it for your scenarios and if the new API has any gaps in supporting them.
• Participate in the design discussions. The API is still at the early stage, and any discussions especially pointing at some "design blunders" are more than welcome.

[alshdavid]

Submit changes to Node.js code as small PRs

In that case, I'd prioritise merging in a minimal & unstable node_embedding_main_run as that is sufficient to get users on board, spread the news and gain traction/induce demand.

That said, I am biased as that's pretty much the only API needed for my use cases, haha - things like improving the module linking API are nice to have but aren't worth adding PR review overhead over given the solution today works, however unergonomic. Improvements can be rolled out additively later.

The ability to create/delete a node_embedding_platform is also nice but not essential. I get around this today by injecting a javascript prelude that coordinates with a napi module to evaluate code within a single runtime instance.

Support running JS in UI thread and existing UI run loop.

My use case is to delegate execution of JavaScript "plugins" to an embedded Nodejs instance, load balancing calls to those plugins across nodejs worker threads.

To do that I spawn an OS thread and run node_start (implementation) within that thread. I trigger JavaScript evaluation via channels/message passing. Consumers of my libnode wrapper simply call;

fn main() {
  let nodejs = Nodejs::load("/path/to/libnode.so");
  nodejs.eval("console.log('hello')").await;
  nodejs.eval("console.log('world')").await;

  let w0 = nodejs.new_worker().await;
  w0.eval("console.log('hello')").await;
  w0.eval("console.log('world')").await;
}

I was able to resolve the challenge of integrating support for non-blocking "async Rust" in "userland" by essentially bolting a Rust event loop onto Nodejs, using a napi module & threadsafe function to yield between execution of the JavaScript event loop and Rust event loop (one Rust event loop per Nodejs worker, running on the same thread).

This can be run on the main thread (UI thread in your case) and requires no changes to libnode. Improvements to this API are welcome but, at least for me, are not required in the initial version.

Figure out the Inspector (JS debugger) story.

My current implementation just passes argv to node_start, so I just pass --inspect-brk in and connect with Chrome's remote debugger. Nodejs workers don't work with the debugger - but that's a Nodejs problem unrelated to libnode. Using node_embedding_platform to replace worker threads would probably be the long term solution.

Have a better story for the flags and options. Passing them as argc/argv does not seem to be convenient. We should have a better approach.

For the final API I'd imagine it's a good idea to take a struct of options but for now, because I wrap the libnode API on my end, I just map a Rust struct to the CLI options before calling node_start with them.

[alshdavid]

Hey @vmoroz, would you consider a smaller footprint PR to get started with? Something like this

I'd imagine this isn't too contentious seeing as it's an unstable API with warnings and doesn't conflict with the overall vision of your PR here. That change is sufficient to enable most of the use cases - with the caveat that supporting features (like the UI thread issue you mentioned earlier) can/must be handled on the consumer side.

[vmoroz]

Hey @vmoroz, would you consider a smaller footprint PR to get started with? Something like this

I'd imagine this isn't too contentious seeing as it's an unstable API with warnings and doesn't conflict with the overall vision of your PR here. That change is sufficient to enable most of the use cases - with the caveat that supporting features (like the UI thread issue you mentioned earlier) can/must be handled on the consumer side.

It is an interesting idea. Let me bring it to the next Node-API discussion this week. The small change like this may be easier to start with. It is a bit surprising to hear that it covers a lot of scenarios.

[alshdavid]

It is a bit surprising to hear that it covers a lot of scenarios.

Yeah! It's a bit hacky because it requires JavaScript shims to get the functionality working. For example; creating a new execution context is done by having the host tell the Nodejs instance to spawn a Worker thread via n-api + JavaScript glue code.

The issue with libuv competing with the main thread (causing deadlocks) can be solved in Rust (not sure if you can do this in C#) by having the consumer bolt on a custom async runtime that works cooperatively with libuv - allowing them to take turns yielding between themselves. Or simply always use Nodejs worker threads for any JavaScript execution haha.

Obviously it's better to solve these problems at the source, that said I have wrapped all of those solutions up behind the following library code https://github.com/alshdavid/edon where I am experimenting with what kind of public API I'd like to interact with/see if it meets my use-cases. Don't judge the source code haha, it's messy at the moment but it functions as a working POC.