node
node copied to clipboard
nodejs
tools: speedup compilation of js2c output
Incremental compilation of Node.js is slow. Currently on a powerful Linux machine, it takes about 9 seconds and 830 MB of memory to compile gen/node_javascript.cc with g++. This is the longest step when recompiling a small change to a Javascript file.
gen/node_javascript.cc contains a lot of large binary literals of our Javascript source code. It is well-known that embedding large binary literals as C/C++ arrays is slow. One workaround is to include the data as string literals instead. This is particularly nice for the Javascript included via js2c, which look better as string literals anyway.
Add a new flag --use-string-literals to js2c. When this flag is set, we emit string literals instead of array literals, i.e.:
// old: static const uint8_t X[] = { ... };
static const uint8_t *X = R"JS2C1b732aee(...)JS2C1b732aee";
// old: static const uint16_t Y[] = { ... };
static const uint16_t *Y = uR"JS2C1b732aee(...)JS2C1b732aee";
This requires some modest refactoring in order to deal with the flag being on or off, but the new code itself is actually shorter.
I only enabled the new flag on Linux/macOS, since those are systems that I have available for testing. On my Linux system with gcc, it speeds up compilation by 5.5s (9.0s -> 3.5s). On my Mac system with clang, it speeds up compilation by 2.2s (3.7s -> 1.5s). (I don't think this flag will work with MSVC, but it'd probably speed up clang on windows.)
The long-term goal here is probably to allow this to occur incrementally per Javascript file & in parallel, to avoid recompiling all of gen/node_javascript.cc. Unfortunately the necessary gyp incantations seem impossible (or at least, far beyond me). Anyway, a 60% speedup is a nice enough win.
Refs: https://github.com/nodejs/node/issues/47984
This commit does not land cleanly on v18.x-staging and will need manual backport in case we want it in v18.
