workerd icon indicating copy to clipboard operation
workerd copied to clipboard

add compile cache support to wd_?s_bundle, enable for node

Open mikea opened this issue 1 year ago • 2 comments

Extracted from #3004, not used on load yet.

downstream pr 9121

mikea avatar Nov 01 '24 21:11 mikea

$ cat bazel-bin/src/node/node.capnp
@0xbcc8f57c63814005;

# generated by @workerd//build/wd_js_bundle.bzl

using Modules = import "/workerd/jsg/modules.capnp";

const nodeBundle :Modules.Bundle = (
  modules = [
    (name = "node:_stream_duplex", src = embed "node__stream_duplex", type = builtin, compileCache = embed "node__stream_duplex_cache", ),
    (name = "node:_stream_passthrough", src = embed "node__stream_passthrough", type = builtin, compileCache = embed "node__stream_passthrough_cache", ),
    (name = "node:_stream_readable", src = embed "node__stream_readable", type = builtin, compileCache = embed "node__stream_readable_cache", ),

.....

mikea avatar Nov 01 '24 21:11 mikea

$ du -b bazel-out/k8-fastbuild/bin/src/node/*_cache
552     bazel-out/k8-fastbuild/bin/src/node/node_assert_cache
560     bazel-out/k8-fastbuild/bin/src/node/node_assert_strict_cache
744     bazel-out/k8-fastbuild/bin/src/node/node_async_hooks_cache
1824    bazel-out/k8-fastbuild/bin/src/node/node_buffer_cache
6424    bazel-out/k8-fastbuild/bin/src/node/node_crypto_cache
3712    bazel-out/k8-fastbuild/bin/src/node/node_diagnostics_channel_cache
456     bazel-out/k8-fastbuild/bin/src/node/node_events_cache
3248    bazel-out/k8-fastbuild/bin/src/node/node-internal_constants_cache
3264    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_dh_cache
3328    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_hash_cache
1960    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_hkdf_cache
5456    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_keys_cache
1288    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_pbkdf2_cache
3304    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_random_cache
1584    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_scrypt_cache
1312    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_spkac_cache
1872    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_util_cache
4896    bazel-out/k8-fastbuild/bin/src/node/node-internal_crypto_x509_cache
872     bazel-out/k8-fastbuild/bin/src/node/node-internal_debuglog_cache
8608    bazel-out/k8-fastbuild/bin/src/node/node-internal_events_cache
4528    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_assert_cache
2616    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_assertionerror_cache
18016   bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_buffer_cache
3568    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_comparisons_cache
1408    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_diffs_cache
13704   bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_errors_cache
18512   bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_inspect_cache
3936    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_path_cache
4936    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_querystring_cache
2400    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_stringdecoder_cache
5072    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_types_cache
3768    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_url_cache
1856    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_utils_cache
8440    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_zlib_base_cache
7640    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_zlib_cache
1904    bazel-out/k8-fastbuild/bin/src/node/node-internal_internal_zlib_constants_cache
5872    bazel-out/k8-fastbuild/bin/src/node/node-internal_legacy_url_cache
4168    bazel-out/k8-fastbuild/bin/src/node/node-internal_mock_cache
1528    bazel-out/k8-fastbuild/bin/src/node/node-internal_process_cache
3456    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_adapters_cache
1656    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_compose_cache
4280    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_duplex_cache
1360    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_legacy_cache
2920    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_pipeline_cache
1016    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_promises_cache
13928   bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_readable_cache
1896    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_transform_cache
8768    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_util_cache
8016    bazel-out/k8-fastbuild/bin/src/node/node-internal_streams_writable_cache
3320    bazel-out/k8-fastbuild/bin/src/node/node-internal_validators_cache
2552    bazel-out/k8-fastbuild/bin/src/node/node_module_cache
1440    bazel-out/k8-fastbuild/bin/src/node/node_path_cache
1456    bazel-out/k8-fastbuild/bin/src/node/node_path_posix_cache
1456    bazel-out/k8-fastbuild/bin/src/node/node_path_win32_cache
456     bazel-out/k8-fastbuild/bin/src/node/node_process_cache
1152    bazel-out/k8-fastbuild/bin/src/node/node_querystring_cache
3064    bazel-out/k8-fastbuild/bin/src/node/node_stream_cache
1176    bazel-out/k8-fastbuild/bin/src/node/node_stream_consumers_cache
784     bazel-out/k8-fastbuild/bin/src/node/node__stream_duplex_cache
592     bazel-out/k8-fastbuild/bin/src/node/node__stream_passthrough_cache
576     bazel-out/k8-fastbuild/bin/src/node/node_stream_promises_cache
960     bazel-out/k8-fastbuild/bin/src/node/node__stream_readable_cache
592     bazel-out/k8-fastbuild/bin/src/node/node__stream_transform_cache
2000    bazel-out/k8-fastbuild/bin/src/node/node_stream_web_cache
824     bazel-out/k8-fastbuild/bin/src/node/node__stream_writable_cache
640     bazel-out/k8-fastbuild/bin/src/node/node_string_decoder_cache
784     bazel-out/k8-fastbuild/bin/src/node/node_test_cache
2048    bazel-out/k8-fastbuild/bin/src/node/node_url_cache
4952    bazel-out/k8-fastbuild/bin/src/node/node_util_cache
2688    bazel-out/k8-fastbuild/bin/src/node/node_util_types_cache
13184   bazel-out/k8-fastbuild/bin/src/node/node_zlib_cache

mikea avatar Nov 01 '24 21:11 mikea

This doesn't also enable it for consumption on load right? I think we should do it on the same PR.

why? This thing is brittle so I'd like to land it together with EW side. Also as of now this is a no-op and has very low (0?) risk in production. Load is different and has to be treated differently. Makes all sense to me to break this down in separete PRs.

mikea avatar Nov 04 '24 16:11 mikea

Unfortunately this is not landable now: as predicted there are issues with cross-compilation downstream. I'll need to massage the run rule to support running under qemu.

mikea avatar Nov 08 '24 16:11 mikea