node-v8 icon indicating copy to clipboard operation
node-v8 copied to clipboard

Reproducible snapshot error

Open targos opened this issue 3 months ago • 14 comments

Seems arm-specific: https://github.com/nodejs/node-v8/actions/runs/19205829620/job/54900847817

@joyeecheung

=== release test-snapshot-reproducible ===
Path: parallel/test-snapshot-reproducible
Error: --- stderr ---
node:internal/assert/utils:146
  throw error;
  ^

AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected

+ [
+   {
+     offset: '0x2ffec0',
+     slice1: '06000400000000000300000059ab0000',
+     slice2: '060004000000000003000000b8aa0000'
+   },
+   {
+     offset: '0x476eb0',
+     slice1: '00000059ab0000580701106100000000',
+     slice2: '000000b8aa0000580701106100000000'
+   }
+ ]
- []

    at Object.<anonymous> (/home/runner/work/node-v8/node-v8/node/test/parallel/test-snapshot-reproducible.js:69:8)
    at Module._compile (node:internal/modules/cjs/loader:1760:14)
    at Object..js (node:internal/modules/cjs/loader:1892:10)
    at Module.load (node:internal/modules/cjs/loader:1480:32)
    at Module._load (node:internal/modules/cjs/loader:1299:12)
    at TracingChannel.traceSync (node:diagnostics_channel:328:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:245:24)
    at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:154:5)
    at node:internal/main/run_main_module:33:47 {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: [
    {
      offset: '0x2ffec0',
      slice1: '06000400000000000300000059ab0000',
      slice2: '060004000000000003000000b8aa0000'
    },
    {
      offset: '0x476eb0',
      slice1: '00000059ab0000580701106100000000',
      slice2: '000000b8aa0000580701106100000000'
    }
  ],
  expected: [],
  operator: 'deepStrictEqual',
  diff: 'simple'
}

Node.js v26.0.0-pre
--- stdout ---
0x0: Write magic 143da19
0x4: Write metadata
0x35: Write snapshot blob
0x258bfc: Write IsolateDataSerializeInfo
0x25a299: Write EnvSerializeInfo
0x25b1c0: Write CodeCacheInfo
0x0: Write magic 143da19
0x4: Write metadata
0x35: Write snapshot blob
0x258bfc: Write IsolateDataSerializeInfo
0x25a299: Write EnvSerializeInfo
0x25b1c0: Write CodeCacheInfo
Command: out/Release/node /home/runner/work/node-v8/node-v8/node/test/parallel/test-snapshot-reproducible.js

===
=== 1 tests failed
===

Failed tests:
out/Release/node /home/runner/work/node-v8/node-v8/node/test/parallel/test-snapshot-reproducible.js

targos avatar Nov 09 '25 09:11 targos

Still happening. Looking at https://github.com/nodejs/node-v8/actions/workflows/test-linux.yml:

  • Oct 27 (V8 14.3.127) and earlier were OK
  • Oct 28 (V8 14.4.7) first failure: https://github.com/v8/v8/compare/14.3.127...14.4.7
  • Oct 29 (V8 14.4.13) back to OK: https://github.com/v8/v8/compare/14.4.7...14.4.13
  • Oct 30 (V8 14.4.20) and later all fail: https://github.com/v8/v8/compare/14.4.13...14.4.20

targos avatar Nov 15 '25 06:11 targos

I suspect https://github.com/v8/v8/commit/db44401fef4522abdc2430652d9c602b6e45baf7 It was relanded on Oct 28, reverted on Oct 29, and relanded again on Oct 30.

targos avatar Nov 15 '25 07:11 targos

Still failing: https://github.com/nodejs/node-v8/actions/runs/19816998138/job/56770380628

targos avatar Dec 02 '25 08:12 targos

Looks like the code cache reproducibility is altered by some recent change, going to take a look when I get back to a powerful enough build machine..

joyeecheung avatar Dec 02 '25 09:12 joyeecheung

I bisected it to https://chromium-review.googlesource.com/c/v8/v8/+/7139418

targos avatar Dec 13 '25 13:12 targos

Looks like it broke the code cache reproducibility somehow

< Write<builtins::CodeCacheInfo>() id = internal/errors, length=97160, hash=3258954470
> Write<builtins::CodeCacheInfo>() id = internal/errors, length=97160, hash=2884610497

joyeecheung avatar Dec 16 '25 16:12 joyeecheung

It's more complex than this. If I revert that commit on the V8 14.4 update branch, the test still fails.

targos avatar Dec 17 '25 08:12 targos

I noticed that the debug build has been failing on canary, this gyp change should fix it: https://github.com/joyeecheung/node/tree/fix-debug - where should I submit this?

joyeecheung avatar Dec 17 '25 18:12 joyeecheung

Also opened https://github.com/nodejs/node/pull/61101 to help diffing the snapshots

joyeecheung avatar Dec 17 '25 19:12 joyeecheung

It seems the diffing object is a class boilerplate, or it appears to be its static_properties_template

0x224633025a69: [ClassBoilerplate] in OldSpace
 - map: 0x0ec5791c1f89 <Map[64](CLASS_BOILERPLATE_TYPE)>
 - arguments_count: 3
 - static_properties_template: 0x224633025969 <DescriptorArray[4]>
 - static_elements_template: 0x0ec5791c8f99 <NumberDictionary[7]>
 - static_computed_properties: 0x0ec5791c0dd1 <FixedArray[0]>
 - instance_properties_template: 0x224633025a31 <DescriptorArray[1]>
 - instance_elements_template: 0x0ec5791c8f99 <NumberDictionary[7]>
 - instance_computed_properties: 0x0ec5791c0dd1 <FixedArray[0]>

I have a feeling that it might be the same as the bug fixed by https://chromium-review.googlesource.com/c/v8/v8/+/7252791, only that this happens to the descriptor array instead of transition array. Looknig..

joyeecheung avatar Dec 17 '25 19:12 joyeecheung

I can fix it locally with this:

diff --git a/deps/v8/src/objects/objects.cc b/deps/v8/src/objects/objects.cc
index aa3c306212..5e8a1ed3c9 100644
--- a/deps/v8/src/objects/objects.cc
+++ b/deps/v8/src/objects/objects.cc
@@ -3997,6 +3997,9 @@ void DescriptorArray::Initialize(Tagged<EnumCache> empty_enum_cache,
   set_enum_cache(empty_enum_cache, SKIP_WRITE_BARRIER);
   set_flags(FastIterableBits::encode(FastIterableState::kUnknown),
             kRelaxedStore);
+#if TAGGED_SIZE_8_BYTES
+  set_optional_padding(0);
+#endif
   MemsetTagged(GetDescriptorSlot(0), undefined_value,
                number_of_all_descriptors() * kEntrySize);
 }

Going to upstream it

joyeecheung avatar Dec 17 '25 23:12 joyeecheung

I noticed that the debug build has been failing on canary, this gyp change should fix it: https://github.com/joyeecheung/node/tree/fix-debug - where should I submit this?

You can directly cherry-pick and push it to canary-base

targos avatar Dec 18 '25 06:12 targos

You can directly cherry-pick and push it to canary-base

Done, thanks for the pointer

joyeecheung avatar Dec 18 '25 14:12 joyeecheung

Upstream CL: https://chromium-review.googlesource.com/c/v8/v8/+/7271645

joyeecheung avatar Dec 18 '25 16:12 joyeecheung

Thanks for the investigation and fix @joyeecheung !

targos avatar Dec 22 '25 14:12 targos