bufbuild
Serialization Performance Degredation Between v1 and v2
Hey, we're looking into upgrading from v1 to v2. However it appears that the serialization throughput has suffered a bit between the two versions.
- Is the performance degradation as expected?
- Can we expect an improvement in the near future?
I saw these PR & issue https://github.com/bufbuild/protobuf-es/pull/964 https://github.com/bufbuild/protobuf-es/issues/333 but they've had their last activity in august.
Test results, running on node v23.4.0. Can pull into a repo & post here if desired. Tested is performance of binary de-/serialization.
Results for 'serialization A (671224 bytes)'
┌─────────┬──────────────────┬─────────┬───────────────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ms) │ Samples │
├─────────┼──────────────────┼─────────┼───────────────────┼─────────┤
│ 0 │ 'bufbuild-es v1' │ '19.86' │ '50.345230' │ 100 │
│ 1 │ 'bufbuild-es v2' │ '16.58' │ '60.320838' │ 83 │
└─────────┴──────────────────┴─────────┴───────────────────┴─────────┘
Results for 'deserialization A (671224 bytes)'
┌─────────┬───────────────────┬──────────┬───────────────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ms) │ Samples │
├─────────┼───────────────────┼──────────┼───────────────────┼─────────┤
│ 0 │ 'bufbuild-es v1' │ '117.65' │ '8.499707' │ 589 │
│ 1 │ 'bufbuild-es v2 ' │ '26.19' │ '38.181408' │ 131 │
└─────────┴───────────────────┴──────────┴───────────────────┴─────────┘
Results for 'serialization B (2053246 bytes)'
┌─────────┬──────────────────┬─────────┬───────────────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ms) │ Samples │
├─────────┼──────────────────┼─────────┼───────────────────┼─────────┤
│ 0 │ 'bufbuild-es v1' │ '4.30' │ '232.616600' │ 22 │
│ 1 │ 'bufbuild-es v2' │ '3.50' │ '285.417292' │ 18 │
└─────────┴──────────────────┴─────────┴───────────────────┴─────────┘
Results for 'deserialization B (2053246 bytes)'
┌─────────┬───────────────────┬─────────┬───────────────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ms) │ Samples │
├─────────┼───────────────────┼─────────┼───────────────────┼─────────┤
│ 0 │ 'bufbuild-es v1' │ '19.14' │ '52.252213' │ 96 │
│ 1 │ 'bufbuild-es v2 ' │ '6.65' │ '150.382447' │ 34 │
└─────────┴───────────────────┴─────────┴───────────────────┴─────────┘
Results for 'serialization C (110073 bytes)'
┌─────────┬──────────────────┬──────────┬───────────────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ms) │ Samples │
├─────────┼──────────────────┼──────────┼───────────────────┼─────────┤
│ 0 │ 'bufbuild-es v1' │ '118.44' │ '8.442998' │ 593 │
│ 1 │ 'bufbuild-es v2' │ '100.69' │ '9.931406' │ 504 │
└─────────┴──────────────────┴──────────┴───────────────────┴─────────┘
Results for 'deserialization C (110073 bytes)'
┌─────────┬───────────────────┬──────────┬───────────────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ms) │ Samples │
├─────────┼───────────────────┼──────────┼───────────────────┼─────────┤
│ 0 │ 'bufbuild-es v1' │ '825.05' │ '1.212047' │ 4126 │
│ 1 │ 'bufbuild-es v2 ' │ '150.96' │ '6.624226' │ 755 │
└─────────┴───────────────────┴──────────┴───────────────────┴─────────┘
Results for 'serialization D (11007 bytes)'
┌─────────┬──────────────────┬──────────┬───────────────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ms) │ Samples │
├─────────┼──────────────────┼──────────┼───────────────────┼─────────┤
│ 0 │ 'bufbuild-es v1' │ '633.65' │ '1.578170' │ 3169 │
│ 1 │ 'bufbuild-es v2' │ '571.39' │ '1.750106' │ 2857 │
└─────────┴──────────────────┴──────────┴───────────────────┴─────────┘
Results for 'deserialization D (11007 bytes)'
┌─────────┬───────────────────┬───────────┬───────────────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ms) │ Samples │
├─────────┼───────────────────┼───────────┼───────────────────┼─────────┤
│ 0 │ 'bufbuild-es v1' │ '3152.96' │ '0.317162' │ 15765 │
│ 1 │ 'bufbuild-es v2 ' │ '1093.42' │ '0.914564' │ 5468 │
└─────────┴───────────────────┴───────────┴───────────────────┴─────────┘
Hey Silas, object initialization can be a bit slower without classes, and the reflection layer adds some overhead. We're aware of that, and are considering to optionally generate speed-optimized code for initialization and serialization (should solve https://github.com/bufbuild/protobuf-es/issues/333), but not in the near future.
https://github.com/bufbuild/protobuf-es/pull/964 uses resizable buffers for encoding. I expect this to give a decent perf bump because it can avoid many allocations, but the ECMAScript feature is not widely available yet. It may be possible to implement it as an alternative BinaryEncoder with the same interface, and let users opt in.
