cbor-x icon indicating copy to clipboard operation
cbor-x copied to clipboard

Streaming Uint8Array very slow

Open MeirionHughes opened this issue 8 months ago • 0 comments

Stumbled upon a weird issue when testing through-put, whereby streaming raw Uint8Array into the encoder/decoder causes orders of magnitude performance issue. I can't see anything on the options to force objectMode (which I assume is the issue).

for sake of repo:

import  { EncoderStream, DecoderStream }  from'cbor-x';

let i = 0, j =0, N = 2, M = 1024;

let payloads = new Array(N).fill([]).map(x=>new Uint8Array(M).fill(0).map(y=>Math.random()*255));

const encoderStream = new EncoderStream();
const decoderStream = new DecoderStream();

encoderStream.pipe(decoderStream);

for(let payload of payloads){
  encoderStream.write(payload);
}
encoderStream.end();

let start = Date.now();

decoderStream.on("data", (chunk)=>{
  console.log("read payload", chunk);
});

let clock = setTimeout(()=>{console.log("timed-out")}, 20000);

decoderStream.on("end", ()=>{
  console.log((N*M) + " bytes took ", + Date.now()-start  + " ms");
  clearTimeout(clock);
});

with encoderStream.write(payload); its about 10 seconds, while as object its 4ms encoderStream.write({payload});

This seems to be an issue if you send Float64Array (or any TypedArray) as the message.

Have I missed an option, not supported or bug?

MeirionHughes avatar Jun 06 '24 15:06 MeirionHughes