Benchmarks

[jz3xyz]

For those interested, here are a few anecdotal benchmarks which suggest a slight improvement in using wasm-lz4 over lz4

bag	lz4	wasm
129MB bag	14.16s	13.86s
129MB bag	14.72s	13.932s
129MB bag	14.14s	13.583s
801MB bag	1m31s	1m26s
801MB bag	1m28s	1m28s
801MB bag	1m28s	1m26s

lz4

    decompress: { lz4: buffer => Buffer.from(lz4.decompress(buffer)) },

wasm-lz4

    decompress: { lz4: decompress },

Hopefully, I'm not doing anything incorrectly to skew the numbers. :+1:

[janpaul123]

Is this in the browser or in nodejs? Do you have a Github Gist or so of how you ran this? From what I remember the difference was much more dramatic, but I think that was mostly in the browser.

[jz3xyz]

It's from nodejs version 12.11.1

node --version
v12.11.1

Code looks like this

const { open, TimeUtil } = require('rosbag')
const lz4 = require('lz4js')
const decompress = require('wasm-lz4')
const uuidv4 = require('uuid/v4')

const { performance } = require('perf_hooks');
const newTopics = []

// Begin Reading from Bag
const t0 = performance.now()
await bag.readMessages({
  topics: newTopics,
  decompress: { lz4: decompress },
}, result => {
  const { timestamp, message, topic } = result;
  // Convert message info to records
})
  .finally(() => { })
  .catch(err => { throw err })
const t1 = performance.now()
console.log(`[Converted] bag to X files in ${((t1 - t0) / 1000).toFixed(2)} seconds`)
//...

[janpaul123]

Ah that's still quite impressive since that library doesn't use any native extensions! 😮