benchmarking

benchmarking copied to clipboard

This is a followup from a conversation I had last night with @littledan.

It would be great if we could put together some benchmarks of "node centric workloads" for the v8 team to profile against.

Chrome has a system called Telemetry for running benchmarks, described at https://www.chromium.org/developers/telemetry . This is how we automate SunSpider, Octane, etc in continuous integration. I can't speak for the infrastructure team, but if you put a patch out for review to integrate your benchmarks into this system, similar to https://code.google.com/p/chromium/codesearch#chromium/src/tools/perf/benchmarks/sunspider.py , then you'll maximize your chances of getting it run in an automated way by us. Another helpful thing would be if the benchmarks were written so that they could be run in the browser, without node. That would make it easier for us to run, and also remove any way for us to say, "that must've been Node's fault". I can't say how we'll prioritize optimizing against these benchmarks, but it can only be helpful for us to have more data.

@ofrobots do you think this still makes sense ? (ie providing benchmarks to the v8 team)

@mhdawson We (node-team at Google) do run AcmeAir internally on a regular basis and provide data & feedback to the V8 team. Having more representative workloads would be most welcome. I am wary of micro-benchmarks though.