Object instantiation benchmark tests

[tshemsedinov]

Prepare execution scripts and run tests with different parameters:

• iteration number (1.000-10.000.000)
• different execution environments:
  • Browsers: Chrome, Firefox
  • Node.js 4.x, 5.x, 6.x, 7.x (a few of them)
  • OS: Fedora, CentOS, Win, FreeBSD, MacOS, etc. (a few of them)
• save results to CSV file and make charts:
  • x (iteration number), y (time)
  • x (iteration number), y (memory allocation) That's your task @NikitaRodan

[aqrln]

Node.js 4.x, 5.x, 6.x, 7.x

Not sure we should waste time on testing 5.x. 7.x is current, 4.x and 6.x are LTS, but 5.x is deprecated and unmaintained.

[tshemsedinov]

6.x and 7.x is enough. Do we need current master? @aqrln

[aqrln]

Would be nice to have. It has more up-to-date V8 version.

[tshemsedinov]

@aqrln it's strange but create empty {} and mixin to it is faster then define { a, b, c }

defineObject.............2303805712 nanoseconds
defineArray..............1530630853 nanoseconds
defineArrayOfString......1802242542 nanoseconds
defineArrayOfNumber......1228746475 nanoseconds
mixinObject..............1761211178 nanoseconds
newPrototype.............4738207423 nanoseconds
newClass.................3524757292 nanoseconds
newObject................1731936724 nanoseconds
objectCreate.............5292502346 nanoseconds
callFactory..............4647938265 nanoseconds

[tshemsedinov]

Oh, it seems that gc influences test significantly, adding --nouse-idle-notification changes test:

defineObject.............1499449812 nanoseconds
defineArray..............2422338520 nanoseconds
defineArrayOfString......1417896249 nanoseconds
defineArrayOfNumber......1303653103 nanoseconds
mixinObject..............1976549087 nanoseconds
newPrototype.............4859534729 nanoseconds
newClass.................3834918285 nanoseconds
newObject................1713897136 nanoseconds
objectCreate.............4541449690 nanoseconds
callFactory..............4426530909 nanoseconds

[aqrln]

@tshemsedinov I'd also check the IR to see how V8 compiles it. Microbenchmarks are very tricky to get done right.

[tshemsedinov]

But the fact is that they have become fast and will often win between competitors. That's a good news for me, I :heart: mixins, you know. Next weekend we will definitely make meetup for benchmarking and V8 performance, initial meetup to discuss method, tools and prepare tasks.

[tshemsedinov]

Another strange case that newPrototype is faster then newClass but I think we need to run each test in separate process one after another and with disabled gc

[aqrln]

@tshemsedinov not necessarily they, that might as well be new optimizations that just eat your benchmarked functions and leave only little crumbs :)

If https://github.com/HowProgrammingWorks/Benchmark/blob/master/JavaScript/3-instantiation.js is the benchmark you are talking about, I'd definitely check out the compiled code before making any conclusions. From what I see, after inlining benchmarked functions, there will be lots of loop invariants that V8 usually pulls out of the loop, so many of these tests most probably benchmark nothing.

[aqrln]

I'd also recommend you to check out this video or other talks by Vyacheslav Egorov, one of the V8 developers. _{Spoiler: due to how V8 works, the best way to write microbenchmarks is to never write microbenchmarks, but if you absolutely must do so, there are certain techniques that should be used and certain situation you have to be aware of.}