Submission @jbachorik

[jbachorik]

Thanks for a very interesting 'etude' :) Here is something using just plain Java with a bit of mmapping and direct byte buffers. Didn't use any of the new shiny Panama stuff, though ...

Class: dev.morling.onebrc.CalculateAverage_jbachorik Specs: Apple M1 Max, 64GiB

Check List:

• [x] Tests pass (./test.sh <username>)
• [x] All formatting changes by the build are committed
• [x] Output matches that of calculate_average_baseline.sh

• Execution time:

real    0m6.717s
user    0m50.071s
sys     0m1.340s

• Execution time of reference implementation:

real    3m22.003s
user    3m18.195s
sys     0m3.946s

[gunnarmorling]

Hey @jbachorik, do the tests pass for you locally?

I'm getting this:

Validating calculate_average_jbachorik.sh -- src/test/resources/samples/measurements-10000-unique-keys.txt
Exception in thread "main" java.lang.IllegalArgumentException: Split size must be at least 128 bytes
	at dev.morling.onebrc.CalculateAverage_jbachorik.mmap(CalculateAverage_jbachorik.java:375)
	at dev.morling.onebrc.CalculateAverage_jbachorik.main(CalculateAverage_jbachorik.java:198)

[gunnarmorling]

Still getting the same error. Please run mvn clean verify && ./test.sh jbachorik to make sure they all pass (please rebase to latest main before doing so to have all the latest tests). Thx!

[jbachorik]

Hi, I apologize for a long reply ... I have been AFK for this week and had just a few moments on internet. I fixed the test (d'oh) and also took the liberty to directly propose my second iteration I have already been working on - the main point of the second take is to avoid double work. With those changes I really seem to be hitting the bottleneck of the bytebuffer where each access is verified and getLong(idx) shows as the major top consumer on my profiles ....

[gunnarmorling]

Hey @jbachorik, so what's your output of ./test.sh jbachorik? I'm still having issues there. I got a warning about using pre-touch, which helped to get rid of one of the unexpected output lines. Still getting this one though:

Validating calculate_average_jbachorik.sh -- src/test/resources/samples/measurements-shortest.txt
java.lang.IndexOutOfBoundsException
	at java.base/java.nio.Buffer$1.apply(Buffer.java:757)
	at java.base/java.nio.Buffer$1.apply(Buffer.java:754)
	at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:213)
	at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:210)
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:98)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:106)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:302)
	at java.base/java.nio.Buffer.checkIndex(Buffer.java:779)
	at java.base/java.nio.DirectByteBuffer.getLong(DirectByteBuffer.java:877)
	at dev.morling.onebrc.CalculateAverage_jbachorik.processChunk(CalculateAverage_jbachorik.java:242)
	at dev.morling.onebrc.CalculateAverage_jbachorik.lambda$main$1(CalculateAverage_jbachorik.java:188)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

real	0m0.200s
user	0m0.170s
sys	0m0.067s
1c1
< {}
---
> {a=1.0/1.0/1.0}

Reason probably is that you still need to rebase to current main, so as to make sure all the tests are run and pass (we added some more earlier this week, covering some corner cases of the spec).

[gunnarmorling]

Yepp, looking good now. 00:15.752 on the eval machine. Thx for participating!