Submitting my version

[obourgain]

My implementation is in dev.morling.onebrc.CalculateAverage_obourgain and runnable with provided script calculate_average_obourgain.sh. Runs with standard JDK 21.

On my computer (M1 pro, 10 cores, 32GB ram) my best run is (file fully in page cache): real 0m2.867s user 0m23.956s sys 0m1.329s

As I wrote in comments in the code, I have a few different roundings that the reference implementation. I have seend that there is an issue about that, but no specific rule yet.

Main points:

• use MemorySegment, it's faster than ByteBuffer
• split the work with 50 MB chunk and distribute to a thread pool
• fast measurement parser by using a lot of domain knowledge
• low allocation
• visiting each byte only once after the setup

Hope I won't have a bad surprise when running on the target server 😱

[gunnarmorling]

This fails one of the tests:

./tests.sh obourgain
Validating calculate_average_obourgain.sh -- src/test/resources/samples/measurements-10000-unique-keys.txt

real	0m0.257s
user	0m0.618s
sys	0m0.079s
Validating calculate_average_obourgain.sh -- src/test/resources/samples/measurements-10.txt

real	0m0.110s
user	0m0.099s
sys	0m0.038s
Validating calculate_average_obourgain.sh -- src/test/resources/samples/measurements-1.txt

real	0m0.126s
user	0m0.094s
sys	0m0.061s
Validating calculate_average_obourgain.sh -- src/test/resources/samples/measurements-20.txt

real	0m0.128s
user	0m0.106s
sys	0m0.052s
Validating calculate_average_obourgain.sh -- src/test/resources/samples/measurements-2.txt

real	0m0.120s
user	0m0.107s
sys	0m0.043s
Validating calculate_average_obourgain.sh -- src/test/resources/samples/measurements-3.txt

real	0m0.130s
user	0m0.113s
sys	0m0.047s
1c1
< {Bosaso=-15.0/1.2/20.0, Petropavlovsk-Kamchatsky=-9.5/0.0/9.5}
---
> {Bosaso=-15.0/1.3/20.0, Petropavlovsk-Kamchatsky=-9.5/0.0/9.5}

Could you take a look? In regards to rounding, the behavior should be the same as that of the RI.

[hastebrot]

@obourgain your version with BitTwiddledMap was already very fast. I wonder what resulted in the tests to fail, and how a fix would influence the performance.

[obourgain]

I pushed a fixed version. All test pass. @hastebrot the test failure was due to a difference in rounding, no measurable difference in rounding. The BitTwiddledMap was from a previous submission from royvanrijn but didn't support collision. I now have a more generic version.

[obourgain]

@gunnarmorling my submission passes the tests now, could you run it again please?

[gunnarmorling]

00:09.062, nice one! Currently on #3. Thx for particpating!