Adding kgeri's solution

[kgeri]

Implementation class: CalculateAverage_kgeri Execution time (i7-10700, 8 cores, 32GB):

158.20user 16.77system 0:15.75elapsed 1110%CPU (0avgtext+0avgdata 9038680maxresident)k
1230133inputs+188outputs (1814194major+106949minor)pagefaults 0swaps

The current implementation is parallel and no-alloc, the heavy lifting done via a MappedMemorySegment which is still in preview (but an awesome feature!).

[kgeri]

Getting this:

Exception in thread "main" java.lang.IndexOutOfBoundsException: Out of bound access on segment MemorySegment{ heapBase: Optional.empty address:140423103467995 limit: 1971184376 }; new offset = 1971184376; new length = 1
	at java.base/jdk.internal.foreign.AbstractMemorySegmentImpl.outOfBoundException(AbstractMemorySegmentImpl.java:426)
	at java.base/jdk.internal.foreign.AbstractMemorySegmentImpl.apply(AbstractMemorySegmentImpl.java:407)
	at java.base/jdk.internal.foreign.AbstractMemorySegmentImpl.apply(AbstractMemorySegmentImpl.java:69)
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:98)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:124)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:448)
	at java.base/jdk.internal.foreign.AbstractMemorySegmentImpl.checkBounds(AbstractMemorySegmentImpl.java:396)
	at java.base/jdk.internal.foreign.AbstractMemorySegmentImpl.checkAccess(AbstractMemorySegmentImpl.java:356)
	at java.base/java.lang.invoke.VarHandleSegmentAsBytes.checkAddress(VarHandleSegmentAsBytes.java:81)
	at java.base/java.lang.invoke.VarHandleSegmentAsBytes.get(VarHandleSegmentAsBytes.java:108)
	at java.base/java.lang.foreign.MemorySegment.get(MemorySegment.java:1361)
	at dev.morling.onebrc.CalculateAverage_kgeri.readChunk(CalculateAverage_kgeri.java:161)
	at dev.morling.onebrc.CalculateAverage_kgeri.lambda$main$0(CalculateAverage_kgeri.java:211)
	at java.base/java.util.stream.LongPipeline$1$1.accept(LongPipeline.java:177)
	at java.base/java.util.stream.Streams$RangeLongSpliterator.forEachRemaining(Streams.java:228)
	at java.base/java.util.Spliterator$OfLong.forEachRemaining(Spliterator.java:777)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.Nodes$SizedCollectorTask.compute(Nodes.java:1887)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:667)
	at java.base/java.util.stream.Nodes.collect(Nodes.java:325)
	at java.base/java.util.stream.ReferencePipeline.evaluateToNode(ReferencePipeline.java:111)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:570)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
	at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
	at dev.morling.onebrc.CalculateAverage_kgeri.main(CalculateAverage_kgeri.java:212)

Thanks - I think that should only happen if the input file doesn't have a \n at the end - committed a fix to handle that case as well.

[kgeri]

The output of this submission differs from that of the baseline impl. They must match :(

Nice... turned out it was a bug in chunking, making it dependent on the exact size of the file, so basically random 🤯 I could reproduce, and think I fixed it - can you check again, please.

[gunnarmorling]

00:28.386. Thx for participating!

[kgeri]

00:28.386. Thx for participating!

Thank you, this was fun and I learnt a lot!