hep icon indicating copy to clipboard operation
hep copied to clipboard

rootio: implement buffer pre-load or buffered I/O

Open sbinet opened this issue 7 years ago • 0 comments

commit 40f83711f2ddda82f19c00cb7f5be5eaf85ebb8f brought the reading of a float64 branch from/to:

BenchmarkScanStruct-4              50000             39159 ns/op             801 B/op        100 allocs/op
BenchmarkScanStruct-4             200000             10070 ns/op               0 B/op          0 allocs/op

ie:

name          old time/op    new time/op    delta
ScanStruct-4    39.2µs ± 0%    10.1µs ± 0%   ~     (p=1.000 n=1+1)

name          old alloc/op   new alloc/op   delta
ScanStruct-4      801B ± 0%       0B ±NaN%   ~     (p=1.000 n=1+1)

name          old allocs/op  new allocs/op  delta
ScanStruct-4       100 ± 0%        0 ±NaN%   ~     (p=1.000 n=1+1)

that's quite good, but we are still very far from ROOT-C++:

$> make bench
g++ -o cxx-read-data -O2 `root-config --libs --cflags` read-data.cxx
go1.9rc2 build -o go-read-data read-data.go

=== NO COMPRESSION ===

=== ROOT ===
real=6.70 user=6.18 sys=0.51 CPU=99% MaxRSS=258296 I/O=0/0
real=6.84 user=6.32 sys=0.51 CPU=99% MaxRSS=257748 I/O=0/0
real=6.82 user=6.29 sys=0.52 CPU=99% MaxRSS=258348 I/O=0/0
real=6.66 user=6.13 sys=0.53 CPU=100% MaxRSS=258440 I/O=0/0

=== go-hep/rootio ===
real=12.94 user=12.39 sys=0.56 CPU=100% MaxRSS=42028 I/O=0/0
real=12.93 user=12.37 sys=0.56 CPU=100% MaxRSS=42072 I/O=0/0
real=12.96 user=12.38 sys=0.58 CPU=100% MaxRSS=41984 I/O=0/0
real=12.94 user=12.36 sys=0.57 CPU=100% MaxRSS=42048 I/O=0/0


=== WITH COMPRESSION ===

=== ROOT ===
real=20.61 user=11.86 sys=0.63 CPU=60% MaxRSS=292640 I/O=869104/0
real=12.56 user=11.54 sys=0.51 CPU=96% MaxRSS=290124 I/O=48104/0
real=12.04 user=11.50 sys=0.52 CPU=99% MaxRSS=290444 I/O=0/0
real=12.05 user=11.54 sys=0.50 CPU=99% MaxRSS=290324 I/O=0/0

=== go-hep/rootio ===
real=36.43 user=35.20 sys=0.69 CPU=98% MaxRSS=81196 I/O=2464/0
real=35.75 user=35.15 sys=0.63 CPU=100% MaxRSS=81644 I/O=0/0
real=35.76 user=35.10 sys=0.69 CPU=100% MaxRSS=81856 I/O=0/0
real=35.70 user=35.18 sys=0.54 CPU=100% MaxRSS=81944 I/O=0/0

we should probably pre-fetch baskets or get some basket-oriented buffered I/O.

sbinet avatar Aug 17 '17 17:08 sbinet