hep
hep copied to clipboard
rootio: implement buffer pre-load or buffered I/O
commit 40f83711f2ddda82f19c00cb7f5be5eaf85ebb8f brought the reading of a float64 branch from/to:
BenchmarkScanStruct-4 50000 39159 ns/op 801 B/op 100 allocs/op
BenchmarkScanStruct-4 200000 10070 ns/op 0 B/op 0 allocs/op
ie:
name old time/op new time/op delta
ScanStruct-4 39.2µs ± 0% 10.1µs ± 0% ~ (p=1.000 n=1+1)
name old alloc/op new alloc/op delta
ScanStruct-4 801B ± 0% 0B ±NaN% ~ (p=1.000 n=1+1)
name old allocs/op new allocs/op delta
ScanStruct-4 100 ± 0% 0 ±NaN% ~ (p=1.000 n=1+1)
that's quite good, but we are still very far from ROOT-C++
:
$> make bench
g++ -o cxx-read-data -O2 `root-config --libs --cflags` read-data.cxx
go1.9rc2 build -o go-read-data read-data.go
=== NO COMPRESSION ===
=== ROOT ===
real=6.70 user=6.18 sys=0.51 CPU=99% MaxRSS=258296 I/O=0/0
real=6.84 user=6.32 sys=0.51 CPU=99% MaxRSS=257748 I/O=0/0
real=6.82 user=6.29 sys=0.52 CPU=99% MaxRSS=258348 I/O=0/0
real=6.66 user=6.13 sys=0.53 CPU=100% MaxRSS=258440 I/O=0/0
=== go-hep/rootio ===
real=12.94 user=12.39 sys=0.56 CPU=100% MaxRSS=42028 I/O=0/0
real=12.93 user=12.37 sys=0.56 CPU=100% MaxRSS=42072 I/O=0/0
real=12.96 user=12.38 sys=0.58 CPU=100% MaxRSS=41984 I/O=0/0
real=12.94 user=12.36 sys=0.57 CPU=100% MaxRSS=42048 I/O=0/0
=== WITH COMPRESSION ===
=== ROOT ===
real=20.61 user=11.86 sys=0.63 CPU=60% MaxRSS=292640 I/O=869104/0
real=12.56 user=11.54 sys=0.51 CPU=96% MaxRSS=290124 I/O=48104/0
real=12.04 user=11.50 sys=0.52 CPU=99% MaxRSS=290444 I/O=0/0
real=12.05 user=11.54 sys=0.50 CPU=99% MaxRSS=290324 I/O=0/0
=== go-hep/rootio ===
real=36.43 user=35.20 sys=0.69 CPU=98% MaxRSS=81196 I/O=2464/0
real=35.75 user=35.15 sys=0.63 CPU=100% MaxRSS=81644 I/O=0/0
real=35.76 user=35.10 sys=0.69 CPU=100% MaxRSS=81856 I/O=0/0
real=35.70 user=35.18 sys=0.54 CPU=100% MaxRSS=81944 I/O=0/0
we should probably pre-fetch baskets or get some basket-oriented buffered I/O.