root icon indicating copy to clipboard operation
root copied to clipboard

[ntuple] Method to prepare cluster commit / flush column write buffers

Open hahnjo opened this issue 1 year ago • 0 comments

With RNTupleWriter::Append / RNTupleParallelWriter::Append, the application must organize synchronization to the underlying TFile. To avoid locking for every Fill, https://github.com/root-project/root/pull/15239 introduced FillNoCommit and gives some guarantees so that locking is only necessary when calling CommitCluster. However that's still not ideal because CommitCluster will first call CommitCluster on every field, which will flush the column write buffers and trigger compression. This can be substantial for many fields or very large page sizes (compared to the cluster size). Ideally a RNTupleFillContext had a method to perform this work outside of the critical section, like PrepareCommitCluster or FlushPageBuffers.

hahnjo avatar Aug 14 '24 15:08 hahnjo