root
root copied to clipboard
[ntuple] Improve RDF MT scheduling
Implement two improvements to the RNTuple RDF scheduler for the multi-threaded case:
- Files of a chain are opened in batches the background: while a batch of files (batch size == number of slots) is processed, the next batch is opened.
- For the tail scheduling (multiple slots sharing the same files), the clones of the page source pointing to the same file do not need to re-read meta-data (anchor, header, footer) again.
The patch results in a speed improvement of about 10% for AGC, EOS cached, 32 cores.
Test Results
13 files 13 suites 2d 17h 14m 49s :stopwatch: 2 651 tests 2 650 :white_check_mark: 0 :zzz: 1 :x: 32 645 runs 32 644 :white_check_mark: 0 :zzz: 1 :x:
For more details on these failures, see this check.
Results for commit 3d953c9a.
:recycle: This comment has been updated with latest results.