imp
imp copied to clipboard
RMF parallel read/write
A common operation in PMI is gathering frames from many RMF files and writing them to a summary RMF file. Currently this can only be done on a single CPU - you have to read the RMF, link the hierarchy, load frame from input, save frame to output, and repeat on another input file. It's quite slow, especially if you have hundreds of files to sort though. It would be nice if we could do it in parallel.
An option is to recursively combine RMFs on multiple CPUs, but this is probably expensive because you'd have to create temporary RMF files to store the intermediate results (unless there is an internal way to store RMF "frames" without actually writing to a file?).
Any other ideas?