Levelized Files

[SSoelvsten]

Motivation

1. A lot of our files are of the temporary read-once-and-produce-output type (specifically arc_file and node_file with move-semantics). Hence, to be able to squeeze more BDDs / ZDDs into a smaller disk, we should continually deallocate the input file.
2. There is an upper limit to the file size for each file system, e.g. ext4 does not support files larger than 16 TiB.

Solution

The meta_file is turned into a list shared_ptr to files (or a file of files?). When a stream hooks into a read-once file, it removes the file for each level, when it has finished reading it.

Other Benefits

If there is one file per level and each node still points to the label rather than the level, then we can map labels to levels and that way support the swapping of two levels for variable reordering!

[SSoelvsten]

Alternative: Incrementally Deallocate Temporary Files

A better alternative to solve motivation 1 and would also make everyone using TPIE love us is to delete the part of a file that already was read. Specifically, this was described by Mathias Rav in https://github.com/thomasmoelhave/tpie/issues/227 .