Tom Ridge

Another comparison: tree benchmark, with gc, irmin main, but ocaml 4.12 vs 4.14. Max memory usage was 1.335G in 4.12, 1.212G in 4.14 (so not much difference). Aside: Wall time...

At this point, it seems that the tree benchmarks do not exhibit 3x memory usage (at least in the runs detailed above). So investigation should continue with the lib_context benchmarks...

Are you saying that the main process uses 133MB maxrss at the point the child is forked, and that the child process then gets to 2442MB? The child process is...

When the worker runs, it creates a new prefix file (sparse file) and a new suffix file. The new suffix file is created by copying directly from the current suffix...

In #1974 it was reported that reads are greatly increased (irmin main vs 3.3.1). Irmin main branch is presumably similar to good-gc. But in the stats reported here, the reads...

Another possible cause of slowdown: when the main process switches to the new prefix (sparse file) it has to load a mapping file. Previously this was loaded very quickly, but...

Another possibility: suppose at the end of a commit, we issue an fsync (I don't know whether we do or not - likely we don't, but let's assume we do...),...

Just to say that the following might be useful sources of documentation: 1. The very old prototype https://github.com/tomjridge/sparse-file had quite a lot of code documentation; there was also some more...

Thanks for the write up (great!). Indeed, `unsafe_to_string` needs to be used with great care. Calling `Entry.decode (Bytes.unsafe_to_string scratch.buffer)` requires that you understand exactly how `Entry.decode` works (including all the...

Related to #1959