Make comitting `O(N)`, as opposed to `O(S)`

[ureeves]

Summary

Currently, in an effort to make commits independent from others, we hard link each page that doesn't change from the previous(base) commit from every contract. This means that during each commit a number of operation is performed that is of the total size of the state (O(S)).

This is clearly undesirable, since having the time to commit scale linearly with the size of the overall state means that it will take a long time to commit for larger states. Not only that but over time, while the state grows, we may find ourselves exceeding the threshold of performance allowed by the consensus, simply for having the chain up for a long time.

It should be possible to have the time to commit scale at least linearly (O(N)) with the number of pages written to, if even with a cost involved when deleting commits. Since crumbles was introduced, squash_commit was removed.

Possible Solution

This might be a good time to introduce the use of a database such as rocksdb, especially if there's some difficulty optimizing the retrieval of pages the squashing of some commits into another.

The current content of a page can be retrieved by following the commit path until the latest contents are retrieved.