Further optimize performances and behavior of SMTs
Based on the following timings (taken on my CPU 6x2.2GHz (turbo 4.1GHz), 8GB RAM):
process_leaves_mnt4_comp
--Add 1000 leaves in a tree of height 23:
---Non-Lazy: 91.6s
---Lazy: 0.423s
--Remove 1000 leaves in a tree of height 23:
---Non-Lazy: 28.7s
---Lazy: 0.343s
Single leaf updates to a BigMerkleTree are still quite slow. Of course if we move away from the huge base field size of the MNT curves and an "inverse S-Box"-based Poseidon Hash, timings will be much much better; still this suggests the possibility to do some further optimizations like properly tuning DB parameters, reducing accesses to DB by means of additional in-memory data structures and so on;
Moreover, right now, the only way to recover a BigMerkleTree and a BigLazyMerkleTree due to an exceptional situation (like a crash or an external cause), is taking the leaves from the DB and rebuild the tree by re-adding the leaves. Even if this can be done much quicker using a BigLazyMerkleTree, this suggests the need to provide some optimized persistency mechanism with the additional requirement of consistency.
Investigate both.