Optimization needed: SoilFileLockRegistry>>#addLock: slow for many locks

[MarcusDenker]

this example stores 130K strings (alltogether ~40MB) in an indexed dictionary.

(This is therefore not critical as this is not something happening in the use cases we target now, but should be fixed at some point)

soil := Soil path: 'soil-bench'.
	soil 
		destroy;
		initializeFilesystem.
dict := SoilBTreeDictionary  new
		keySize: 4;
		maxLevel: 8; "ignored for BTree"
		yourself.
		
tx := soil newTransaction.
tx root: dict.

Smalltalk globals allMethods do: [ :meth |
	dict at: meth sourcePointer put: meth sourceCode.
	].
tx commit.

The commit is very slow. Time is spend in SoilFileLockRegistry>>#addLock: seemingly

I had to comment out the already locked check to make it work. Doing a query works nicely after is is commited:

tx := soil newTransaction.
(tx root at: (OrderedCollection>>#do:) sourcePointer) soilRealObject

Problem:

The locks are in an OrderedCollection, checking them gets too slow when there are many (the example creates 130K locks).

We need a DataStucture that allows us to check interval overlap fast, e.g.

https://en.wikipedia.org/wiki/Interval_tree

[MarcusDenker]

Splitting the commits at a class boundary is much faster (25 seconds):

Smalltalk globals allClasses do: [ :cl  | | tr |
		tr := soil newTransaction.
		cl localMethods do:
			[ :meth |
			tr root at: meth sourcePointer put: meth sourceCode].
	tr commit.
]

[pdebruic]

Years ago I made an implementation of a Centered Interval Tree in pharo:

https://github.com/pdebruic/centered-interval-tree

I don't remember it being trash but it might be ;)