Reasoner retrievable traversals should be async and parallel

[flyingsilverfin]

Problem to Solve

In #6209 we reverted a prior change #6176, which enabled the main traversals in reasoner to run using asynchronous producers. However, this caused random stalling in traversals in CI (but not locally), and we were unable to track down the issue in a timely manner. We should re-visit this feature and ensure the stalling issue is not a critical race condition in asynchronous producers as well.

Current Workaround

Reasoner only does traversals synchronously, using iterators.

Proposed Solution

Reasoner should run large traversals asynchronously using the TraversalEngine.producer() method, rather than .iterator(). We should at begin by introducing it for RetrievableResolver, which does most large traversals.

We should also consider that this could over-parallelise, so we could tune parallelisation based on depth of the reasoning execution - nearer the root should be more parallel compared to leaves.

Additional Information