kamu-cli icon indicating copy to clipboard operation
kamu-cli copied to clipboard

Implement batch loading of event sourcing aggregates (Flows, Tasks)

Open zaychenko-sergei opened this issue 1 year ago • 1 comments

Currently we do not support batch loading of flows & tasks, and load them in sequential loops like below:

            // TODO: implement batch loading
            for flow_id in relevant_flow_ids {
                let flow = Flow::load(flow_id, self.flow_event_store.as_ref()).await.int_err()?;
                yield flow.into();
            }

Knowing a set of queries (id's in our case), it should be possible to load all related events in 1 SQL query in chronological order. Then event_sourcing crate could post-process the loaded collection of events, and re-construct multiple aggregates in 1 call.

This should optimize database roundtrips, esp. for listing flows:

image

zaychenko-sergei avatar Sep 25 '24 14:09 zaychenko-sergei

Cool, tracing is the best visual way to know that optimization is needed.

Yes, we knew this before, but when you can see this need on the graph, superb!

s373r avatar Sep 25 '24 15:09 s373r

@demusdev Consult with @rmn-boiko for detailed supervision

zaychenko-sergei avatar Dec 26 '24 09:12 zaychenko-sergei