openmeter icon indicating copy to clipboard operation
openmeter copied to clipboard

Published 20 hours ago • verified publisher icon openmeterio

RFC: split StreamingConnector into ingestion and aggregation component

Open sagikazarmark opened this issue 1 year ago • 3 comments

Currently ingestion and aggregation are tightly coupled together. I believe they should be two separate domains, even though the components of the underlying technology are tightly coupled as well.

By separating these modules it becomes easier to reason about the system as each component (ingestion, aggregation) will be responsible for their part only. It also becomes easier to cover the system with tests.

sagikazarmark avatar Jun 13 '23 08:06 sagikazarmark

Ingestion component has been moved out to its own module. Aggregation needs some more refactors.

sagikazarmark avatar Jul 17 '23 15:07 sagikazarmark

@hekike can you share your architectural diagram here?

I think we are close to getting this done.

sagikazarmark avatar Aug 22 '23 00:08 sagikazarmark

Interface@2x (5)

hekike avatar Aug 23 '23 15:08 hekike