java-design-patterns icon indicating copy to clipboard operation
java-design-patterns copied to clipboard

Published 20 hours ago • verified publisher icon iluwatar

Store and Process pattern

Open RemcoBuddelmeijer opened this issue 6 years ago • 16 comments

Description: The Store and Process design pattern focuses on decoupling data storage from data processing to allow for flexible and scalable data stream handling. This pattern is particularly useful in scenarios where data needs to be ingested, stored, and then processed asynchronously. The main elements of this pattern include:

  1. Data Store: A storage component that receives and holds incoming data. This component should be capable of handling high-throughput data ingestion and provide reliable storage.
  2. Processor: A processing component that retrieves data from the store, processes it, and outputs the results. This processor can work independently of the data ingestion process, allowing for flexible and scalable data handling.
  3. Decoupling: By separating the storage and processing responsibilities, the system can scale each component independently, optimizing for performance and resource usage.

References:

Acceptance Criteria:

  1. Implement a data storage component that can ingest and store high-throughput data reliably.
  2. Implement a data processing component that retrieves data from the storage component and processes it asynchronously.
  3. Ensure the data storage and processing components are decoupled to allow independent scaling and optimization.

RemcoBuddelmeijer avatar Aug 15 '19 07:08 RemcoBuddelmeijer

@iluwatar Can I pick this up?

RemcoBuddelmeijer avatar Aug 22 '19 11:08 RemcoBuddelmeijer

Sure @RemcoBuddelmeijer please go ahead

iluwatar avatar Aug 22 '19 13:08 iluwatar

@RemcoBuddelmeijer are you still workin on this?

iluwatar avatar Jul 06 '20 16:07 iluwatar

@iluwatar I had a bit of a break, but I will continue with it some time later.

RemcoBuddelmeijer avatar Jul 12 '20 14:07 RemcoBuddelmeijer

Ok @RemcoBuddelmeijer, let me know if you need help

iluwatar avatar Jul 14 '20 12:07 iluwatar

@iluwatar No need, I forgot about this PR due to volume of work. Reminder helped me.

RemcoBuddelmeijer avatar Jul 19 '20 15:07 RemcoBuddelmeijer

@RemcoBuddelmeijer are you still working on this?

iluwatar avatar Oct 09 '21 19:10 iluwatar

This issue is free for taking again.

iluwatar avatar Oct 24 '21 15:10 iluwatar

Reopening the issue, since it hasn't been implemented yet

iluwatar avatar Jul 01 '22 15:07 iluwatar

I just now noticed that this has been going under my radar for all those years. I think 3 years is the charm. I'll be happy to pick this up and fix it up immediatelly @iluwatar

RemcoBuddelmeijer avatar Jul 01 '22 17:07 RemcoBuddelmeijer

Haha, I'll assign the issue to you @RemcoBuddelmeijer. Sorry for late reply!

iluwatar avatar Sep 21 '22 18:09 iluwatar

Haha, I'll assign the issue to you @RemcoBuddelmeijer. Sorry for late reply!

No worries at all! Looks like we are both prone to giving late responses 😄

RemcoBuddelmeijer avatar Sep 21 '22 20:09 RemcoBuddelmeijer

Bump. Will start working on this issue this weekend. Busy week but will attempt to finish soon.

Update: 22nd of October. Right now I don't see any foreseeable time that I can work on this.

RemcoBuddelmeijer avatar Oct 06 '22 10:10 RemcoBuddelmeijer

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Nov 28 '22 14:11 stale[bot]