java-design-patterns icon indicating copy to clipboard operation
java-design-patterns copied to clipboard

Published 20 hours ago • verified publisher icon iluwatar

Store and Process pattern

Open NLRemco opened this issue 5 years ago • 14 comments

Description: The Store and Process design pattern focuses on decoupling data storage from data processing to allow for flexible and scalable data stream handling. This pattern is particularly useful in scenarios where data needs to be ingested, stored, and then processed asynchronously. The main elements of this pattern include:

  1. Data Store: A storage component that receives and holds incoming data. This component should be capable of handling high-throughput data ingestion and provide reliable storage.
  2. Processor: A processing component that retrieves data from the store, processes it, and outputs the results. This processor can work independently of the data ingestion process, allowing for flexible and scalable data handling.
  3. Decoupling: By separating the storage and processing responsibilities, the system can scale each component independently, optimizing for performance and resource usage.

References:

Acceptance Criteria:

  1. Implement a data storage component that can ingest and store high-throughput data reliably.
  2. Implement a data processing component that retrieves data from the storage component and processes it asynchronously.
  3. Ensure the data storage and processing components are decoupled to allow independent scaling and optimization.

NLRemco avatar Aug 15 '19 07:08 NLRemco

@iluwatar Can I pick this up?

NLRemco avatar Aug 22 '19 11:08 NLRemco

Sure @RemcoBuddelmeijer please go ahead

iluwatar avatar Aug 22 '19 13:08 iluwatar

@RemcoBuddelmeijer are you still workin on this?

iluwatar avatar Jul 06 '20 16:07 iluwatar

@iluwatar I had a bit of a break, but I will continue with it some time later.

NLRemco avatar Jul 12 '20 14:07 NLRemco

Ok @RemcoBuddelmeijer, let me know if you need help

iluwatar avatar Jul 14 '20 12:07 iluwatar

@iluwatar No need, I forgot about this PR due to volume of work. Reminder helped me.

NLRemco avatar Jul 19 '20 15:07 NLRemco

@RemcoBuddelmeijer are you still working on this?

iluwatar avatar Oct 09 '21 19:10 iluwatar

This issue is free for taking again.

iluwatar avatar Oct 24 '21 15:10 iluwatar

Reopening the issue, since it hasn't been implemented yet

iluwatar avatar Jul 01 '22 15:07 iluwatar

I just now noticed that this has been going under my radar for all those years. I think 3 years is the charm. I'll be happy to pick this up and fix it up immediatelly @iluwatar

NLRemco avatar Jul 01 '22 17:07 NLRemco

Haha, I'll assign the issue to you @RemcoBuddelmeijer. Sorry for late reply!

iluwatar avatar Sep 21 '22 18:09 iluwatar

Haha, I'll assign the issue to you @RemcoBuddelmeijer. Sorry for late reply!

No worries at all! Looks like we are both prone to giving late responses 😄

NLRemco avatar Sep 21 '22 20:09 NLRemco

Bump. Will start working on this issue this weekend. Busy week but will attempt to finish soon.

Update: 22nd of October. Right now I don't see any foreseeable time that I can work on this.

NLRemco avatar Oct 06 '22 10:10 NLRemco

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Nov 28 '22 14:11 stale[bot]