samples-typescript

samples-typescript copied to clipboard

What was changed

This PR introduces a sample showcasing the integration of Temporal with AWS SQS, specifically focusing on FIFO queues.

Why?

I wrote an article about this here: https://michaelangelo.io/blog/temporal-sqs

Using Temporal IO alongside an AWS FIFO queue (or AWS queue in general) can be beneficial for several reasons:

• Decoupling of Services: In environments where two services cannot communicate over the same local network due to being in different clusters or policy constraints, a queue acts as a mediator. AWS SQS can store messages until they're consumed, ensuring that messages aren't lost even if the consuming service isn't immediately available.

• Integration with Existing Architecture: For organizations already invested in AWS and using SQS queues, integrating Temporal can enhance the processing capabilities without a complete overhaul. Temporal can be introduced to handle the business logic, retries, and workflows, while SQS continues to act as the message broker.

• Ordered Processing: FIFO queues ensure that messages are processed in the order they are sent, which is crucial in scenarios like financial transactions or data synchronization.

Also, customers will be happy to see it's possible to integrate temporal into their existing stack 😉

Checklist

1. Closes #296

2. How was this tested: Follow the steps for spinning up the stack, and everything should work smoothly (should you provide the correct AWS inputs)!

I can write some unit tests; however, I doubt this would be beneficial as this is simply a sample.

3. Any docs updates needed? Included a README as part of this PR.

---

I'm open to any feedback! Just let me know! 👍

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

:white_check_mark: michaelangeloio
:x: michaelangrivera

---

michaelangrivera seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

Thank you very much for this sample project. I'll review next week, but in the mean time, that looks like a very interesting example.

My only concern at this point is either we really want to maintain this as part of this samples repo. We've been recently discussing about restructuring the repo so that it is both easier to maintain and better fulfill its purpose. But let me check.

@michaelangeloio amazing example, thanks! Been wanting to implement something similar lately.

What are your thoughts on using worker_threads instead of cluster?

@michaelangeloio amazing example, thanks! Been wanting to implement something similar lately.

What are your thoughts on using worker_threads instead of cluster?

Can you provide some context? You can theoretically use worker_threads, however, drawbacks include having to pass a file (that's compiled from TS to js) or string into the worker thread.

I'm about to port an existing SQS worker to our already existing Temporal infrastructure. I had a couple strategies in my mind:

1. fire "poller workflows", that would basically poll the queue using a long-running activity, process messages, and call continueAsNew periodically. Pros: I only use a single worker. Cons: I have workflows that run forever, which are a bit harder to update than normal.
2. have my current SQS worker read messages, and fire a workflow for the message. The workflow then deletes the message once it's done processing. Pros: easier to implement. Cons: two separate services instead of one.
3. your approach, which is kind of a mashup of the previous two. But my first instinct was to open one thread per poller, instead of a process. However, you have a point, cluster might be a good fit here.