streamify icon indicating copy to clipboard operation
streamify copied to clipboard

Published 20 hours ago • verified publisher icon ankurchavda

Do we need two separate VMs for this project?

Open stephenllh opened this issue 2 years ago • 3 comments

From the setup.md, it seems that you have two VMs set up. Just curious, is it necessary?

stephenllh avatar Apr 12 '22 11:04 stephenllh

Hey @stephenllh, It is not a necessity. But if you plan to send a good volume of data from Eventsim, it is going to take up a lot of memory. I had two Eventsim programs writing to Kafka on the VM and I had only around 300MB of memory left out of 16GB. So adding Airflow into that would be a nightmare. I'd say keep a separate VM for Airflow but choose a smaller one.

ankurchavda avatar Apr 12 '22 12:04 ankurchavda

I see. Thanks for telling me. I am actually using this repo as some kind of tutorial for my own learning. You really did a good job, especially the documentation.

stephenllh avatar Apr 12 '22 13:04 stephenllh

Thank you! I hope you enjoy setting up this project and building over it :D

ankurchavda avatar Apr 12 '22 13:04 ankurchavda