Crawling-Infrastructure
Crawling-Infrastructure copied to clipboard
Consider Kubernetes for architecture
Dear Nikolai,
I'm a big fan of your projects. I'm also studying at the University of Hamburg and of course also have a passion for sraping. I recently tried out this puppeteer architecture on Kubernetes: https://www.openfaas.com/blog/puppeteer-scraping/
It runs smooth as butter with a multi-cloud environment :)
Of course docker swarm is similar in a way.
If you want to exchange experiences let me know!