Heroku already has built-in load balancing

[mars]

From the Heroku documentation, HTTP Routing:

So scaling an app’s capacity to handle web traffic involves scaling the number of web dynos:

$ heroku ps:scale web+5

A random selection algorithm is used for HTTP request load balancing across web dynos

Stated another way: when you scale up a Heroku app to multiple dynos (server instances), the platform automatically routes incoming requests using a randomized load-balancing algorithm.

So, what problem does this project solve? It seems like this would make your Heroku app more complex & brittle, harder to maintain, and more difficult to onboard new developers.

[dmytrostriletskyi]

@mars, hello!

Consider the next situation. On the screenshot, I have the instance with 4 dynos for $100 and it solves my problem. But I could solve it in the same way with 4 single instances and one load balancer (all of them are on the Hobby plan) and it will cost me $35 ($7x4 and $7).

Am I wrong that with separated load balancer as the application I will pay ~3 time less?

Also, I have had the situation in the production, when I needed to use Heroku (very fast continuous delivery platform for small projects), but the project I have hosted was CPU-bounded. I knew that Heroku does not fit me in this case.

From the Heroku documentation, Optimizing dyno usage in case of CPU:

If you are processing individual requests slowly due to CPU or other shared resource 
constraints (such as database), then optimizing concurrency on the dyno may not 
help your application’s throughput at all.

...

I tried to choose another plan even from Performance, but it hasn't increased my performance too much to pay a few hundred dollars for this. So I just created tens of the instances with my software and put a load balancer before them. It was my fast and simple solution which worked for me even cheaper if Performance made the same performance.

[mars]

Sounds like you found a clever solution to your problem. The README makes it sound like Heroku has no load balancing, which is quite false.

Keep in mind that this solution requires multiple, technically independent apps. The apps do not behave as a single app:

1. any add-ons must be manually attached to each app (makes operations more complex)
2. all logging is spread across apps (makes debugging harder)
3. performance metrics are spread across the apps (makes understanding app behavior harder)
4. the Heroku platform does not operate them as a single app (could cause downtime during deployments or daily dyno cycling)
5. when the single load balancer [Hobby] dyno cycles (restarts) each day or on deployment, the entire app will go offline temporarily
6. added request latency (another two HTTP hops in front of the Heroku router)

If you understand these caveats then this project might be a nice trick to save cash!

[dmytrostriletskyi]

@mars, thank you for the information, great points about Heroku architecture to never forget. I will remove impudent notes that Heroku does not have the load balancer in the readme. Instead, will clarify pros and cons of my solution, and leave the link to this issue.

Great conversation, I have enjoyed it.