hammer icon indicating copy to clipboard operation
hammer copied to clipboard

A bit more explanation on why ETS is bad in production?

Open ryanwinchester opened this issue 4 years ago • 1 comments

Hammer.Backend.ETS (provided with Hammer for testing and dev purposes, not very good for production use)

Sorry, I was just wondering what are the reasons why ETS is "not very good" for production use?

In your docs you also say:

There may come a time when ETS just doesn’t cut it, for example if we end up load-balancing across many nodes and want to keep our rate-limiter state in one central store.

That suggests it may be fine in production, depending on the circumstances.

I think it would be useful for users to know the reasons it is not recommended in the README, so they can weigh their decisions of if they should start with ETS and switch to Redis later, and help decide when they should plan on making the switch... or just not consider it at all.

Thanks so much :)

ryanwinchester avatar May 22 '20 08:05 ryanwinchester

Okay, I see this PR https://github.com/ExHammer/hammer/pull/41.

Is it just because it has been single-threaded because the calls were behind GenServers?

ryanwinchester avatar May 22 '20 08:05 ryanwinchester

:wave: here for an update on that one too, after spotting the README note.

Is there any concern still since the ets calls are made directly?

aerosol avatar Apr 06 '23 09:04 aerosol

+1 - it is very convenient to not add an extra service like Redis and just rely on ETS - so it'd be great to know what the production concerns are. Maybe @epinault / @robmadole , you would know per your involvement in #41 ? Thank you very much :)

selimyoussry avatar Jul 08 '23 08:07 selimyoussry

I don t think there are as many concerns. the original concerns is that having a gen server in front of an ETS table creates a singleton (therefore performance botttleneck). I believe I have it fixed in latest version where the write and read are straight to the ETS but the creation and management is via the GenServer so that removed a bit concern

The other issue is that if you have multiple pods/VM. Those ETS table will not know about each others .. So you are only throttling at the pod level unlike Redis where it would be global to your application. But you can lower the throttling per pod to achieve similar things

epinault avatar Jul 08 '23 16:07 epinault