kitsune icon indicating copy to clipboard operation
kitsune copied to clipboard

Remove Redis dependence

Open aumetra opened this issue 11 months ago • 2 comments

Is your feature request related to a problem? Please describe. Kitsune right now is dependent on Redis even though it really doesn't need to be. All of its functionality can be replaced by PostgreSQL with maybe 1 or 2 extensions (where pg-trunks is an excellent way to install them)

Describe the solution you'd like Add a mode for the job scheduler where it can run without Redis and instead with something like a combination of pgmq and pg_cron. Not sure about the choice of plugins yet, it's subject to bikeshedding.

aumetra avatar Mar 15 '24 19:03 aumetra

Used similar solution in my previous job, so I can implement similar solution without extension (my previous implementation was implemented purely using PL/PGSQL, which does not require any extension)

However, I'm a little bit worried about performance, because PostgreSQL is not designed to be used like Redis (so some benchmarking should be done I guess)

perillamint avatar Apr 02 '24 07:04 perillamint

Yeah, exactly. That performance concern was the original reason why I removed the old queue in the first place.

The idea right now is to implement it as a secondary mode.

Leave the redis functionality as-is and add a second database mode, so that you don't have to use redis, but can if you want higher performance.

aumetra avatar Apr 02 '24 07:04 aumetra