TorPaste icon indicating copy to clipboard operation
TorPaste copied to clipboard

[WIP] Add Redis Backend Support

Open daknob opened this issue 9 years ago • 1 comments

Due to limitations of the filesystem backend, I decided to push a redis backend which will store all pastes and metadata to a Redis instance. The current data storage scheme is:

Paste Storage

Key: <paste_id>
Value: <paste_content>

Paste Metadata Storage

Key: <paste_id>.<metadata_key>
Value: <metadata_value>

Due to the List Pastes feature, this backend has to run the Redis KEYS command, which according to the official documentation is not recommended for production. However, after checking the recommended options a little bit, this command might be necessary for the app to work. In addition to that, some benchmarks I ran against the KEYS command and the filesystem backend indicate that the KEYS command can run up to hundreds of thousands of keys (currently 1/3rd of them will be pastes) in under one second, which is much faster than the backend filesystem.

I am open of course to other options to store data using Redis and will not merge until some time, so we can evaluate alternatives. Until then, use of this backend will of course not be supported and there will be no guarantees of compatibility across versions. These guarantees only exist after merging to master.

daknob avatar Nov 13 '16 11:11 daknob

Metadata will most likely have to be reimplemented using Redis Hashes. I am currently investigating this option.

daknob avatar Nov 19 '16 11:11 daknob