fluent-plugin-redis icon indicating copy to clipboard operation
fluent-plugin-redis copied to clipboard

Add support for LIST and CHANNEL data types

Open iridian-ks opened this issue 4 years ago • 1 comments

Problem

We have existing log stacks that use LIST or CHANNEL data types and does not support HASH_MAP.

Steps to replicate

Use a log consumer that only supports LIST or CHANNEL data types.

Expected Behavior

The fluent-plugin-redis configuration allows to specify which data type to use. Currently, only HASH MAP is supported.

Suggested changes

  1. Add a new config option in the plugin called data_type. Default this to HASH_MAP. Allow for options HASH_MAP, LIST, CHANNEL.
  2. HASH_MAP type works as it does today with no changes
  3. LIST option will use RPUSH
  4. CHANNEL option will use PUBLISH
  5. Document that allow_duplicate_key and possibly ttl is only valid for HASH_MAP
  6. Document that key_prefix can be used to select a different LIST or CHANNEL

I am prepared to make the updates to the plugin but want a confirmation that such changes will be accepted in a PR.

Thank you!! :)

iridian-ks avatar Jun 18 '21 18:06 iridian-ks

@iridian-ks FWIW, I implemented Pub/Sub support in #33. Please comment to that pull request, if you have any feedback.

fujimotos avatar Sep 29 '21 08:09 fujimotos