nats-server Distributed atomic counter support in JetStream KV

trafficstars

Feature Request

Add Distributed Atomic Counter support to JetStream KV.

Use Case:

Slack thread

Proposed Change:

add supporting code to kv
add command-line support
- a new flag for kv add subcommand to specify whether the stream backing a bucket should be a file stream or a memory stream, e.g.:
  - nats kv add my_bucket --replicas 1 --history 5 --STORAGE=FILE
  - nats kv add my_bucket --replicas 1 --history 5 --STORAGE=MEMORY
- two new subcommands:
  - incrby x, e.g.: nats kv INCRBY 20 my_bucket my_key_1
  - decrby x, e.g.: nats kv DECRBY 10 my_bucket my_key_1
allow interacting with the counters programmatically
- 1st phase: via publishing messages to dedicated subjects; this would allow the new feature to be used before the official clients catch up.
- 2nd phase: via Synadia official JetStream client(s).

Who Benefits From The Change(s)?

Synadia - by closing the feature gap between JetStream and more popular solutions.
Synadia - by making clients and users happy. Who doesn't need or use counters in production?
JetSstream users - who can say no to happiness ?

Alternative Approaches

The competition
- Apache Kafka
- Apache Pulsar
Other software
- Redis
- RDBMS
- etc

Oct 30 '21 19:10 FalxDraco

The Slack thread describing the use case has been archived, but I hope that the use cases are still easy enough to imagine.

I am currently doing this this by publishing +1s and -1s to a stream, then consuming that stream with another client that eventually puts the accumulated results into a kv store. It would be nice to skip that step.