nats-server icon indicating copy to clipboard operation
nats-server copied to clipboard

Support for Key Search with Wildcards in NATS KV Methods

Open burl21 opened this issue 1 year ago • 6 comments

Proposed change

I would like to request support for wildcard searches in the following NATS KV methods: ls and rm. This feature would enhance the flexibility and usability of NATS KV by allowing users to list and delete multiple keys that match a certain pattern.

Use case

In scenarios where multiple keys follow a naming pattern, such as cache entries, logs, or configuration settings, being able to fetch or remove them all with a single wildcard query would save time and reduce code complexity. This feature is particularly useful in large-scale applications where the number of keys can be substantial.

Consider a bucket named test with the following entries:

•	test.cache1
•	test.cache2

Using the ls and rm methods with a wildcard, such as nats kv ls test "test.*" or nats kv rm test "test.*", should list and delete both test.cache1 and test.cache2, respectively. This functionality is crucial for efficiently managing related key-value pairs without having to handle each key individually.

Contribution

No response

burl21 avatar Jun 15 '24 06:06 burl21

The CLI ls is for buckets. However in the Go client, ListKeys can take a filter subject. Delete has no DeleteMulti at this point in time, but might be a good addition.

Not sure this is a server issues though. /cc @ripienaar and @piotrpio and @Jarema

derekcollison avatar Jun 18 '24 03:06 derekcollison

nats kv ls BUCKET for keys, I will add a subject filter.

ripienaar avatar Jun 18 '24 06:06 ripienaar

Actually, I do not see how ListKeys can take a filter - internally it always use WatchAll. We certainly could support that though.

ripienaar avatar Jun 18 '24 07:06 ripienaar

Since delete is a put in reality, I am reluctant to do a range as we cannot do that in an atomic manner today

ripienaar avatar Jun 18 '24 08:06 ripienaar

Here is the ADR describing the new API in the clients: https://github.com/nats-io/nats-architecture-and-design/pull/288

Clients will soon follow up with implementing them. Here is the reference issue in Go client: https://github.com/nats-io/nats.go/issues/1655

This does not include delete with wildcards, as that is a more complex story that requires server changes.

Jarema avatar Jun 20 '24 04:06 Jarema

I hade an idea to virtual-bucket that is related to this issue https://github.com/nats-io/nats-server/issues/5204#issuecomment-2081554896

Zetanova avatar Jul 04 '24 10:07 Zetanova