kivik icon indicating copy to clipboard operation
kivik copied to clipboard

Is there a way to use _shards endpoint?

Open Noksa opened this issue 3 years ago • 6 comments

How can I retrieve information about databases' shards using this https://docs.couchdb.org/en/stable/api/database/shard.html#api-db-shards ?

Thanks!

Noksa avatar Dec 01 '21 12:12 Noksa

There is not at the moment. It may be possible to add the functionality, but I'm curious how you would be using this, to help gauge the urgency.

flimzy avatar Dec 01 '21 21:12 flimzy

@flimzy Thanks for the quick answer!

I'm developing an internal k8s operator that is responsible for a couchdb's lifecycle. So when a new node has come to a k8s cluster we should determine which shard should be moved to this node and so on.

Currently I retrieve info about all shards from internal _dbs/{dbname} db where we can find byrange bynode and so on info about shards that are related to a specific db. So to retrieve all shards I have to go through all _dbs for each db.

It would be more convenient to list all shards using only one request.

I'm using couchdb 2.3.1 if it is important (we can't move to 3 right now).

Currently I already started to use a workaround - I just call curl inside a couchdb pod and then do unmarshal to a struct.

So it is not required for me anymore, but would be cool if it was built-in func from kivik.

Noksa avatar Dec 02 '21 07:12 Noksa

That sounds like a nice project!

The main reason I haven't included support for _shards and other "system"-ish endpoints (there are quite a lot of them) is that it feels like it's not part of the core functionality that a normal client would use. They also tend to change a lot more frequently than the core API. I didn't really want to pollute the library's API with all of those things that are rarely (if ever) used.

However, for utility libraries, like what you're describing, as well as kouch, access to some of these system endpoints could make sense.

One compromise could be to use chttp, which serves as a backend to the CouchDB driver component of Kivik, and provides convenience functions for producing and handling CouchDB requests and responses respectively. It's not really designed for public consumption, but it is available.

It might make sense to expose some of that through the kivik client interface directly, so that a client consumer can do arbitrary queries against a CouchDB server.

What are your thoughts about this?

flimzy avatar Dec 02 '21 17:12 flimzy

@flimzy I'll take a look at chttp, thanks for sharing this!

I think it is up to you what should be opened to a client consumer :) Because now I want one endpoint and maybe sometime ago I will need another one. So it sounds like chttp is the best thing to do arbitrary requests at the moment.

Noksa avatar Dec 02 '21 17:12 Noksa

I think the biggest disadvantage of using chttp right now is that you'll have to make two client connections, since I don't believe there's a provided way to get access to the backing chttp object used by the kivik client.

I may consider exposing that at some point. That wasn't needed for kouch, since there's only ever one connection per run of a CLI tool anyway :)

flimzy avatar Dec 02 '21 17:12 flimzy

@flimzy Anyway I just use curl and then do whatever I need with json :)

Thanks for help!

Noksa avatar Dec 02 '21 18:12 Noksa