featurebase icon indicating copy to clipboard operation
featurebase copied to clipboard

import endpoints should always forward to all replicas

Open jaffee opened this issue 5 years ago • 0 comments

...we have some that do, and some that don't, this is bad.

Pilosa is the only entity which actually knows at any given point which nodes own a shard, if the client is relying on old info about this, some of it's requests might fail, and some succeed... then it has to re-fetch the shard/node info and try again.

If the client is responsible for sending imports to all replicas, Pilosa can't elegantly manage forwarding a request which came to an incorrect node. It doesn't know if the client is already sending requests to a subset of the replicas for that node.

If Pilosa is being contacted through a proxy, then Pilosa has to do the forwarding, unless the proxy can intelligently forward them or something, but this sounds like a nightmare.

So overall it seems that life would be a lot simpler if Pilosa handled all replication internally. I know that the import-values endpoint needs to change, but that may be it.

jaffee avatar Aug 27 '19 21:08 jaffee