Retry strategy for gRPC clients

[guilload]

Exponential backoff + jitter + round robin on server pool

[fmassot]

All endpoints are not idempotent, we need to work on that too.

[fmassot]

Ideally I would see the retry at the tonic level. We need also to take into account that some gRPC clients use a tower balance service.

[fulmicoton]

@guilload Can you complete this ticket

[guilload]

The retry work in tower is not going to land anytime soon, so we need a workaround solution in the meantime. Let's implement a RetryingMetastore that implements the usual exponential backoff with jitter retry logic. Let's replicate or reuse what we have in quickwit-aws.

[guilload]

Closed via #2335.