No per Rpc call timeout configuration available currently

[mukund-ananthu]

1. We observe that the timeout field here corresponds to timeout that spans across multiple retry attempts and not the timeout associated with individual retry attempts.
2. This was observed for the timeout set for our repository, where we see that multiple retries occur in the 60s duration, after which the retries immediately get a DEADLINE EXCEED for each retry thereafter.
3. This appears to be the case because the timeout field mentioned above is used as the timeout of the wrapped function here
4. We would like to be able to configure per Rpc timeout in addition to the timeout for the wrapped function.
5. Could you please let us know if there is an existing field that we could use for this or if a new field can be added to allow us to configure this.

[mukund-ananthu]

cc: @parthea

[ohmayr]

@mukund-ananthu Currently, the libraries only support configuring the timeout for the entire operation.

Configuring per-RPC timeout seems like a feature request. Can you provide a reason / use-case for why this is needed?

[mukund-ananthu]

1. There are fine grained settings for retries in other languages, for example that allow setting per Rpc timeout. For consistency, we would require ability to configure retries similarly in Python. In general, I think given retry settings apply across languages, the configurability of Retries should be consistent across the different languages
2. Given that deadline already provides timeouts configured across retries, I do not understand what value add default_timeout provides if it also provides timeouts across retries.

[mukund-ananthu]

cc: @kamalaboulhosn

[kamalaboulhosn]

As @mukund-ananthu mentions we do have this in other languages and so in order to provide a consistent experience to customers, it seems like we should have this feature in the GAPIC-generated code for each language. For Pub/Sub's case specifically, users want messages to get published as quickly as possible and so if one request takes too long, we want to initiate a second request. This helps if the time it's taking to publish is due to a stuck server or something where retrying may hit a different server and be able to succeed.