ice
ice copied to clipboard
Revisit Ice retries
We should consider revisiting Ice reties. Some use cases:
- an invocation raises UnkonwnException or ObjectNotExistException. Depending on the configuration of the proxy (is connection caching enabled?), its number of endpoints, the connections currently opened with the communicator, the Ice runtime might always retry on the same endpoint or consider other endpoints and retries are limited by the retry count (i.e.: we don't retry on all the endpoints first).
- an invocation raises TimeoutException. The Ice runtime closes the connection and always consider all the proxy endpoints for the retry (but can also possibly retry on the same endpoint as the first invocation...).
- an invocation raises ObjectNotException. The Ice runtime doesn't retry if the proxy is direct, it retries if the proxy is indirect. If it retries, it retries as mentioned above for an UnknownException.
- the Ice runtime never retries on MarshalException but retries for other kind of exceptions which aren't worth retrying (SSL exceptions)
- there's currently no way to retry on UserException.
- connection establishment has its own retry loop which can be confusing. While the Ice runtime tries all the endpoints for connection establishment, it doesn't try all the endpoints when an invocation fails with an exception that doesn't result in connection closure (UnknownException, ONE, etc).
The fact that we have different behaviors depending on the kind of exception, on the configuration and the kind of the proxy is complicated to document and understand.
I feel this could be simplified. A possibility would be the ability to tag exceptions which can be retried or not and if an exception is tagged as "retriable", we should have the same retry semantics regardless of the proxy configuration, proxy type or exception kind (e.g.: retry on all the endpoints, if failure, increment the retry count and try again on all the endpoints after configurable delay and until retry count isn't reached).
See also #166, #225 and #226