gapic-generator-python

gapic-generator-python copied to clipboard

GoogleAdsService provides search(), which accepts a Retry(). Apply the Retry decorator on all request pages, not just the first.

Original-Report: https://github.com/googleads/google-ads-python/issues/597 Resolves: https://github.com/googleapis/gapic-generator-python/issues/1242

@parthea I've rebased. Running tests locally (took a long time!):

I'd have some trouble fixing these tests as I'm unfamiliar with the development workflow (e.g. working with templated sourcecode, how to run individual tests), and I notice there are failures of mocked expectations of internal call patterns. (Probably due to the refactor in the second commit.)

Can you help/advise? Would you prefer I omit the refactor?

@parthea Any updates on this, please? It's been quite some time (and I've already had to rebase around an upstream refactor, once before).

@BenRKarl taking you up on your offer to help coordinate.

I'm just going to give up on this, due to lack of support from Google. Luckily, this is Python, so it's possible to peer under the interface and develop a custom workaround.

If you're here reading this, takeaways so you can implement/workaround/fix yourself:

• The built-in retry & timeout mechanism for both search_stream() and search() is broken & unreliable
  • search_stream() - completely broken, likely cannot be mitigated at the Python level
    • retry only applies to "the returning of the non-materialized generator", i.e. useless
    • timeout wraps the entire stream, so you'll need to know the result size ahead of time
  • search() - only applies retry & timeouts to the first paginated request
• search() can be made resilient by applying retry and timeout to every request, not just the first