Retry-after for 429?

[tomerharduf-vayyar]

I am running a script that uploads a lot of files in parallel and I get this Exception: "(None, None, '429 Client Error: for url blahblah.blah)". Now I understand this Exception arises when too mant requests are sent to the server. I also understand that usually the server sends a "retry-after" back so the client would know how long to wait before sending new requests. Do you support it? If so, how do I do this?

[jneuendorf]

As I understand the API, you have to use .execute_query_retry instead of execute_query.

[sant3e]

I also have this issue. I've incorporated my script into a Pentaho (kettle) automatic job, that runs the script 2 times daily. In a given week 80% of the time i get this error, even though i did:

• put execute_query_retry(timeout_secs=5)
• put a retry transformation inside the Spoon Job to rerun the script 4 times after 5 seconds from failure

My use case is pulling data from a SP List with about 3k records

[hmoffatt]

There's an execute_query_with_incremental_retry() method in client_context.py which looks like it would handle the 429s

To try it I replaced

target_folder.files.create_upload_session(local_path, size_chunk, bar.update).execute_query()

with

target_folder.files.create_upload_session(local_path, size_chunk, bar.update).context.execute_query_with_incremental_retry()

My uploads seem to fail with error 503 though, not 429.