iceberg-python icon indicating copy to clipboard operation
iceberg-python copied to clipboard

Suppress duplicate OAuth token fetching in rest catalog client

Open TennyZhuang opened this issue 1 year ago • 1 comments

Feature Request / Improvement

In the rest catalog client, we implemented the OAuth token refresh based on retry mechanism.

https://github.com/apache/iceberg-python/blob/4148edb5e28ae88024a55e0b112238e65b873957/pyiceberg/catalog/rest.py#L126-L128

If multiple calls are made to the REST client API simultaneously, they may fail at the same time, resulting in thousands of _refresh_token calls. The behavior is likely acceptable with no exceptions, but it appears wasteful and not as expected.

A potential solution is introducing a threading.Lock to protect the fetching process. Every call to _refresh_token should acquire the Lock first, and check whether the token is same as the expired token. If it's changed by another thread, we can skip the refetching and retry the original call directly.

TennyZhuang avatar Apr 06 '24 06:04 TennyZhuang

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

github-actions[bot] avatar Oct 04 '24 00:10 github-actions[bot]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'

github-actions[bot] avatar Oct 18 '24 00:10 github-actions[bot]