Nessie S3 remote signing endpoint not refreshed

[lwfitzgerald]

Apache Iceberg version

main (development)

Please describe the bug 🐞

When accessing Iceberg tables provided by an instance of the Nessie catalog, if remote S3 signing is enabled on Nessie, signing requests use a stale endpoint after 3 hours, causing S3 requests to fail.

Nessie supports S3 remote signing by setting a custom S3 signer endpoint via s3.signer.endpoint in the table configs it returns. As part of the s3.signer.endpoint it includes a HMAC signature with an expiry of 3 hours.

pyiceberg does not appear to refresh the s3.signer.endpoint (from the table config) after the first interaction with a table, meaning that after 3 hours signing requests to Nessie start failing.

This issue also exists in the Iceberg implementation for Spark - https://github.com/apache/iceberg/issues/12602.

Willingness to contribute

• [ ] I can contribute a fix for this bug independently
• [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• [x] I cannot contribute a fix for this bug at this time

[Fokko]

Thanks for raising this @lwfitzgerald. When we refresh the token, we could also recreate a new FileIO, which does not look entirely unreasonable. Let's hear what comes out of apache/iceberg#12602

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions[bot]]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'