Iceberg Insert Occasionally Fails with Failed to write json to file TrinoException on ADLS (abfs://) – Retry Mitigates

[DarsheetP]

Hello, I’m using Trino (version 475) with the Iceberg connector writing to Azure Data Lake Storage (ABFS). Most inserts work, but sometimes I encounter the following error:

io.trino.spi.TrinoException: Failed to commit the transaction during insert: Failed to write json to file: abfs://@.dfs.core.windows.net//metadata/.metadata.json at io.trino.plugin.iceberg.IcebergMetadata.commitTransaction(IcebergMetadata.java:2079)

Retrying the query usually allows it to succeed. From my research, it seems that enabling or tuning the retry configuration can help reduce the frequency of this error, which I believe is related to concurrent writes and optimistic concurrency control in Iceberg.

Questions: Is this a known concurrency/control issue with Iceberg on ADLS?

Are there recommended retry settings (attempts, delay, backoff, jitter) to use with Trino Iceberg writes to minimize these commit failures?

Are there any best practices for concurrent writes or other mitigations that you recommend for this scenario?

Thank you for your help!