Eduard Tudenhoefner

@jbonofre FYI

The PR is already quite large and given that we need to fix those things across all Flink/Spark versions I'd suggest to extract the Flink/Spark related things into a separate...

should be good to go once https://github.com/apache/iceberg/pull/10583/files#r1663651011 has been addressed

> > should be good to go once https://github.com/apache/iceberg/pull/10583/files#r1663651011 has been addressed > > Sorry, it's really out of scope of this PR ("Adress IntelliJ Inspection findings"). We're improving the...

`Premature end of Content-Length delimited message body (expected: 297,221,738; received: 143,086,296)` is likely indicating that either the client or the server prematurely closed the connection before retrieving/sending all the data....

@paulpaul1076 do you have a chance to try with `http-client.type=urlconnection`? It's of course also possible that there's a bug in `RewriteDataFilesProcedure` that went unnoticed. If you have an easy way...

> Looks like iceberg-aws-bundle doesn't have this class: > > `Exception in thread "main" java.lang.NoClassDefFoundError: software/amazon/awssdk/http/urlconnection/UrlConnectionHttpClient$Builder` > > Should I just add the awsask to classpath also? I thought that...

> Anyways, got it to work, now there's a similar exception, but written a bit different: > > ``` > org.apache.iceberg.exceptions.RuntimeIOException: java.io.EOFException: Reached the end of stream with 8009878 bytes...

Thanks @paulpaul1076, I will try and reproduce this next week on my end

@paulpaul1076 I don't have an Airflow setup but I ran a streaming job locally and created 4000+ files. The specific setup I used was from the [Spark quickstart example](https://iceberg.apache.org/spark-quickstart/) with...