Russell Spitzer

We have seen scala/scala#8289 as well in some of our internal builds (as well as those using Spark)

No, at the moment that isn't possible and I don't think we would want to add this capability. I think if you have records you are sure can be removed...

Ah I misread you are looking for such an option. That we could do, I think it wouldn't be that hard to put a try catch in add-files that is...

Yes, Take a look at the Tasks API and you should see we have the option to put a flag on "Suppress Exceptions", I think we also have a "handleException"...

I use IntelliJ idea and the iceberg project works for me out of the box. The only additional config is for check style which has config notes in our docs...

I'm a little worried about the instance in which every file was written with the correct sort order BUT were written by independent writes. In this case I have dozens...

I want to say i'm not against heuristic optimizations, but I do think we need to make them optional, explicitly enabled/disabled, and they should do the right thing at least...

We probably will want to do some perf testing around this change since this is adding a try catch into a pretty tight loop.

Mostly I was worried about the Try/Catch block in the very inner loop of the conversion here. Sorry I've been so long in responding, I do think that for fixing...

Do we have a SPARKC ticket for this yet? I couldn't find one in jira? https://datastax-oss.atlassian.net/issues/