Ryan Blue

@ConeyLiu, thanks for breaking this up, but this is still far too large for a single PR. There's no need to update all of the writers in a single PR....

> Is this something we can do or need to wait until V3? @szehon-ho, this is something we can do because it is backward compatible. Older readers will ignore new...

@szehon-ho, the reason why we use `all_manifests` is to avoid the expensive driver-side manifest list file reads. `all_files` has to read through every manifest list in the table from the...

@hililiwei, can you open separate PRs to backport those? I think this is too much for one PR.

Thanks for pinging me, @singhpk234. Looks like we need to convert the properties map over to a HashMap rather than an immutable map. Good catch!

I think we should be very careful about introducing new methods to the `Catalog` API. There needs to be a strong justification for adding to it, and I don't think...

> How about reuse registerTable procedure? If you mean to add more arguments to `registerTable` so that it is an atomic operation, I don't really see much point to it....

@szehon-ho, what I originally meant was that it would be fine for the caller to manually call `dropTable` first. I'm hesitant to add a `force` option because it's basically the...

@RussellSpitzer, I see what you mean. What about fixing the Hive catalog so that the commit succeeds if table metadata can't be loaded? It sounds like we should have better...

@szehon-ho, in part, yes. I think that if we have situations where a table is corrupted then we should handle those in a better way than just having a table...