iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Add API changes for statistics information in table metadata

Open findepi opened this issue 3 years ago • 8 comments

API for https://github.com/apache/iceberg/pull/4945 Implementation PR: https://github.com/apache/iceberg/pull/4741

Extracted from https://github.com/apache/iceberg/pull/4741 per https://github.com/apache/iceberg/pull/4741#discussion_r895213091

findepi avatar Jun 13 '22 08:06 findepi

@rdblue thank you for your review. Updated accordingly.

findepi avatar Jun 14 '22 08:06 findepi

Currently, based on https://github.com/apache/iceberg/pull/4978

findepi avatar Jun 20 '22 07:06 findepi

@rdblue updated this to have one stats file for a table.

findepi avatar Jun 22 '22 19:06 findepi

Per offline ask from @rdblue , I extracted prep change here: https://github.com/apache/iceberg/pull/5129

findepi avatar Jun 24 '22 10:06 findepi

Updated after merge of https://github.com/apache/iceberg/pull/4945.

@rdblue can you please take a look?

findepi avatar Aug 01 '22 15:08 findepi

@rdblue thank you for a careful review. Comments applied, the code is much better now. The potentially immature changes to TableMetadata are moved to https://github.com/apache/iceberg/pull/5450

Can you please take another look?

findepi avatar Aug 05 '22 18:08 findepi

Updated to apply https://github.com/apache/iceberg/pull/5450#discussion_r939727956.

findepi avatar Aug 08 '22 09:08 findepi

Build failure

> Could not resolve all files for configuration ':classpath'.
   > Could not resolve com.palantir.baseline:gradle-baseline-java:4.0.0.
     Required by:
         project :
      > Could not resolve com.palantir.baseline:gradle-baseline-java:4.0.0.
         > Could not get resource 'https://plugins.gradle.org/m2/com/palantir/baseline/gradle-baseline-java/4.0.0/gradle-baseline-java-4.0.0.pom'.
            > Could not GET 'https://plugins.gradle.org/m2/com/palantir/baseline/gradle-baseline-java/4.0.0/gradle-baseline-java-4.0.0.pom'.
               > The server may not support the client's requested TLS protocol versions: (TLSv1.2, TLSv1.3). You may need to configure the client to allow other protocols to be used. See: https://docs.gradle.org/7.4.2/userguide/build_environment.html#gradle_system_properties
                  > Received fatal alert: handshake_failure

looks unrelated.

Let me push amended commit to restart the build.

findepi avatar Aug 09 '22 14:08 findepi

Currently, depends on https://github.com/apache/iceberg/pull/5450

findepi avatar Aug 23 '22 12:08 findepi

Currently, depends on #5450

rebased after that one is merged.

@rdblue please take a look

findepi avatar Aug 25 '22 11:08 findepi

@findepi, this is looking close. I think we just need to finalize the UpdateStatistics methods. Do you have a PR for the UpdateStatistics implementation? We should be able to work on that in parallel.

rdblue avatar Aug 26 '22 17:08 rdblue

I have applied the requested changes. @rdblue please take another look

findepi avatar Aug 27 '22 21:08 findepi

Thank you for the merge!

findepi avatar Aug 30 '22 17:08 findepi