iceberg
iceberg copied to clipboard
apache
Apache Iceberg
### Query engine iceberg-flink-runtime-1.14-0.14.0.jar，help me，Thanks ### Question org.apache.iceberg.exceptions.ValidationException: Cannot commit, found new delete for replaced data file: GenericDataFile{content=data, file_path=hdfs://dev-001:8020/iceberg/flink_hive_iceberg/flink_hive_db.db/test_repository_1/data/news_postdate=2022-07-31/00002-0-8b3590ea-a593-4734-b84a-a6084a426b95-00093.parquet, file_format= PARQUET, spec_id=0, partition=PartitionData{news_postdate=2022-07-31}, record_count=106, file_size_in_bytes=110049, column_sizes=null, value_counts=null, null_value_co unts=null, nan_value_counts=null,...
When the metadata file doesn't exist any more, drop table operation will fail. This PR aims to support deleting the table even if the table was corrupted.
In `SnapshotUtil`, when a snapshot does not have a schema id (written before schema id was added to snapshots), we fall back to reading each of the previous metadata files...
fix https://github.com/apache/iceberg/issues/5163 Add 'force' option to register existing table, this is optional usage: `CALL mycatalogname.system.register_table('mycatalogname.mydb.mytablename','xx://xxxxx/metadata/xx.metadata.json','force')`
Adding support for Initializing S3 client during the S3FileIO initialization. This is needed to deal with a scenario in which S3 files need to be deleted after table has been...
By default, `ExpireSnapshotsSparkAction` cleans the expired files but the log says not cleaning the files. `22/08/09 11:03:43 INFO RemoveSnapshots: Cleaning up manifest and data files disabled, leaving them in place`...
This change updates the implementation of dropping table data and metadata to use bulk delete if applicable.
Most of the time using a non-generic Counter API with a long value is simpler, especially when it comes down to serializing/deserializing Counter results (since we'd need to know the...