yunlou11
yunlou11
```text java -jar nessie-gc-0.99.0.jar gc ``` failed: ```text Caused by: java.lang.RuntimeException: Failed to get manifest files for ICEBERG_TABLE robot_dev.robot_data, content-ID fc122060-bf21-44d3-b776-fbecb2d23715 at commit 47a35e9867b0408c65feb09d7140a29d198354edf3a1aa0dc7cc09d192b07c27 via s3://ice-lake/robot_dev/robot_data/metadata/00000-6b65b94d-6370-4c43-9baa-b40ed0770c5d.metadata.json ``` After expire snapshot...
> @yunlou11 thanks for the information. But what are all the necessary steps to get to that error message? Aka, what was all done before running GC? Sorry, It's maybe...
> > @yunlou11 thanks for the information. But what are all the necessary steps to get to that error message? Aka, what was all done before running GC? > >...
```sql CALL nessie.system.remove_orphan_files(table => 'nessie.robot_dev.robot_data') ``` ```text Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "s3" at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3443) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3466) at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521) ``` @RussellSpitzer
@rahil-c please check if its true below: listWithPrefix maybe list too much unexpected files, such as 2 tables: sample , sample_part matchingFiles will contain: ```text ... DeleteOrphanFilesSparkAction: match files: s3a://ice-lake/warehouse/ice_db/sample/metadata/snap-4835616794401450947-1-f5bb6d24-162f-4c9d-a426-893c07cac506.avro...
new content: ``` text .... [2024-08-27 10:08:28,193 I 115 115] (raylet) local_resource_manager.cc:287: Object store memory is not idle. [2024-08-27 10:08:28,297 I 115 115] (raylet) local_resource_manager.cc:287: Object store memory is not...
> I think we can make it a debug log. let me create a PR. Regarding why the logs are printed for this environment, I will follow up @rkooo567 I...
> The general recommendation is to use a tool like https://linux.die.net/man/8/logrotate. That's how we rotate logs in anyscale platform as well I think It maybe work below if raylet used...
> Can you create a test for this fixing? Okay, I'm willing to do this.