hadoop-connectors
hadoop-connectors copied to clipboard
Hadoop FileSystem delete method returns false even though file is actually deleted
In my Java application I have an implementation for a file-system, where my file class is a wrapper for Hadoop filesystem methods. I am upgrading the from hadoop3-1.9.17 to hadoop3-2.2.8 and I am using the shaded jar of the new version.
My delete method implementation is as follows:
@Override
public boolean delete(boolean recursively) {
return fs.delete(path, recursively);
}
Where fs is an instance of org.apache.hadoop.fs.FileSystem
so this method calls org.apache.hadoop.fs.FileSystem.delete(Path, boolean)
Frequently when calling the above delete method on a directory which contains files it returns false 'meaning dir/file NOT deleted' however when checking the directory is actually deleted at Google Cloud Storage.
Is there a change in the upgraded GCS connector causing this (caching of file-status i.e) or is there a configuration to fix this issue? I also created this [stackoverflow question](https://stackoverflow.com/questions/74445091/gcs-connector-hadoop3-hadoop3-2-2-8-filesystem-delete-returns-false-but-file-dir but got no answers