hadoop-connectors icon indicating copy to clipboard operation
hadoop-connectors copied to clipboard

GoogleHadoopFileSystem.getIOStatistics() logs NPE "statistic key for HTTP_DELETE_REQUEST must not be null"

Open steveloughran opened this issue 3 years ago • 0 comments

while writing some tests which collect FS stats in teardown, my logs are full of stack traces.

Jan 21, 2022 2:56:17 PM com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem setHttpStatistics
WARNING: Error while getting GCS statistics
java.lang.NullPointerException: statistic key for HTTP_DELETE_REQUEST must not be null
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:994)
        at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem.lambda$setHttpStatistics$18(GoogleHadoopFileSystem.java:1062)
        at com.google.common.collect.RegularImmutableMap.forEach(RegularImmutableMap.java:196)
        at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem.setHttpStatistics(GoogleHadoopFileSystem.java:1059)
        at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem.getIOStatistics(GoogleHadoopFileSystem.java:1045)
        at org.apache.hadoop.fs.statistics.IOStatisticsSupport.retrieveIOStatistics(IOStatisticsSupport.java:82)
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.AbstractManifestCommitterTest.teardown(AbstractManifestCommitterTest.java:344)
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.TestJobThroughManifestCommitter.teardown(TestJobThroughManifestCommitter.java:172)

this is from a build on top of commit 9f9d5f82

steveloughran avatar Jan 21 '22 15:01 steveloughran