HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.

[ArafatKhan2198]

What changes were proposed in this pull request?

warning messages should be removed from console logs. We can do this by suppressing the respective messages in the log4j.properties file.

for i in $(seq 1 6)

do /opt/cloudera/parcels/CDH/bin/ozone fs -copyFromLocal /tmp/5GB ofs://ozone1/vol2/buck1/key$i done

22/02/14 12:17:23 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
22/02/14 12:17:24 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
22/02/14 12:17:24 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system started
22/02/14 12:17:24 INFO metrics.MetricRegistries: Loaded MetricRegistries class org.apache.ratis.metrics.impl.MetricRegistriesImpl
22/02/14 12:20:28 WARN io.KeyOutputStream: Encountered exception java.io.IOException: Unexpected Storage Container Exception: java.util.concurrent.CompletionException: Failed to write chunk 109611004723200001_chunk_1 into block conID: 1 locID: 109611004723200001 bcsId: 0 on the pipeline Pipeline[ Id: c665901b-87bf-4943-8ec6-da745db56901, Nodes: c221dcbd-398c-4e76-ab13-dcde074041ae{ip: 172.27.128.2, host: quasar-usholy-5.quasar-usholy.root.hwx.site, ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}c9da661f-99ae-4de3-8eb8-306763283723{ip: 172.27.126.198, host: quasar-usholy-6.quasar-usholy.root.hwx.site, ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}69a59a13-75d6-4cde-8cb3-5713ae9de7cd{ip: 172.27.26.1, host: quasar-usholy-2.quasar-usholy.root.hwx.site, ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}, ReplicationConfig: RATIS/THREE, State:OPEN, leaderId:c9da661f-99ae-4de3-8eb8-306763283723, CreationTimestamp2022-02-14T05:43:11.228Z[UTC]]. The last committed block length is 0, uncommitted data length is 33554432 retry count 0

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-7104

How was this patch tested?

Tested it out on the local cluster, the messages will get suppressed

[ArafatKhan2198]

@adoroszlai @ChenSammi @siddhantsangwan can you help review this !!

[ChenSammi]

Thanks @ArafatKhan2198 , can you tried other fs/CLI commands, to see if their WARNING messages are resolved by this PR?

[ArafatKhan2198]

Thanks @ArafatKhan2198 , can you tried other fs/CLI commands, to see if their WARNING messages are resolved by this PR?

I have tested out most of the FileSystem commands out there and none of them give out the suppressed warnings :-

1. Creating Volumes
2. Creating FSO and OBS buckets
3. Creating directories inside buckets
4. Listing the buckets
5. Creating keys inside buckets
6. Move Commands on keys
7. Deleting a key
8. Deleting a Bucket
9. Deleting a Volume

[ArafatKhan2198]

@ArafatKhan2198 ozone-shell-log4j.properties does not need to be changed. I think logging to a file is enough to avoid annoying users with useless warnings.

However, we need to configure ozone fs command to use ozone-shell-log4j.properties:

https://github.com/apache/ozone/blob/5f84fd1c9f35d82f2558b9c05dbe9c4d0b252900/hadoop-ozone/dist/src/shell/ozone/ozone#L188-L191

similar to how it's done for ozone sh:

https://github.com/apache/ozone/blob/5f84fd1c9f35d82f2558b9c05dbe9c4d0b252900/hadoop-ozone/dist/src/shell/ozone/ozone#L149-L154

See also #3706 for an improvement being done for ozone sh to allow custom log filenames.

Yes I am testing it out now, Ill be updating the patch soon with the changes