ozone
ozone copied to clipboard
HDDS-11405. Implement setrep command for Ozone FS.
What changes were proposed in this pull request?
Currently this command does not work, as on demand changing of replication is not supported in Ozone. However the new atomic rewriteKey API , makes it possible to rewrite key with the new replication and setrep can be implemented using this. This is only for RATIS keys and does not apply for EC keys as there is no point of replication factor in EC keys.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11405
How was this patch tested?
Unit tests added. Also tested the ozone fs
shell
bash-4.4$ ozone sh key info vol1/buck1/key1 | grep "replicationFactor"
"replicationFactor" : "ONE",
bash-4.4$ ozone fs -ls ofs://om/vol1/buck1/key1
-rw-rw-rw- 1 hadoop hadoop 4068 2024-09-05 08:36 ofs://om/vol1/buck1/key1
bash-4.4$ ozone fs -setrep -w 3 ofs://om/vol1/buck1/key1
Replication 3 set: ofs://om/vol1/buck1/key1
Waiting for ofs://om/vol1/buck1/key1 ... done
bash-4.4$ ozone sh key info vol1/buck1/key1 | grep "replicationFactor"
replicationFactor : THREE,
bash-4.4$ ozone fs -setrep -w 2 ofs://om/vol1/buck1/key1
setrep: Replication factor of 2 not supported
setrep with and without -w
bash-4.4$ ozone fs -setrep 3 ofs://om/s3v/buck/key2
-setrep: Asynchronous set rep is not supported,Please use -w arg
Usage: ozone fs [generic options]
[-appendToFile [-n] <localsrc> ... <dst>]
[-cat [-ignoreCrc] <src> ...]
[-checksum [-v] <src> ...]
[-chgrp [-R] GROUP PATH...]
[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
[-chown [-R] [OWNER][:[GROUP]] PATH...]
[-concat <target path> <src path> <src path> ...]
[-copyFromLocal [-f] [-p] [-l] [-d] [-t <thread count>] [-q <thread pool queue size>] <localsrc> ... <dst>]
[-copyToLocal [-f] [-p] [-crc] [-ignoreCrc] [-t <thread count>] [-q <thread pool queue size>] <src> ... <localdst>]
[-count [-q] [-h] [-v] [-t [<storage type>]] [-u] [-x] [-e] [-s] <path> ...]
[-cp [-f] [-p | -p[topax]] [-d] [-t <thread count>] [-q <thread pool queue size>] <src> ... <dst>]
[-createSnapshot <snapshotDir> [<snapshotName>]]
[-deleteSnapshot <snapshotDir> <snapshotName>]
[-df [-h] [<path> ...]]
[-du [-s] [-h] [-v] [-x] <path> ...]
[-expunge [-immediate] [-fs <path>]]
[-find <path> ... <expression> ...]
[-get [-f] [-p] [-crc] [-ignoreCrc] [-t <thread count>] [-q <thread pool queue size>] <src> ... <localdst>]
[-getfacl [-R] <path>]
[-getfattr [-R] {-n name | -d} [-e en] <path>]
[-getmerge [-nl] [-skip-empty-file] <src> <localdst>]
[-head <file>]
[-help [cmd ...]]
[-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [-e] [<path> ...]]
[-mkdir [-p] <path> ...]
[-moveFromLocal [-f] [-p] [-l] [-d] <localsrc> ... <dst>]
[-moveToLocal <src> <localdst>]
[-mv <src> ... <dst>]
[-put [-f] [-p] [-l] [-d] [-t <thread count>] [-q <thread pool queue size>] <localsrc> ... <dst>]
[-renameSnapshot <snapshotDir> <oldName> <newName>]
[-rm [-f] [-r|-R] [-skipTrash] [-safely] <src> ...]
[-rmdir [--ignore-fail-on-non-empty] <dir> ...]
[-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
[-setfattr {-n name [-v value] | -x name} <path>]
[-setrep [-R] [-w] <rep> <path> ...]
[-stat [format] <path> ...]
[-tail [-f] [-s <sleep interval>] <file>]
[-test -[defswrz] <path>]
[-text [-ignoreCrc] <src> ...]
[-touch [-a] [-m] [-t TIMESTAMP (yyyyMMdd:HHmmss) ] [-c] <path> ...]
[-touchz <path> ...]
[-truncate [-w] <length> <path> ...]
[-usage [cmd ...]]
Generic options supported are:
-conf <configuration file> specify an application configuration file
-D <property=value> define a value for a given property
-fs <file:///|hdfs://namenode:port> specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.
-jt <local|resourcemanager:port> specify a ResourceManager
-files <file1,...> specify a comma-separated list of files to be copied to the map reduce cluster
-libjars <jar1,...> specify a comma-separated list of jar files to be included in the classpath
-archives <archive1,...> specify a comma-separated list of archives to be unarchived on the compute machines
The general command line syntax is:
command [genericOptions] [commandOptions]
Usage: ozone fs [generic options] -setrep [-R] [-w] <rep> <path> ...
bash-4.4$ ozone fs -setrep -w 3 ofs://om/s3v/buck/key2
Replication 3 set: ofs://om/s3v/buck/key2
Waiting for ofs://om/s3v/buck/key2 ... done