ozone
ozone copied to clipboard
HDDS-14114. [DiskBalancer] Fix DiskBalancer tmp directory to be under hdds/<cluster-id>/tmp
What changes were proposed in this pull request?
The DiskBalancer service creates two separate tmp directories due to inconsistent path resolution:
- At startup:
/data/tmp/diskBalancer(incorrect, empty, cleaned up) - During operations:
/data/hdds/CID-xxx/tmp/diskBalancer/(correct, actually used)
[root disk1]# tree -d
.
└── hadoop-ozone
└── datanode
└── data
├── hdds
│ └── CID-322ew-ee56-4c35-8e49-f5d7c0f35645
│ ├── current
│ │ ├── containerDir15
│ │ │ ├── 8058
│ │ │ │ ├── chunks
│ │ │ │ └── metadata
│ │ │ └── 8064
│ │ │ ├── chunks
│ │ │ └── metadata
│ │ ├── containerDir17
│ │ │ ├── 9008
│ │ │ │ ├── chunks
│ │ │ │ └── metadata
│ │ │ └── 9068
│ │ │ ├── chunks
│ │ │ └── metadata
│ │ ├── containerDir19
│ │ │ ├── 10007
│ │ │ │ ├── chunks
│ │ │ │ └── metadata
│ │ │ └── 10082
│ │ │ ├── chunks
│ │ │ └── metadata
│ │ └── containerDir21
│ │ └── 11003
│ │ ├── chunks
│ │ └── metadata
│ ├── DS-aa9c8571-9e8e-4948-9c7a-b20bdd75c666
│ │ ├── container.db
│ │ ├── db.checkpoints
│ │ └── db.snapshots
│ │ └── checkpointState
│ └── tmp
│ ├── deleted-containers
│ ├── diskBalancer
│ │ └── 8048
│ │ └── chunks
│ └── disk-check
└── tmp
├── container-copy
└── diskBalancer
This needs to be corrected.
Solution:
Correct the initialisation logic of DiskBalancer to create diskBalancer tmp at correct place. Introduce DiskBalancer.start() to construct tmp dir.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14114
How was this patch tested?
Added unit tests in TestDiskBalancerService
And check using docker cluster:
Before fix:
bash-5.1$ pwd
/data/hdds3/tmp/diskBalancer
After Fix:
bash-5.1$ pwd
/data/hdds3/hdds/CID-2a0edc92-45ba-4e5a-a292-08d3ed3d4809/tmp/diskBalancer