apache
HDDS-10111. TestListKey: Add new test cases for OFS mode.
What changes were proposed in this pull request?
Add new test cases for OFS mode.
What is the link to the Apache JIRA
HDDS-10111.
How was this patch tested?
- local test
- CI test: https://github.com/TaiJuWu/ozone/actions/runs/7677750808
@adoroszlai Could you take a look? I'm not sure this CI test is related to this PR.
Hello @adoroszlai, When I try adding test for object store in this commit. These results always show empty, is it excepted? or it's an issue?
results always show empty
can you please clarify which results you refer to?
results always show empty
can you please clarify which results you refer to?
My exception is no matter use Legacy mode or object store mode. The results of testListKey should be same. But all testcases for object store mode, all results are empty list.
For example, in this testcase the results of both should be same, but in object store mode its results is [].
Expected :[a1/b2/, a1/b2/d1/, a1/b2/d2/, a1/b2/d3/]
Actual :[]
My exception is no matter use Legacy mode or object store mode. The results of testListKey should be same. But all testcases for object store mode, all results are empty list.
For example, in this testcase the results of both should be same, but in object store mode its results is [].
Expected :[a1/b2/, a1/b2/d1/, a1/b2/d2/, a1/b2/d3/] Actual :[]
@sadanand48 can you please help with this?
The problem here is during creation of keys we have a slash at the beginning of all keys
private static void buildNameSpaceTree(OzoneBucket ozoneBucket) throws Exception {
LinkedList<String> keys = new LinkedList<>();
keys.add("/a1/b1/c1111.tx");
keys.add("/a1/b1/c1222.tx");
keys.add("/a1/b1/c1333.tx");
FSO and legacy(fsenabled=true by default) normalize the initial slash and store only the after-paths like vol/buck/a1/b1 in the DB however in the OBS case the initial slash is also stored like vol/buck//a1/b1/.
This is causing an issue during listKeys as the seek-key that is getting constructed is vol/buck/a1/b1 and there is no such key present in the table thereby returning an empty list. When we do a listKeys we should provide initial slash only then will it get the proper results.
One solution if we want all to test this against all bucket types is in the creation is to remove the initial slashes of all keys.
The problem here is during creation of keys we have a slash at the beginning of all keys
private static void buildNameSpaceTree(OzoneBucket ozoneBucket) throws Exception { LinkedList<String> keys = new LinkedList<>(); keys.add("/a1/b1/c1111.tx"); keys.add("/a1/b1/c1222.tx"); keys.add("/a1/b1/c1333.tx");FSO and legacy(fsenabled=true by default) normalize the initial slash and store only the after-paths like
vol/buck/a1/b1in the DB however in the OBS case the initial slash is also stored likevol/buck//a1/b1/. This is causing an issue during listKeys as the seek-key that is getting constructed is vol/buck/a1/b1 and there is no such key present in the table thereby returning an empty list. When we do a listKeys we should provide initial slash only then will it get the proper results. One solution if we want all to test this against all bucket types is in the creation is to remove the initial slashes of all keys.
Got it. Thanks for your answer. Can this issue can be fix by modifying OBS layout?
@adoroszlai Could you take a look this PR? This PR is not related above question. If you need to fix above issue I mentioned, maybe we can open another Jira.
@TaiJuWu can you please resolve conflicts?
Done. There is a build error in my computer, this also happened on master branch.
java: cannot find symbol
symbol: class NetworkNode
location: class org.apache.hadoop.hdds.protocol.proto.HddsProtos
How could I resolve it?