[YUNIKORN-1957] Add user/group limit config change e2e test

[pegasas]

What is this PR for?

use the unit test cases in yunikorn-core for add e2e test

What type of PR is it?

• [ ] - Bug Fix
• [ ] - Improvement
• [x] - Feature
• [ ] - Documentation
• [ ] - Hot Fix
• [ ] - Refactoring

Todos

• [ ] - Task

What is the Jira issue?

[YUNIKORN-1957] Add user/group limit config change e2e test

How should this be tested?

Specific user/group limits limits changes to ensure order of precedence:

5. set user limit & wild card user limit only and ensure it has been honoured. 5a) When the user limit is specified, it should be considered for that specific user 5b) When the user limit is changed, make sure it meets new limit of for that specific group
6. set group limit & wild card group limit only and ensure it has been honoured. 6a) When the group limit is specified, it should be considered for that specific group 6b) When the group limit is changed, make sure it meets new limit of for that specific group
7. Repeat the above for max resources and max applications individually and together as well

Screenshots (if appropriate)

Questions:

• [ ] - The licenses files need update.
• [ ] - There is breaking changes for older versions.
• [ ] - It needs documentation.

[pbacsko]

@pegasas please resolve the conflict to proceed

[pegasas]

@pegasas please resolve the conflict to proceed

Hi, @pbacsko , thank you for your quick reply. conflicts are resolved. Also, I am referencing the sample involved in https://github.com/apache/yunikorn-core/commit/0a05068035b71f1a810fad7c38f8cb0eb7af58f6, it seems not to work as expected.

It is my case not in right way or there is bug with wildcard?

[doupache]

Hi @pegasas, thanks for the effort! I am working on https://issues.apache.org/jira/browse/YUNIKORN-1950 But i found a bug and try to fix it.

add to following test to the end TestUserGroupLimitChange test.

//remove Limits , so we can run TestApp2 again
conf.Queues[0].Queues[0].Limits = nil
assert.NilError(t, manager.UpdateConfig(conf.Queues[0], "root"))

increased = manager.IncreaseTrackedResource(queuePathParent, TestApp2, usage, tc.user)
assert.Equal(t, increased, true, "unable to increase tracked resource: queuepath "+queuePathParent+", app "+TestApp2+", res "+usage.String())

decreased = manager.DecreaseTrackedResource(queuePathParent, TestApp2, usage, tc.user, true)
assert.Equal(t, decreased, true, "unable to decreased tracked resource: queuepath "+queuePathParent+", app "+TestApp2+", res "+usage.String())

//set newLimits again, we should not run TestApp2 now
conf.Queues[0].Queues[0].Limits = tc.newLimits
assert.NilError(t, manager.UpdateConfig(conf.Queues[0], "root"))

increased = manager.IncreaseTrackedResource(queuePathParent, TestApp2, usage, tc.user)
assert.Equal(t, increased, false, "should not increase tracked resource: queuepath "+queuePathParent+", app "+TestApp2+", res "+usage.String())

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (2738571) 69.43% compared to head (dec525b) 69.46%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #735      +/-   ##
==========================================
+ Coverage   69.43%   69.46%   +0.02%     
==========================================
  Files          50       50              
  Lines        7993     7993              
==========================================
+ Hits         5550     5552       +2     
+ Misses       2254     2252       -2     
  Partials      189      189              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[pbacsko]

@manirajv06, you have more context regarding the feature. Please check if the comments above.

[manirajv06]

@pegasas please resolve the conflict to proceed

Hi, @pbacsko , thank you for your quick reply. conflicts are resolved. Also, I am referencing the sample involved in apache/yunikorn-core@0a05068, it seems not to work as expected.

It is my case not in right way or there is bug with wildcard?

I don't think above mentioned core commit should affect this test behaviour. Can you resolve the conflict and try running again?

[pegasas]

I don't think above mentioned core commit should affect this test behaviour. Can you resolve the conflict and try running again?

Thanks @manirajv06

It seems conflicts was introduced by https://github.com/apache/yunikorn-k8shim/commit/dd02f2e9055e1c262295a5aa87e9072a6f9d9c83 which was merged yesterday

I have fixed this and re-commit.

[pbacsko]

ping @pegasas have you had time to check the test failures?