product-is icon indicating copy to clipboard operation
product-is copied to clipboard

Published 20 hours ago • verified publisher icon wso2

SCIM2 add user negative test (testAddUserFailure) fails intermittently

Open darshanasbg opened this issue 10 months ago • 4 comments

Describe the issue:

SCIM2UserTestCase.testAddUserFailure() failed during a pr-builder-test run.

 ==========================================================
 
 [INFO] Results:
 [INFO] 
 [ERROR] Failures: 
 [ERROR] org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure(org.wso2.identity.integration.test.scim2.SCIM2UserTestCase)
 [INFO] Run 1: PASS
 [ERROR] Run 2: SCIM2UserTestCase.testAddUserFailure:272 expected:<1> but was:<3>
 [INFO] 
 [INFO] 
 [ERROR] Tests run: 2031, Failures: 1, Errors: 0, Skipped: 2

How to reproduce: Since this was failing in the intermittent nature a log has been introduced with https://github.com/wso2/product-is/pull/19671 to isolate the root cause.

The related log can be found below.

2024-04-24T11:29:37.7009994Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.scim2.rest.api.customSchema.SCIM2CustomSchemaMeTestCase.testDeleteUser ===================
2024-04-24T11:29:37.8143361Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== On test success org.wso2.identity.integration.test.scim2.rest.api.customSchema.SCIM2CustomSchemaMeTestCase.testDeleteUser ===================
2024-04-24T11:29:37.8278415Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-04-24T11:29:37.8380571Z INFO  [org.wso2.carbon.automation.extensions.servers.utils.ServerLogReader] - [2024-04-24 11:29:37,837] [f405e33a-9afb-4ea6-9591-37d0877fad7b] ERROR {org.wso2.carbon.identity.input.validation.mgt.listener.InputValidationListener} - Failed to validate password for user. The minimum length of password should be 8.
2024-04-24T11:29:37.8658857Z INFO  [org.wso2.identity.integration.test.scim2.SCIM2UserTestCase] - Response of testAddUserFailure method:{"schemas":["urn:ietf:params:scim:api:messages:2.0:Error"],"scimType":"invalidValue","detail":"The minimum length of password should be 8.","status":"400"}
2024-04-24T11:29:37.8663544Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== On test success org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-04-24T11:29:37.8667618Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-04-24T11:29:37.8899984Z INFO  [org.wso2.identity.integration.test.scim2.SCIM2UserTestCase] - Response of testAddUserFailure method:{"meta":{"created":"2024-04-24T11:29:37.878567Z","location":"https:\/\/localhost:9853\/t\/wso2.com\/scim2\/Users\/3df6d71a-00fd-4391-9823-25e296dc3e1f","lastModified":"2024-04-24T11:29:37.879547Z","resourceType":"User"},"schemas":["urn:ietf:params:scim:schemas:core:2.0:User","urn:ietf:params:scim:schemas:extension:enterprise:2.0:User","urn:scim:wso2:schema"],"roles":[{"audienceValue":"2bba64f9-4bb1-439b-9747-53776d99c110","display":"everyone","audienceType":"organization","value":"e3aa8d3e-72d4-4675-968f-e4798a815aef","$ref":"https:\/\/localhost:9853\/t\/wso2.com\/scim2\/v2\/Roles\/e3aa8d3e-72d4-4675-968f-e4798a815aef","audienceDisplay":"wso2.com"}],"id":"3df6d71a-00fd-4391-9823-25e296dc3e1f","userName":"passwordIncompatibleUser","urn:ietf:params:scim:schemas:extension:enterprise:2.0:User":{"lastPasswordUpdateTime":"1713958177879"}}
2024-04-24T11:29:37.8962061Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== On test failure org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-04-24T11:29:37.8973196Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testCreateUserWithCharsetEncodingHeader ===================
2024-04-24T11:29:37.9130096Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== On test success org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testCreateUserWithCharsetEncodingHeader ===================

Expected behavior: Should pass the test without and issue.

darshanasbg avatar Apr 24 '24 13:04 darshanasbg

In a successful run the log should be as follows.

2024-04-24T12:32:51.2357650Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-04-24T12:32:51.2462683Z INFO  [org.wso2.carbon.automation.extensions.servers.utils.ServerLogReader] - [2024-04-24 12:32:51,245] [d54e594e-b4ad-4c30-b73f-ba5731dd75dc] ERROR {org.wso2.carbon.identity.input.validation.mgt.listener.InputValidationListener} - Failed to validate password for user. The minimum length of password should be 8.
2024-04-24T12:32:51.2725230Z INFO  [org.wso2.identity.integration.test.scim2.SCIM2UserTestCase] - Response of testAddUserFailure method:{"schemas":["urn:ietf:params:scim:api:messages:2.0:Error"],"scimType":"invalidValue","detail":"The minimum length of password should be 8.","status":"400"}
2024-04-24T12:32:51.2729807Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== On test success org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-04-24T12:32:51.2771775Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-04-24T12:32:51.2830728Z INFO  [org.wso2.carbon.automation.extensions.servers.utils.ServerLogReader] - [2024-04-24 12:32:51,282] [d24c37a3-0e6e-4fcf-8c73-883bb625cd78] ERROR {org.wso2.carbon.identity.input.validation.mgt.listener.InputValidationListener} - Failed to validate password for user. The minimum length of password should be 8.
2024-04-24T12:32:51.2847360Z INFO  [org.wso2.identity.integration.test.scim2.SCIM2UserTestCase] - Response of testAddUserFailure method:{"schemas":["urn:ietf:params:scim:api:messages:2.0:Error"],"scimType":"invalidValue","detail":"The minimum length of password should be 8.","status":"400"}
2024-04-24T12:32:51.2851653Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== On test success org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================

darshanasbg avatar Apr 24 '24 14:04 darshanasbg

https://github.com/wso2/product-is/actions/runs/9210354409 failed due to the same reason.

Test Result Summary

 [INFO] Results:
 [INFO] 
 [ERROR] Failures: 
 [ERROR] org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure(org.wso2.identity.integration.test.scim2.SCIM2UserTestCase)
 [ERROR] Run 1: SCIM2UserTestCase.testAddUserFailure:272 expected:<1> but was:<3>
 [INFO] Run 2: PASS
 [INFO] 
 [INFO] 
 [ERROR] Tests run: 2035, Failures: 1, Errors: 0, Skipped: 2

Logs for the Tests (both runs)

2024-05-23T15:24:36.4375861Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-05-23T15:24:36.4626567Z INFO  [org.wso2.identity.integration.test.scim2.SCIM2UserTestCase] - Response of testAddUserFailure method:{"meta":{"created":"2024-05-23T15:24:36.445783Z","location":"https:\/\/localhost:9853\/scim2\/Users\/242aae83-1fad-4cf3-ac6c-7e2186a660e6","lastModified":"2024-05-23T15:24:36.445783Z","resourceType":"User"},"schemas":["urn:ietf:params:scim:schemas:core:2.0:User","urn:ietf:params:scim:schemas:extension:enterprise:2.0:User","urn:scim:wso2:schema"],"roles":[{"audienceValue":"10084a8d-113f-4211-a0d5-efe36b082211","display":"everyone","audienceType":"organization","value":"646b8225-8c9f-486d-85b2-376c909102da","$ref":"https:\/\/localhost:9853\/scim2\/v2\/Roles\/646b8225-8c9f-486d-85b2-376c909102da","audienceDisplay":"Super"}],"id":"242aae83-1fad-4cf3-ac6c-7e2186a660e6","userName":"passwordIncompatibleUser"}
2024-05-23T15:24:36.4700171Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== On test failure org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-05-23T15:24:36.4711276Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================
2024-05-23T15:24:36.4904100Z INFO  [org.wso2.carbon.automation.extensions.servers.utils.ServerLogReader] - [2024-05-23 15:24:36,489] [91dcd780-2cae-4fcc-9698-5fbe4ad6332e] ERROR {org.wso2.carbon.identity.input.validation.mgt.listener.InputValidationListener} - Failed to validate password for user. The minimum length of password should be 8.
2024-05-23T15:24:36.5561390Z INFO  [org.wso2.identity.integration.test.scim2.SCIM2UserTestCase] - Response of testAddUserFailure method:{"schemas":["urn:ietf:params:scim:api:messages:2.0:Error"],"scimType":"invalidValue","detail":"The minimum length of password should be 8.","status":"400"}
2024-05-23T15:24:36.5565980Z INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== On test success org.wso2.identity.integration.test.scim2.SCIM2UserTestCase.testAddUserFailure ===================

darshanasbg avatar May 23 '24 16:05 darshanasbg

Could see a weird behaviour when using invalid password(password length is less than the minimum requirement) with scim user creation with API calls(attached the recording). Intermittently the user creation getting successful. That should be cause for the above issue as well. In the above log the schema attribute in response should only have one value if it is error but actually it is having 3 attributes because the testAddUserFailure method adding the user successfully.

Need to investigate more to fix the issue.

https://github.com/user-attachments/assets/a1b94371-8dfd-40ed-bfae-574543399548

nilasini avatar Sep 25 '24 04:09 nilasini