robottelo icon indicating copy to clipboard operation
robottelo copied to clipboard

Published 20 hours ago • verified publisher icon SatelliteQE

Test leapp upgrade with non-admin user BZ#2257302

Open shweta83 opened this issue 1 year ago • 25 comments

Problem Statement

Missing scenario of Leapp client upgrade from non-admin user

Solution

Added test scenario to test BZ#2257302

Related Issues

shweta83 avatar Feb 15 '24 07:02 shweta83

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Feb 15 '24 07:02 shweta83

PRT Result

Build Number: 5735
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin
Test Result : ========== 1 passed, 1 deselected, 11 warnings in 3009.08s (0:50:09) ===========

Satellite-QE avatar Feb 15 '24 08:02 Satellite-QE

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Feb 26 '24 14:02 shweta83

PRT Result

Build Number: 5839
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin
Test Result : ========== 1 failed, 1 deselected, 11 warnings in 2054.09s (0:34:14) ===========

Satellite-QE avatar Feb 26 '24 15:02 Satellite-QE

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Feb 27 '24 09:02 shweta83

PRT Result

Build Number: 5855
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin
Test Result : ====== 1 failed, 1 deselected, 11 warnings, 1 error in 2828.30s (0:47:08) ======

Satellite-QE avatar Feb 27 '24 09:02 Satellite-QE

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Feb 27 '24 10:02 shweta83

PRT Result

Build Number: 5858
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin
Test Result : ====== 1 failed, 1 deselected, 11 warnings, 1 error in 2906.87s (0:48:26) ======

Satellite-QE avatar Feb 27 '24 11:02 Satellite-QE

PRT Result

Build Number: 5862
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin
Test Result : ====== 1 failed, 1 deselected, 11 warnings, 1 error in 3031.22s (0:50:31) ======

Satellite-QE avatar Feb 27 '24 13:02 Satellite-QE

Looks ~OK but consider these:

1. don't make another test, make it a parameter of the original test whether the user is/is not an admin

There are 2 reasons because of which I did not make this parametrized with the existing test, first is there are more actions for non-admin user to make it parametrized which will just make it complicated. The second reason is I wanted to have the first test to mark it for end_to_end and upgrade so created different test to cover this scenario. 2. we usually prefix tests with test_positive or test_negative This I can improve in the test name.

shweta83 avatar Mar 05 '24 11:03 shweta83

I tried to compare this new (non-admin) test with the original (admin) one and I see some differences...

  • The original test uses flag reboot=True, but new one has reboot=False. And what is Channel flag for? I think this should be described in docstring or at least in a comment.

  • Why don't we clean cache and check if OS version has changed? Last 3 lines of original test...

    custom_leapp_host.clean_cached_properties()
    new_ver = str(custom_leapp_host.os_version)
    assert new_ver == upgrade_path['target_version']

rmynar avatar Mar 05 '24 16:03 rmynar

I tried to compare this new (non-admin) test with the original (admin) one and I see some differences...

  • The original test uses flag reboot=True, but new one has reboot=False. And what is Channel flag for? I think this should be described in docstring or at least in a comment.
  • Why don't we clean cache and check if OS version has changed? Last 3 lines of original test...
    custom_leapp_host.clean_cached_properties()
    new_ver = str(custom_leapp_host.os_version)
    assert new_ver == upgrade_path['target_version']

@rmynar The difference between these 2 tests are that the first one uses api for the job which doesn't break the ssh connection post upgrade. With this test, I have used cli since the test module is under cli but the issue with this is ssh connection breaks post upgrade which causes error establishing connection after reboot, hence I set reboot to False and run the reboot in next step which doesn't break my ssh connection. The answer to your second question is we have introduced new option for remote execution in stream which is needed. We need to update the first test as well so it doesn't fail on stream as the parameter is mandatory.

shweta83 avatar Mar 06 '24 08:03 shweta83

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Mar 07 '24 06:03 shweta83

PRT Result

Build Number: 5969
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin --external-logging
Test Result : ====== 1 failed, 1 deselected, 11 warnings, 1 error in 3073.03s (0:51:13) ======

Satellite-QE avatar Mar 07 '24 07:03 Satellite-QE

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Mar 13 '24 10:03 shweta83

PRT Result

Build Number: 6045
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin --external-logging
Test Result : ======= 1 failed, 1 deselected, 1 warning, 1 error in 2749.89s (0:45:49) =======

Satellite-QE avatar Mar 13 '24 13:03 Satellite-QE

PRT Result

Build Number: 6048
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin --external-logging
Test Result : ======= 1 failed, 1 deselected, 1 warning, 1 error in 3008.01s (0:50:08) =======

Satellite-QE avatar Mar 13 '24 14:03 Satellite-QE

PRT Result

Build Number: 6069
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin --external-logging
Test Result : ======= 1 failed, 1 deselected, 1 warning, 1 error in 2898.71s (0:48:18) =======

Satellite-QE avatar Mar 14 '24 05:03 Satellite-QE

The test is currently failing because of the hammer bug: 24017. I am moving this PR to draft until the issue resolves.

shweta83 avatar Mar 20 '24 06:03 shweta83

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Apr 03 '24 19:04 shweta83

PRT Result

Build Number: 6303
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin --external-logging
Test Result : ====== 1 failed, 1 deselected, 4 warnings, 1 error in 3125.90s (0:52:05) =======

Satellite-QE avatar Apr 03 '24 20:04 Satellite-QE

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Apr 08 '24 08:04 shweta83

PRT Result

Build Number: 6341
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin --external-logging
Test Result : ====== 1 failed, 1 deselected, 4 warnings, 1 error in 3074.88s (0:51:14) =======

Satellite-QE avatar Apr 08 '24 09:04 Satellite-QE

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin env: ROBOTTELO_server__deploy_workflow: 'deploy-satellite' ROBOTTELO_server__deploy_arguments__deploy_rhel_version: '8' ROBOTTELO_server__deploy_arguments__deploy_sat_version: '6.15' ROBOTTELO_server__deploy_arguments__deploy_snap_version: 'ystream'

shweta83 avatar Apr 08 '24 09:04 shweta83

PRT Result

Build Number: 6345
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin --external-logging
Test Result : ================= 1 deselected, 3 warnings, 1 error in 48.99s ==================

Satellite-QE avatar Apr 08 '24 09:04 Satellite-QE

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

shweta83 avatar Apr 30 '24 07:04 shweta83

trigger: test-robottelo pytest: tests/foreman/cli/test_leapp_client.py -k test_leapp_upgrade_rhel_non_admin

omkarkhatavkar avatar May 06 '24 10:05 omkarkhatavkar