uyuni icon indicating copy to clipboard operation
uyuni copied to clipboard

Published 20 hours ago • verified publisher icon uyuni-project

QE: Update test suite to Ruby 2.7

Open nodeg opened this issue 2 years ago • 5 comments

What does this PR change?

At the moment this is only a dummy PR to be able to use the PR test pipeline. See https://github.com/SUSE/spacewalk/issues/17431

GUI diff

No difference.

  • [x] DONE

Documentation

  • No documentation needed: only internal and user invisible changes

  • [x] DONE

Test coverage

  • Cucumber tests were added

  • [x] DONE

Links

  • [x] DONE

Changelogs

Make sure the changelogs entries you are adding are compliant with https://github.com/uyuni-project/uyuni/wiki/Contributing#changelogs and https://github.com/uyuni-project/uyuni/wiki/Contributing#uyuni-projectuyuni-repository

If you don't need a changelog check, please mark this checkbox:

  • [x] No changelog needed

If you uncheck the checkbox after the PR is created, you will need to re-run changelog_test (see below)

Re-run a test

If you need to re-run a test, please mark the related checkbox, it will be unchecked automatically once it has re-run:

  • [ ] Re-run test "changelog_test"
  • [ ] Re-run test "backend_unittests_pgsql"
  • [ ] Re-run test "java_pgsql_tests"
  • [ ] Re-run test "schema_migration_test_pgsql"
  • [ ] Re-run test "susemanager_unittests"
  • [ ] Re-run test "javascript_lint"
  • [ ] Re-run test "spacecmd_unittests"

nodeg avatar Feb 01 '23 12:02 nodeg

A full test suite run with Ruby 2.7 and the default gems were successful and showed only 2 failures in secondary which are considered flaky tests.

ruby27_consoletext.txt

image image

However I see some other errors that are not reported:

[2023-02-02T22:55:56.611Z] Scenario: One search result for State/Province                                 # features/secondary/srv_advanced_search.feature:32
[2023-02-02T22:55:56.611Z] This scenario ran at: 2023-02-02 23:55:11 +0100
[2023-02-02T22:55:56.611Z] #<Thread:0x000055c9e44652e0 /root/spacewalk/testsuite/features/support/api_test.rb:64 run> terminated with exception (report_on_exception is true):
[2023-02-02T22:55:56.611Z] /root/spacewalk/testsuite/features/support/http_client.rb:99:in `call': API failure: Error communicating with search server (RuntimeError)
[2023-02-02T22:55:56.611Z] from /root/spacewalk/testsuite/features/support/api_test.rb:68:in `block (2 levels) in call'
[2023-02-02T22:55:56.611Z] from /root/spacewalk/testsuite/features/support/api_test.rb:65:in `synchronize'
[2023-02-02T22:55:56.611Z] from /root/spacewalk/testsuite/features/support/api_test.rb:65:in `block in call'
[2023-02-02T22:55:56.611Z] Given I have a property "State/Province" with value "Surrey" on "sle_minion" # features/step_definitions/navigation_steps.rb:1069
[2023-02-02T22:55:56.611Z] Search by hostname: suma-pr9-min-sles15
[2023-02-02T22:55:56.611Z] rhn-search still not responding.
[2023-02-02T22:55:56.611Z] Error message: API failure: Error communicating with search server
[2023-02-02T22:55:56.611Z] Search by hostname: suma-pr9-min-sles15
[2023-02-02T22:55:56.611Z] []
[2023-02-02T22:55:56.611Z] rhn-search still not responding.
[2023-02-02T22:55:56.611Z] Error message: undefined method `[]' for nil:NilClass
[2023-02-02T22:55:56.611Z] Search by hostname: suma-pr9-min-sles15
[2023-02-02T22:55:56.611Z] [{"hostname"=>"suma-pr9-min-sles15.mgr.prv.suse.net", "ip"=>"10.84.222.132", "name"=>"suma-pr9-min-sles15.mgr.prv.suse.net", "id"=>1000010030, "las
t_checkin"=>"Feb 2, 2023, 11:53:36 PM"}]
[2023-02-02T22:55:56.611Z] When I follow the left menu "Systems > Advanced Search"                      # features/step_definitions/navigation_steps.rb:343
(...)
[2023-02-02T18:07:48.550Z] Scenario: Cleanup: cancel all scheduled actions # features/secondary/srv_maintenance_windows.feature:122
[2023-02-02T18:07:48.550Z] This scenario ran at: 2023-02-02 19:07:38 +0100
[2023-02-02T18:07:48.550Z] #<Thread:0x0000561fff8089f8 /root/spacewalk/testsuite/features/support/api_test.rb:64 run> terminated with exception (report_on_exception is true):
[2023-02-02T18:07:48.550Z] /root/spacewalk/testsuite/features/support/http_client.rb:99:in `call': API failure: Cannot cancel an action in PICKED UP state. (RuntimeError)
[2023-02-02T18:07:48.550Z] from /root/spacewalk/testsuite/features/support/api_test.rb:68:in `block (2 levels) in call'
[2023-02-02T18:07:48.551Z] from /root/spacewalk/testsuite/features/support/api_test.rb:65:in `synchronize'
[2023-02-02T18:07:48.551Z] from /root/spacewalk/testsuite/features/support/api_test.rb:65:in `block in call'
[2023-02-02T18:07:48.551Z] When I cancel all scheduled actions           # features/step_definitions/api_common.rb:403
[2023-02-02T18:07:48.551Z]  - Try to cancel "Package Install/Upgrade scheduled by admin" action
(...)
[2023-02-03T00:42:45.425Z] Scenario: Bootstrap via API a non-existing system                           # features/secondary/min_bootstrap_api.feature:64
[2023-02-03T00:42:45.425Z] This scenario ran at: 2023-02-03 01:42:23 +0100
[2023-02-03T00:42:45.425Z] #<Thread:0x000055c9e4f588c8 /root/spacewalk/testsuite/features/support/api_test.rb:64 run> terminated with exception (report_on_exception is true):
[2023-02-03T00:42:45.425Z] /root/spacewalk/testsuite/features/support/http_client.rb:99:in `call': API failure: [An error has occurred during salt execution: unable to establish ssh connection.] (RuntimeError)
[2023-02-03T00:42:45.425Z] from /root/spacewalk/testsuite/features/support/api_test.rb:68:in `block (2 levels) in call'
[2023-02-03T00:42:45.425Z] from /root/spacewalk/testsuite/features/support/api_test.rb:65:in `synchronize'
[2023-02-03T00:42:45.425Z] from /root/spacewalk/testsuite/features/support/api_test.rb:65:in `block in call'
[2023-02-03T00:42:45.425Z] When I call system.bootstrap() on unknown host, I should get an API fault # features/step_definitions/api_common.rb:31
[2023-02-03T00:42:45.425Z] This scenario took: 5 seconds
[2023-02-03T00:42:45.425Z] 
[2023-02-03T00:42:45.426Z] Scenario: Bootstrap a salt-ssh system with activation key and default contact method                                                                  # features/secondary/min_bootstrap_api.feature:67
[2023-02-03T00:42:45.426Z] This scenario ran at: 2023-02-03 01:42:28 +0100
[2023-02-03T00:42:45.426Z] #<Thread:0x000055c9e4a5d8c8 /root/spacewalk/testsuite/features/support/api_test.rb:64 run> terminated with exception (report_on_exception is true):
[2023-02-03T00:42:45.426Z] /root/spacewalk/testsuite/features/support/http_client.rb:99:in `call': API failure: [Selected activation key cannot be used as its contact method is not compatible with the salt-ssh systems.] (RuntimeError)
[2023-02-03T00:42:45.426Z] from /root/spacewalk/testsuite/features/support/api_test.rb:68:in `block (2 levels) in call'
[2023-02-03T00:42:45.426Z] from /root/spacewalk/testsuite/features/support/api_test.rb:65:in `synchronize'
[2023-02-03T00:42:45.426Z] from /root/spacewalk/testsuite/features/support/api_test.rb:65:in `block in call'
[2023-02-03T00:42:45.426Z] When I call system.bootstrap() on a Salt minion with saltSSH = true, but with activation key with default contact method, I should get an API fault # features/step_definitions/api_common.rb:42

nodeg avatar Feb 03 '23 08:02 nodeg

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] avatar Sep 20 '23 01:09 github-actions[bot]

:wave: Hello! Thanks for contributing to our project. Acceptance tests will take some time (aprox. 1h), please be patient :coffee: You can see the progress at the end of this page and at https://github.com/uyuni-project/uyuni/pull/6559/checks Once tests finish, if they fail, you can check :eyes: the cucumber report. See the link at the output of the action. You can also check the artifacts section, which contains the logs at https://github.com/uyuni-project/uyuni/pull/6559/checks.

If you are unsure the failing tests are related to your code, you can check the "reference jobs". These are jobs that run on a scheduled time with code from master. If they fail for the same reason as your build, it means the tests or the infrastructure are broken. If they do not fail, but yours do, it means it is related to your code.

Reference tests:

  • https://github.com/uyuni-project/uyuni/actions/workflows/acceptance_tests_secondary_parallel.yml?query=event%3Aschedule

  • https://github.com/uyuni-project/uyuni/actions/workflows/acceptance_tests_secondary.yml?query=event%3Aschedule

KNOWN ISSUES

Sometimes the build can fail when pulling new jar files from download.opensuse.org . This is a known limitation. Given this happens rarely, when it does, all you need to do is rerun the test. Sorry for the inconvenience.

For more tips on troubleshooting, see the troubleshooting guide.

Happy hacking! :warning: You should not merge if acceptance tests fail to pass. :warning:

github-actions[bot] avatar Oct 30 '23 14:10 github-actions[bot]

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] avatar Dec 31 '23 01:12 github-actions[bot]

This PR was closed because it has been stalled for 10 days with no activity.

github-actions[bot] avatar Jan 11 '24 01:01 github-actions[bot]

This will get reopened/worked on after the 5.0 FCS.

nodeg avatar Mar 05 '24 13:03 nodeg

deprecated Object#=~

  • https://bugs.ruby-lang.org/issues/15231
  • https://ruby-doc.org/core-3.1.1/Object.html#method-i-3D~

Ruby < 2.7.6

image

Ruby >= 2.7.6

image image

The match? method provides a more explicit and readable way to perform regular expression matching. It returns a boolean indicating whether there is a match.

Occurrences

https://github.com/search?q=repo%3Auyuni-project%2Fuyuni+%3D%7E+language%3ARuby&type=code&l=Ruby

Links

  • https://www.rubyguides.com/2015/06/ruby-regex/
  • https://stackoverflow.com/questions/3025838/what-is-the-operator-in-ruby

nodeg avatar Aug 02 '24 09:08 nodeg 

dominik-ctl:~ # ps aux | grep chrome
root     20760  0.6  0.9 33648216 18384 pts/1  Sl   16:43   0:05 /usr/bin/chromedriver --port=9515
root     20768  0.7  9.7 34278968 196732 pts/1 S<l  16:43   0:06 /usr/lib64/chromium/chrome --no-sandbox --allow-pre-commit-input --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-dev-shm-usage --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --enable-automation --enable-logging --headless=new --ignore-certificate-errors --js-flags=--max_old_space_size=2048 --log-level=0 --no-first-run --no-service-autorun --password-store=basic --remote-debugging-port=0 --test-type=webdriver --use-mock-keychain --user-data-dir=/tmp/.org.chromium.Chromium.hj591W --window-size=2048,2048 --noerrdialogs --ozone-platform=headless --ozone-override-screen-size=800,600 --use-angle=swiftshader-webgl data:,
root     20775  0.0  0.2 33570456 4472 ?       Sl   16:43   0:00 /usr/lib64/chromium/chrome_crashpad_handler --monitor-self --monitor-self-annotation=ptype=crashpad-handler --database=/root/.config/chromium/Crash Reports --annotation=channel=stable --annotation=lsb-release=openSUSE Leap 15.5 --annotation=plat=Linux --annotation=prod=Chrome_Linux --annotation=ver=125.0.6422.141 --initial-client-fd=137 --shared-client-connection
root     20778  0.0  3.3 34021252 68184 pts/1  S    16:43   0:00 /usr/lib64/chromium/chrome --type=zygote --no-zygote-sandbox --no-sandbox --enable-logging --headless=new --log-level=0 --crashpad-handler-pid=0 --enable-crash-reporter=,stable --noerrdialogs --user-data-dir=/tmp/.org.chromium.Chromium.hj591W --change-stack-guard-on-fork=enable --no-first-run --ozone-platform=headless --ozone-override-screen-size=800,600 --use-angle=swiftshader-webgl
root     20779  0.0  3.4 34021256 68756 pts/1  S    16:43   0:00 /usr/lib64/chromium/chrome --type=zygote --no-sandbox --enable-logging --headless=new --log-level=0 --crashpad-handler-pid=0 --enable-crash-reporter=,stable --noerrdialogs --user-data-dir=/tmp/.org.chromium.Chromium.hj591W --change-stack-guard-on-fork=enable --no-first-run --ozone-platform=headless --ozone-override-screen-size=800,600 --use-angle=swiftshader-webgl
root     20780  0.0  0.2 33562244 4264 ?       S    16:43   0:00 /usr/lib64/chromium/chrome_crashpad_handler --no-periodic-tasks --monitor-self-annotation=ptype=crashpad-handler --database=/root/.config/chromium/Crash Reports --annotation=channel=stable --annotation=lsb-release=openSUSE Leap 15.5 --annotation=plat=Linux --annotation=prod=Chrome_Linux --annotation=ver=125.0.6422.141 --initial-client-fd=4 --shared-client-connection
root     20805  3.1  5.3 34132292 108564 pts/1 S<l  16:43   0:25 /usr/lib64/chromium/chrome --type=gpu-process --no-sandbox --disable-dev-shm-usage --headless=new --ozone-platform=headless --use-angle=swiftshader-webgl --crashpad-handler-pid=0 --enable-crash-reporter=,stable --noerrdialogs --user-data-dir=/tmp/.org.chromium.Chromium.hj591W --change-stack-guard-on-fork=enable --gpu-preferences=WAAAAAAAAAAgAAAMAAAAAAAAAAAAAAAAAABgAAEAAAA4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGAAAAAAAAAAYAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAA== --shared-files --field-trial-handle=3,i,15621469101444341480,12443019721556723055,262144 --disable-features=PaintHolding --variations-seed-version --enable-logging --log-level=0
root     20806  0.1  5.0 34080328 101816 pts/1 Sl   16:43   0:00 /usr/lib64/chromium/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --lang=en-US --service-sandbox-type=none --no-sandbox --disable-dev-shm-usage --ignore-certificate-errors --use-angle=swiftshader-webgl --ignore-certificate-errors --crashpad-handler-pid=0 --enable-crash-reporter=,stable --noerrdialogs --user-data-dir=/tmp/.org.chromium.Chromium.hj591W --change-stack-guard-on-fork=enable --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,15621469101444341480,12443019721556723055,262144 --disable-features=PaintHolding --variations-seed-version --enable-logging --log-level=0
root     20809  0.0  2.2 34063276 45712 pts/1  Sl   16:43   0:00 /usr/lib64/chromium/chrome --type=utility --utility-sub-type=storage.mojom.StorageService --lang=en-US --service-sandbox-type=utility --no-sandbox --disable-dev-shm-usage --ignore-certificate-errors --use-angle=swiftshader-webgl --ignore-certificate-errors --crashpad-handler-pid=0 --enable-crash-reporter=,stable --noerrdialogs --user-data-dir=/tmp/.org.chromium.Chromium.hj591W --change-stack-guard-on-fork=enable --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,15621469101444341480,12443019721556723055,262144 --disable-features=PaintHolding --variations-seed-version --enable-logging --log-level=0
root     20844 13.0  9.7 1186718436 196848 pts/1 Sl 16:43   1:45 /usr/lib64/chromium/chrome --type=renderer --crashpad-handler-pid=0 --enable-crash-reporter=,stable --noerrdialogs --user-data-dir=/tmp/.org.chromium.Chromium.hj591W --change-stack-guard-on-fork=enable --no-sandbox --disable-dev-shm-usage --enable-automation --remote-debugging-port=0 --test-type=webdriver --allow-pre-commit-input --js-flags=--max_old_space_size=2048 --ozone-platform=headless --disable-gpu-compositing --lang=en-US --num-raster-threads=1 --renderer-client-id=7 --time-ticks-at-unix-epoch=-1722601050575533 --launch-time-ticks=8745117724 --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,15621469101444341480,12443019721556723055,262144 --disable-features=PaintHolding --variations-seed-version --enable-logging --log-level=0
root     20854  0.0  3.5 1186679868 71280 pts/1 S<l 16:43   0:00 /usr/lib64/chromium/chrome --type=renderer --crashpad-handler-pid=0 --enable-crash-reporter=,stable --noerrdialogs --user-data-dir=/tmp/.org.chromium.Chromium.hj591W --change-stack-guard-on-fork=enable --no-sandbox --disable-dev-shm-usage --enable-automation --remote-debugging-port=0 --test-type=webdriver --allow-pre-commit-input --js-flags=--max_old_space_size=2048 --ozone-platform=headless --disable-gpu-compositing --lang=en-US --num-raster-threads=1 --renderer-client-id=8 --time-ticks-at-unix-epoch=-1722601050575533 --launch-time-ticks=8745239025 --shared-files=v8_context_snapshot_data:100 --field-trial-handle=3,i,15621469101444341480,12443019721556723055,262144 --disable-features=PaintHolding --variations-seed-version --enable-logging --log-level=0
root     21638  0.0  0.0   8212   816 pts/2    S+   16:56   0:00 grep --color=auto chrome
dominik-ctl:~ # ps aux | grep ruby
root     20724  0.0  1.2 480168 25740 pts/1    Sl+  16:43   0:00 ruby.ruby3.3 /usr/bin/rake cucumber:reposync
root     20727  1.2  4.8 984200 96780 pts/1    Sl+  16:43   0:09 ruby.ruby3.3 /usr/bin/cucumber --profile default -f html -o results/output_20240802164307-reposync.html -f json -o results/output_20240802164307-reposync.json -f junit -o results_junit -f pretty -r features features/reposync/srv_disable_scheduled_reposync.feature features/reposync/srv_sync_channels.feature features/reposync/srv_sync_products.feature features/reposync/srv_enable_sync_products.feature features/reposync/srv_create_fake_channels.feature features/reposync/srv_create_fake_repositories.feature features/reposync/srv_sync_fake_channels.feature features/reposync/srv_create_activationkey.feature features/reposync/allcli_update_activationkeys.feature

image

nodeg avatar Aug 02 '24 14:08 nodeg

Minitest error

Fixed with another minitest import in env.rb.

dominik-ctl:~/spacewalk/testsuite # rake cucumber:sanity_check
/usr/bin/ruby.ruby3.3 -S bundle exec cucumber --profile default -f html -o results/output_20240812095247-sanity_check.html -f json -o results/output_20240812095247-sanity_check.json -f junit -o results_junit -f pretty -r features features/core/allcli_sanity.feature
DEBUG MODE ENABLED.
Capybara APP Host: https://dominik-srv.mgr.suse.de:8888
uninitialized constant MiniTest (NameError)
/root/spacewalk/testsuite/features/support/env.rb:107:in `<top (required)>'

https://github.com/uyuni-project/uyuni/blob/09c6af7be4525f1d1edb8fc584b83d232c541378/testsuite/features/support/env.rb#L74-L75 https://github.com/uyuni-project/uyuni/blob/09c6af7be4525f1d1edb8fc584b83d232c541378/testsuite/features/support/env.rb#L116-L117

nodeg avatar Aug 15 '24 08:08 nodeg

Closed in favor of https://github.com/uyuni-project/uyuni/pull/9181

nodeg avatar Aug 20 '24 11:08 nodeg