Filter test collection based on files changed in upstream PRs

[tpapaioa]

Problem Statement

Filter tests based on the files modified in an upstream (foreman or katello) PR.

Solution

The new upstream_pr pytest plugin uses the PyGithub library to get the list of files modified by the given PR, and filters the test collection based on the mapping of upstream repo paths/filenames to test markers in robottelo.

For example, with the following new settings block settings.github:

$ cat conf/github.yaml
GITHUB:
  FOREMAN:
    REPO: foreman
    ORG: theforeman
    RULES:
      - PATH: "app/models/concerns/hostext/"
        MARKER: component
        MARKER_ARG: Hosts
      - PATH: "webpack/assets/javascripts/react_app/components/SettingUpdateModal/"
        MARKER: component
        MARKER_ARG: Settings
  KATELLO:
    REPO: katello
    ORG: Katello
    RULES:
      - PATH: "app/controllers/katello/api/v2/errata_controller.rb"
        MARKER: component
        MARKER_ARG: ErrataManagement

then the following Foreman PR, which modifies files under webpack/assets/javascripts/react_app/components/SettingUpdateModal/, should filter the tests on the Settings component. In this case, --upstream-pr foreman/9969 is equivalent to --component Settings:

Fixes #37013 - change the 'All hosts' menu item's url based on the new_host_page setting https://github.com/theforeman/foreman/pull/9969

>>> FOREMAN_PR_ID = 9969
>>> from github import Github
>>> from pprint import pp
>>> pr = Github().get_repo("theforeman/foreman").get_pull(FOREMAN_PR_ID)
>>> filenames = [file.filename for commit in pr.get_commits() for file in commit.files]
>>> pp(filenames)
['app/registries/foreman/settings/general.rb',
 'webpack/assets/javascripts/react_app/Root/Context/ForemanContext.js',
 'webpack/assets/javascripts/react_app/Root/ReactApp.js',
 'webpack/assets/javascripts/react_app/components/Layout/LayoutConstants.js',
 'webpack/assets/javascripts/react_app/components/Layout/LayoutHelper.js',
 'webpack/assets/javascripts/react_app/components/Layout/Navigation.js',
 'webpack/assets/javascripts/react_app/components/Layout/index.js',
 'webpack/assets/javascripts/react_app/components/SettingUpdateModal/SettingUpdateModalConstants.js',
 'webpack/assets/javascripts/react_app/components/SettingUpdateModal/components/SettingForm/SettingForm.js',
 'webpack/test_setup.js']

$ pytest tests/foreman --co --upstream-pr foreman/9969
[...]
=== 54/5426 tests collected (5372 deselected) in 16.91s ===

$ pytest tests/foreman --co --component Settings
[...]
=== 54/5426 tests collected (5372 deselected) in 15.14s ===

Similarly, for this Katello PR:

Fixes #37394 - content_view_filter_id only works for ID-type filters https://github.com/Katello/katello/pull/10985

the file app/controllers/katello/api/v2/errata_controller.rb is matched exactly, and tests with the ErrataManagement component are selected:

>>> KATELLO_PR_ID = 10985
>>> pr = Github().get_repo("Katello/katello").get_pull(KATELLO_PR_ID)
>>> filenames = [file.filename for commit in pr.get_commits() for file in commit.files]
>>> pp(filenames)
['app/controllers/katello/api/v2/debs_controller.rb',
 'app/controllers/katello/api/v2/errata_controller.rb',
 'app/controllers/katello/api/v2/packages_controller.rb']

$ pytest tests/foreman --co --upstream-pr katello/10985
[...]
=== 70/5426 tests collected (5356 deselected) in 15.45s ===

$ pytest tests/foreman --co --component ErrataManagement
[...]
== 70/5426 tests collected (5356 deselected) in 13.78s ===