packit
Packit API enhancement - make /testing-farm/results parametrizable, expose ref
Description
Gathering data for test results from the Testing Farm is very time demanding and tedious with growing number of pipelines. One needs to go over each check, click the respective hyperlink and scroll through the results. We have our own way to easily parse the results from the result page/artifact URL, however getting to the pipeline_id and or URL is the pain point.
One way to get to the result page/pipeline ID "faster" is to parse the Packit's API /testing-farm/results and/or /projects/{forge}/{namespace}/{repo_name}/prs namespace.
-
The issue with the
/testing-farm/resultsis, that it contains all the projects' testing-farm requests handled by packit with no way to parametrize the query. Being able to parametrize the query to narrow the results to a specific project would speed up the data gathering immensely. -
The other option, however not exclusive to the testing-farm namespace enhancement is exposing
refand possibly any type of timestamp for the/projects/{forge}/{namespace}/{repo_name}/prsresponse body. Specifically inside the tests field.
Benefit
If proven working, we could provide the artifact URL gathering and reporting functionality to packit itself. This could help to resolve https://github.com/packit/packit-service/issues/2205
Additionally, there could be /packit report feature provided in the future to report the test results by Packit in a comment.
❯ tesar report -f tesar_latest_jobs_20230920143852
Reporting for the requested tasks:
AlmaLinux OS 8.6.20220901 x86_64 tier0/non-destructive COMPLETE
Rocky-8-EC2-Base-8.8-20230518.0.x86_64 tier0/non-destructive COMPLETE
+--------------------------------------+----------------------------------------+-----------------------------+--------+
| UUID | Target | Test Plan | Result |
+--------------------------------------+----------------------------------------+-----------------------------+--------+
| 2ae608e4-3456-47ca-93b5-1dbf2c420b97 | AlmaLinux OS 8.6.20220901 x86_64 | | |
| | | plans/tier0/non-destructive | FAILED |
| 87ae5111-27ea-4718-9c5a-ab1e03e01718 | Rocky-8-EC2-Base-8.8-20230518.0.x86_64 | | |
| | | plans/tier0/non-destructive | FAILED |
+--------------------------------------+----------------------------------------+-----------------------------+--------+
❯ tesar report -f tesar_latest_jobs_20230920143852 -l2 --short
Reporting for the requested tasks:
AlmaLinux OS 8.6.20220901 x86_64 tier0/non-destructive COMPLETE
Rocky-8-EC2-Base-8.8-20230518.0.x86_64 tier0/non-destructive COMPLETE
+--------------------------------------+----------------------------------------+-----------------+---------------------------------------------+--------+
| UUID | Target | Test Plan | Test Case | Result |
+--------------------------------------+----------------------------------------+-----------------+---------------------------------------------+--------+
| 2ae608e4-3456-47ca-93b5-1dbf2c420b97 | AlmaLinux OS 8.6.20220901 x86_64 | | | |
| | | non-destructive | | FAILED |
| | | | simultaneous_runs | ERROR |
| | | | convert_successful_report | ERROR |
| | | | failures_and_skips_in_report | ERROR |
| | | | successful_report | ERROR |
| | | | clean_cache | ERROR |
| | | | c2r_version_latest_or_newer | ERROR |
| | | | c2r_version_older_no_envar | ERROR |
| | | | c2r_version_older_with_envar | ERROR |
| | | | data_collection_acknowledgement | ERROR |
| | | | disable_data_collection | ERROR |
| | | | incomplete_rollback_in_analyze | ERROR |
| | | | log_file_exists | ERROR |
| | | | log_rhsm_error | ERROR |
| | | | manpage | ERROR |
| | | | root_privileges | ERROR |
| | | | smoke | ERROR |
| | | | variant_message | ERROR |
| | | | config_cli_priority | ERROR |
| | | | config_custom_path_custom_filename | ERROR |
| | | | config_custom_path_standard_filename | ERROR |
| | | | config_password_file_priority | ERROR |
| | | | config_standard_paths_priority | ERROR |
| | | | config_standard_paths_priority_diff_methods | ERROR |
| | | | custom_invalid_repo_provided | ERROR |
| | | | custom_valid_repo_provided | ERROR |
| | | | passing_activation_key_to_submgr | ERROR |
| | | | passing_password_to_submgr | ERROR |
| | | | available_connection | ERROR |
| | | | unavailable_connection | ERROR |
| | | | custom_kernel | ERROR |
| | | | failed_repoquery | ERROR |
| | | | non_latest_kernel_error | ERROR |
| | | | yum_exclude_kernel | ERROR |
| | | | custom_module_loaded | ERROR |
| | | | custom_module_not_loaded | ERROR |
| | | | unsupported_kmod_with_envar | ERROR |
| | | | force_loaded_kmod | ERROR |
| | | | tainted_kernel | ERROR |
| | | | logfile_starts_with_command | ERROR |
| | | | modified_releasever_in_configs | ERROR |
| | | | modified_releasever_to_unknown_release | ERROR |
| | | | packages_untracked_graceful_rollback | ERROR |
| | | | rhsm_cleanup | ERROR |
| | | | registration_start | ERROR |
| | | | registration_success | ERROR |
| | | | test_missing_credentials_rollback | ERROR |
| | | | package_download_error | ERROR |
| | | | transaction_validation_error | ERROR |
| | | | missing_system_release | ERROR |
| | | | backup_os_release_no_envar | ERROR |
| | | | backup_os_release_with_envar | ERROR |
| | | | unsuccessful_satellite_registration | ERROR |
+--------------------------------------+----------------------------------------+-----------------+---------------------------------------------+--------+
Importance
high It's not blocking us to use Packit per se, but it's faster to query for results if we use our wrapper to schedule test jobs and parse the resulting request pipeline IDs.
What is the impacted category (job)?
Testing Farm tests
Workaround
- [X] There is an existing workaround that can be used until this feature is implemented.
Participation
- [X] I am willing to submit a pull request for this issue. (Packit team is happy to help!)
hi @danmyway ! Thanks for the issue. I see this could improve your situation a lot. The only thing that might cause trouble is our current DB architecture, with more complicated queries we have bumped into performance issues previously.
You can see the current DB models architecture here.
As for the 1.option, how do you propose the parametrizing? Would ref be enough here as well?
As for the second one, if we would do this in general (not only for /prs), this could enable some improvements for dashboard as well (somehow related are https://github.com/packit/dashboard/issues/196 and https://github.com/packit/dashboard/issues/257). However, we will need to have a closer look how demanding these queries would be.
hi @danmyway ! What is the status of this issue for you and its current importance?
Hi @lbarcziova, I'm sorry for the unresponsiveness, I have somehow missed this. Given there is no result retention in the GitHub GUI and getting to the results through the Packit Dashboard sifting through dozens of PRs is very time-consuming, I would say it is pretty important to me. We have a tooling ready, which could parse the testing farm results based on the request UUID, the data leading to the UUIDs exist, but there is no easy way to connect these endpoints to get the desired results nowadays.