server
server copied to clipboard
MariaDB
Test pull request to keep track of currently failing tests on 11.6
Investigating MariaDB CI Failures
When developers contribute to the open-source MariaDB project, their code is subject to continuous integration (CI) checks, which include running the entire MTR test suite in the repository. However, the CI process can be ambiguous because there are tests that fail on the base 11.6 branch itself. This ambiguity causes confusion for developers who cannot easily determine whether the CI failures on their pull requests are due to issues in the base 11.6 branch or their changes. To address this confusion, I have identified the tests that consistently fail, aiming to clarify the source of these failures.
| Test Runner | Test Name | Failure Count | Reason | Most Recent Fail Time | Ticket |
|---|---|---|---|---|---|
| amd64-debian-11-msan | Command timeout | 1/10 | Command timed out after 950 seconds without output. | N/A | N/A |
| amd64-debian-11-msan | perfschema.lowercase_fs_off | 6/10 | Extra row returned, result length mismatch. | 2024-08-02 16:47:58 | N/A |
| amd64-debian-11-msan-16 | rpl.rpl_change_master_demote 'mix' | 1/10 | IO thread still running after START SLAVE UNTIL command. | 2024-08-01 23:09:39 | MDEV-34554 |
| amd64-fedora-38-last-N-failed | main.mysql_upgrade-28915 | 1/10 | Test timed out, could not connect to MySQL server. | 2024-08-01 23:29:23 | N/A |
| amd64-fedora-38-last-N-failed | main.tmp_table_count-7586 | 10/10 | Expected 0 files, but 60 temporary files created. | 2024-08-02 16:41:37 | MDEV-34605 |
| amd64-ubuntu-2004-debug | encryption.debug_key_management 'undo' | 1/10 | 'innodb_encrypt_tables' variable not recognized. | 2024-07-26 23:11:28 | MDEV-13841 |
| amd64-ubuntu-2004-debug | perfschema.lowercase_fs_off | 3/10 | Extra row returned, result length mismatch. | 2024-08-02 00:09:16 | N/A |
| amd64-ubuntu-2204-debug-ps | perfschema.lowercase_fs_off | 1/10 | Extra row returned, result length mismatch. | 2024-08-02 00:07:43 | N/A |
| amd64-ubuntu-2204-debug-ps | rpl.rpl_change_master_demote 'mix' | 2/10 | IO thread still running after START SLAVE UNTIL command. | 2024-08-01 23:57:25 | MDEV-33316 |
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.
Note that the failing tests aren’t always consistent, though most reoccur frequently.
Note that the failing tests aren’t always consistent, though most reoccur frequently.
Thanks for the heads up. I'll be doing 10 runs and note down which ones are known to fail.
Note you can just look at:
- https://buildbot.mariadb.org/#/grid?branch=11.6
Note that we track this in tickets linked to MDEV-33073. As an example MDEV-34605 fixes the common amd64-fedora-38-last-N-failed
Note that we track this in tickets linked to MDEV-33073. As an example MDEV-34605 fixes the common
amd64-fedora-38-last-N-failed
@LinuxJedi Is there an up-to-date Jira ticket that lists all the current failing tests, allowing developers to refer to it when checking test failures?
Note that we track this in tickets linked to MDEV-33073. As an example MDEV-34605 fixes the common
amd64-fedora-38-last-N-failed@LinuxJedi Is there an up-to-date Jira ticket that lists all the current failing tests, allowing developers to refer to it when checking test failures?
MDEV-33073 as mentioned is the best we have. I can't remember what JIRA privileges are needed but under More -> Link you might be able to make issues part of this issue.
New bug reports welcome after searching. https://buildbot.mariadb.org/cr/ may assist searching test cases to get a full perspective on what branches they are failing and perhaps about when it was introduced.
Discussions welcome on https://mariadb.zulipchat.com/#narrow/stream/118759-general/topic/Test.20failures or a new topic per failure if desired.
Related mailing list thread: https://lists.mariadb.org/hyperkitty/list/[email protected]/thread/NRMKHZ6JRDEWQ73HKV4XYVKHSEY7X3WF/ discussing protected branch rules which might help prevent CI failing by automatically gatekeeping that commits that did not pass CI are not accepted in git push/merge operations.
See also MDEV-33073: always green buildbot which links to CI failure Jiras.
For the record, latest commit on branch main is still failing..
FYI @cvicentiu that this PR has notes on all the failures.
@HashimSyedUBC thanks for accumulating failing tests. As it was discussed previously, there is long-term MDEV-33073: always green buildbot that tracks failing tests.
I also created short-term MDEV-36647 - No red leaves in the forest and linked all issues you've outlined. Missing issues were reported. It aims to fix 20-30 outstanding current failures.
Speaking of the main goal of this PR, that is listing failing tests, there is Buildbot cross-reference, which provides up-to-date list for all MariaDB branches.
So, feels like we're done with this PR, right?
