Trilinos
Trilinos copied to clipboard
trilinos
Ifpack2_MTSGS_belos_MPI_1 randomly failing in build Trilinos-atdm-cee-rhel6_intel-18.0.2_mpich2-3.2_openmp_static_opt starting before 2019-04-24
Bug Report
CC: @trilinos/ifpack2, @srajama1 (Trilinos Linear Solvers Product Lead), @bartlettroscoe, @fryeguy52
Next Action Status
Description
As shown in this query the test:
- Ifpack2_MTSGS_belos_MPI_1
is failing in the build:
- Trilinos-atdm-cee-rhel6_intel-18.0.2_mpich2-3.2_openmp_static_opt
- Trilinos-atdm-ats1-knl_intel-19.0.4_mpich-7.7.15_openmp_static_opt
This has failed 9 times in the last 4 weeks with something similar to:
Achieved tolerance: 6.44895e-10
Actual iters(20) > expected number of iterations (19), or resid-norm(0) >= 1.e-7
proc 0 total program time: 0.0258739
End Result: TEST FAILED
Current Status on CDash
Current 2 week history on CDash
Steps to Reproduce
One should be able to reproduce this failure on a machine with a cee rhel6 environment as described in:
- https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md
More specifically, the commands given for a machine with a cee rhel6 environment are provided at:
- https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#cee-rhel6-environment
The exact commands to reproduce this issue should be:
$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh Trilinos-atdm-cee-rhel6_intel-18.0.2_mpich2-3.2_openmp_static_opt
$ cmake \
-GNinja \
-DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
-DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_Ifpack2=ON \
$TRILINOS_DIR
$ make NP=16
$ ctest -j16
FYI: I think this thing started randomly failing when we (based on @nmhamster's suggestion) updated OMP_NUM_THREADS from 2 to 3 for this build. This just shows we need to be testing with a greater variety of threads (hence #2422).
Test results for issue #5006 as of 2020-08-16
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-18.0.2_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 3 | 11 | 19 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-08-23
Tests with issue trackers Failed: twif=1
Detailed test results: (click to expand)
Tests with issue trackers Failed: twif=1
| Site | Build Name | Test Name | Status | Details | Consecutive Non-pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-18.0.2_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Failed | Completed (Failed) | 1 | 11 | 18 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-08-30
Tests with issue trackers Failed: twif=1
Detailed test results: (click to expand)
Tests with issue trackers Failed: twif=1
| Site | Build Name | Test Name | Status | Details | Consecutive Non-pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-18.0.2_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Failed | Completed (Failed) | 1 | 10 | 19 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-09-06
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 2 | 0 | 2 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-09-13
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 1 | 3 | 6 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-09-20
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 1 | 5 | 11 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-09-27
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 4 | 7 | 16 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-10-04
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 1 | 9 | 21 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-10-11
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 8 | 7 | 23 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-10-18
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 15 | 5 | 25 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-10-25
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 6 | 3 | 27 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-11-01
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 13 | 2 | 28 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-11-08
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 20 | 1 | 29 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-11-15
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 27 | 1 | 29 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-11-22
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 5 | 1 | 29 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-11-29
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 12 | 1 | 29 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-12-06
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 1 | 4 | 26 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-12-21
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 4 | 8 | 20 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2020-12-27
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel6 | Trilinos-atdm-cee-rhel6_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 10 | 8 | 20 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2021-01-10
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel7 | Trilinos-atdm-cee-rhel7_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 8 | 0 | 8 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2021-01-17
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel7 | Trilinos-atdm-cee-rhel7_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 1 | 3 | 12 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2021-01-24
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel7 | Trilinos-atdm-cee-rhel7_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 8 | 3 | 19 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
Test results for issue #5006 as of 2021-01-31
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel7 | Trilinos-atdm-cee-rhel7_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 14 | 3 | 24 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
@srajama1 - Do we have an assignee for this one? I am still seeing failures on various builds dating back as far as the data goes (currently July.) It looks to be on multiple builds, not just the one in the original report.
@prwolfe @brian-kelley knows this code, however this seems to be some random failure. We can probably adjust the tolerance, not actually fix anything. This is code that is used in production, so we are reasonably confident there are no failures.
Thanks @srajama1 - I did not look into the details of the failures and if you believe this is noise removing it is appropriate.
Paul
Test results for issue #5006 as of 2021-02-07
Tests with issue trackers Passed: twip=1
Detailed test results: (click to expand)
Tests with issue trackers Passed: twip=1
| Site | Build Name | Test Name | Status | Details | Consecutive Pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel7 | Trilinos-atdm-cee-rhel7_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Passed | Completed | 2 | 5 | 23 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.
This cropped up last night in Trilinos-atdm-ats1-knl_intel-19.0.4_mpich-7.7.15_openmp_static_opt: https://testing-dev.sandia.gov/cdash/test/42899798.
Test results for issue #5006 as of 2021-02-14
Tests with issue trackers Failed: twif=1
Detailed test results: (click to expand)
Tests with issue trackers Failed: twif=1
| Site | Build Name | Test Name | Status | Details | Consecutive Non-pass Days | Non-pass Last 30 Days | Pass Last 30 Days | Issue Tracker |
|---|---|---|---|---|---|---|---|---|
| cee-rhel7 | Trilinos-atdm-cee-rhel7_intel-19.0.3_mpich2-3.2_openmp_static_opt | Ifpack2_MTSGS_belos_MPI_1 | Failed | Completed (Failed) | 1 | 5 | 23 | #5006 |
This is an automated comment generated by Grover. Each week, Grover collates and reports data from CDash in an automated way to make it easier for developers to stay on top of their issues. Grover saw that there are tests being tracked on CDash that are associated with this open issue. If you have a question, please reach out to Ross. I'm just a cat.