E3SM-Project
add new default production spa file
This is based on a v3 LR (ne30pg2) experiment, with everything corrected in the recently identified bug. Some experiments were done. Documentation to follow.
Internal documentation of creating the new file.
[Non-B4B]
PR Preview Action v1.4.7
:---:
:rocket: Deployed preview to https://E3SM-Project.github.io/scream/pr-preview/pr-2786/
on branch gh-pages at 2024-04-19 04:34 UTC
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:
Pull Request Auto Testing STARTING (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5152
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5503
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Using Repos:
- Repo: SCREAM (E3SM-Project/scream)
- Branch: mahf708/spa/new-spa-file
- SHA: eb49619b659aeb194ef99a898b87f8bc2bc2993b
- Mode: TEST_REPO
Pull Request Author: mahf708
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED
Pull Request Auto Testing has PASSED (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5152
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5503
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging NO REVIEWS HAVE BEEN PERFORMED ON THIS PULL REQUEST!
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...
@hassanbeydoun and/or @brhillman can you confirm this is the correct SPA data file? I'm inclined to believe it is, but since the production runs will be impacted I want to check before merging to master.
@AaronDonahue what should we do about ne4pg2? I was going to wait for Luca to come back because I believe he was the last one to do anything with the SPA file at ne4pg2, but maybe you know what we should here?
Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...
The base branch has been updated since the last successful testing.
- last PASS base branch sha: inished; status = PASSED
- current base branch sha : a6c73f47ede28cf44f6610a7146c5ecb96f7202f The AutoTester will discard the last PASS, and re-test the PR from scratch
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:
Pull Request Auto Testing STARTING (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5169
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5517
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Using Repos:
- Repo: SCREAM (E3SM-Project/scream)
- Branch: mahf708/spa/new-spa-file
- SHA: eb49619b659aeb194ef99a898b87f8bc2bc2993b
- Mode: TEST_REPO
Pull Request Author: mahf708
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED
Pull Request Auto Testing has PASSED (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5169
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5517
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...
The base branch has been updated since the last successful testing.
- last PASS base branch sha: inished; status = PASSED
- current base branch sha : a6c73f47ede28cf44f6610a7146c5ecb96f7202f The AutoTester will discard the last PASS, and re-test the PR from scratch
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:
Pull Request Auto Testing STARTING (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5170
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5518
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Using Repos:
- Repo: SCREAM (E3SM-Project/scream)
- Branch: mahf708/spa/new-spa-file
- SHA: eb49619b659aeb194ef99a898b87f8bc2bc2993b
- Mode: TEST_REPO
Pull Request Author: mahf708
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED
Pull Request Auto Testing has PASSED (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5170
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5518
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...
The base branch has been updated since the last successful testing.
- last PASS base branch sha: inished; status = PASSED
- current base branch sha : a6c73f47ede28cf44f6610a7146c5ecb96f7202f The AutoTester will discard the last PASS, and re-test the PR from scratch
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:
Pull Request Auto Testing STARTING (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5174
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5522
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Using Repos:
- Repo: SCREAM (E3SM-Project/scream)
- Branch: mahf708/spa/new-spa-file
- SHA: eb49619b659aeb194ef99a898b87f8bc2bc2993b
- Mode: TEST_REPO
Pull Request Author: mahf708
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:
Pull Request Auto Testing STARTING (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5180
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5524
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Using Repos:
- Repo: SCREAM (E3SM-Project/scream)
- Branch: mahf708/spa/new-spa-file
- SHA: eb49619b659aeb194ef99a898b87f8bc2bc2993b
- Mode: TEST_REPO
Pull Request Author: mahf708
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED
Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.
Pull Request Auto Testing has FAILED (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5180
- Status: FAILED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5524
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
SCREAM_PullRequest_Autotester_Mappy # 5180 FAILED (click to see last 100 lines of console output)
Waiting for tests to finish
PASS ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1.C.20240411_074620_3hm19r
PASS ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-4 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-4.C.20240411_074620_3hm19r
PASS ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-rad_frequency_2--scream-output-preset-5 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-rad_frequency_2--scream-output-preset-5.C.20240411_074620_3hm19r
PASS ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu9.scream-output-preset-2 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu9.scream-output-preset-2.C.20240411_074620_3hm19r
PASS ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu9 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu9.C.20240411_074620_3hm19r
PASS PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1.C.20240411_074620_3hm19r
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu9.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu9.scream-mam4xx-optics.C.20240411_074620_3hm19r
PASS SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu9.scream-output-preset-3 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu9.scream-output-preset-3.C.20240411_074620_3hm19r
test-scheduler took 764.3368895053864 seconds'
+ errors='Waiting for tests to finish
PASS ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1.C.20240411_074620_3hm19r
PASS ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-4 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-4.C.20240411_074620_3hm19r
PASS ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-rad_frequency_2--scream-output-preset-5 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-rad_frequency_2--scream-output-preset-5.C.20240411_074620_3hm19r
PASS ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu9.scream-output-preset-2 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu9.scream-output-preset-2.C.20240411_074620_3hm19r
PASS ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu9 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu9.C.20240411_074620_3hm19r
PASS PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1.C.20240411_074620_3hm19r
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu9.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu9.scream-mam4xx-optics.C.20240411_074620_3hm19r
PASS SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu9.scream-output-preset-3 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu9.scream-output-preset-3.C.20240411_074620_3hm19r
test-scheduler took 764.3368895053864 seconds'
+ V1_FAILURES_DETAILS+='Waiting for tests to finish
PASS ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1.C.20240411_074620_3hm19r
PASS ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-4 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-4.C.20240411_074620_3hm19r
PASS ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-rad_frequency_2--scream-output-preset-5 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-rad_frequency_2--scream-output-preset-5.C.20240411_074620_3hm19r
PASS ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu9.scream-output-preset-2 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu9.scream-output-preset-2.C.20240411_074620_3hm19r
PASS ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu9 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu9.C.20240411_074620_3hm19r
PASS PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1.C.20240411_074620_3hm19r
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu9.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu9.scream-mam4xx-optics.C.20240411_074620_3hm19r
PASS SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu9.scream-output-preset-3 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu9.scream-output-preset-3.C.20240411_074620_3hm19r
test-scheduler took 764.3368895053864 seconds'
+ set +x
######################################################
FAILS DETECTED:
SCREAM V1 TESTING FAILED!
Waiting for tests to finish
PASS ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1.C.20240411_074620_3hm19r
PASS ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-4 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-4.C.20240411_074620_3hm19r
PASS ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-rad_frequency_2--scream-output-preset-5 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-rad_frequency_2--scream-output-preset-5.C.20240411_074620_3hm19r
PASS ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu9.scream-output-preset-2 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu9.scream-output-preset-2.C.20240411_074620_3hm19r
PASS ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu9 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu9.C.20240411_074620_3hm19r
PASS PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu9.scream-output-preset-1.C.20240411_074620_3hm19r
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu9.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu9.scream-mam4xx-optics.C.20240411_074620_3hm19r
PASS SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu9.scream-output-preset-3 RUN
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu9.scream-output-preset-3.C.20240411_074620_3hm19r
test-scheduler took 764.3368895053864 seconds
######################################################
Build step 'Execute shell' marked build as failure
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 22393 killed;
[ssh-agent] Stopped.
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash -le
cd $WORKSPACE/${BUILD_ID}/
./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
We're having issues with some test-launcher job hanging forever. So let's make sure we clean all penting test-launcher jobs
squeue -o"%.7i %u %40j" | grep e3sm-jenkins | grep test-launcher | awk '{ print $1 }' | xargs -r scancel
[SCREAM_PullRequest_Autotester_Mappy] $ /bin/bash -le /tmp/jenkins5266578179955217794.sh
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Sending e-mails to: [email protected]
Finished: FAILURE
SCREAM_PullRequest_Autotester_Weaver # 5524 PASSED (click to see last 100 lines of console output)
Start 111: shoc_p3_nudging_glob_novert
111/128 Test #111: shoc_p3_nudging_glob_novert ............................. Passed 1.94 sec
Start 112: homme_shoc_cld_p3_rrtmgp_np1
112/128 Test #112: homme_shoc_cld_p3_rrtmgp_np1 ............................ Passed 5.84 sec
Start 113: homme_shoc_cld_p3_rrtmgp_baseline_cmp
113/128 Test #113: homme_shoc_cld_p3_rrtmgp_baseline_cmp ................... Passed 0.15 sec
Start 114: homme_shoc_cld_p3_rrtmgp_pg2_np1
114/128 Test #114: homme_shoc_cld_p3_rrtmgp_pg2_np1 ........................ Passed 4.13 sec
Start 115: homme_shoc_cld_p3_rrtmgp_pg2_baseline_cmp
115/128 Test #115: homme_shoc_cld_p3_rrtmgp_pg2_baseline_cmp ............... Passed 0.13 sec
Start 116: model_baseline
116/128 Test #116: model_baseline .......................................... Passed 6.94 sec
Start 117: model_initial
117/128 Test #117: model_initial ........................................... Passed 4.11 sec
Start 118: model_restart
118/128 Test #118: model_restart ........................................... Passed 4.74 sec
Start 119: restarted_vs_monolithic_check_np1
119/128 Test #119: restarted_vs_monolithic_check_np1 ....................... Passed 0.09 sec
Start 120: homme_shoc_cld_spa_p3_rrtmgp_np1
120/128 Test #120: homme_shoc_cld_spa_p3_rrtmgp_np1 ........................ Passed 4.66 sec
Start 121: homme_shoc_cld_spa_p3_rrtmgp_baseline_cmp
121/128 Test #121: homme_shoc_cld_spa_p3_rrtmgp_baseline_cmp ............... Passed 0.22 sec
Start 122: homme_shoc_cld_spa_p3_rrtmgp_128levels_np1
122/128 Test #122: homme_shoc_cld_spa_p3_rrtmgp_128levels_np1 .............. Passed 6.04 sec
Start 123: homme_shoc_cld_spa_p3_rrtmgp_128levels_tend_check_np1
123/128 Test #123: homme_shoc_cld_spa_p3_rrtmgp_128levels_tend_check_np1 ... Passed 1.17 sec
Start 124: homme_shoc_cld_spa_p3_rrtmgp_128levels_baseline_cmp
124/128 Test #124: homme_shoc_cld_spa_p3_rrtmgp_128levels_baseline_cmp ..... Passed 0.77 sec
Start 125: homme_shoc_cld_spa_p3_rrtmgp_dp_np1
125/128 Test #125: homme_shoc_cld_spa_p3_rrtmgp_dp_np1 ..................... Passed 17.61 sec
Start 126: homme_shoc_cld_spa_p3_rrtmgp_dp_baseline_cmp
126/128 Test #126: homme_shoc_cld_spa_p3_rrtmgp_dp_baseline_cmp ............ Passed 0.26 sec
Start 127: homme_shoc_cld_p3_mam_optics_rrtmgp_np1
127/128 Test #127: homme_shoc_cld_p3_mam_optics_rrtmgp_np1 ................. Passed 8.23 sec
Start 128: homme_shoc_cld_p3_mam_optics_rrtmgp_baseline_cmp
128/128 Test #128: homme_shoc_cld_p3_mam_optics_rrtmgp_baseline_cmp ........ Passed 0.25 sec
100% tests passed, 0 tests failed out of 128
Label Time Summary:
baseline_cmp = 59.35 secproc (16 tests)
baseline_gen = 135.97 secproc (18 tests)
bfbhash = 0.90 secproc (1 test)
check = 1.11 secproc (1 test)
cld = 8.38 secproc (3 tests)
cld_fraction = 2.34 secproc (1 test)
cxx baseline_cmp = 8.51 secproc (2 tests)
diagnostics = 33.68 secproc (19 tests)
driver = 21.40 secproc (8 tests)
dynamics = 5.17 secproc (3 tests)
fail = 34.95 secproc (4 tests)
io = 39.67 secproc (12 tests)
mam4_optics = 3.90 secproc (1 test)
nudging = 7.75 secproc (2 tests)
p3 = 136.95 secproc (7 tests)
p3_sk = 71.66 secproc (2 tests)
physics = 228.13 secproc (19 tests)
remap = 3.71 secproc (1 test)
rrtmgp = 19.13 secproc (9 tests)
shoc = 11.87 secproc (7 tests)
spa = 8.46 secproc (4 tests)
surface_coupling = 2.36 secproc (1 test)
Total Test time (real) = 592.25 sec
Testing '''ee4ec6634caa706730558428f10ca216b1cb4317''' for test '''release'''
RUN: taskset -c 104-155 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/release/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/release -DBUILD_NAME_MOD=release -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Release -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/release" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/release
Testing '''ee4ec6634caa706730558428f10ca216b1cb4317''' for test '''full_debug'''
RUN: taskset -c 0-51 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/full_debug/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/full_debug -DBUILD_NAME_MOD=full_debug -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Debug -DEKAT_DEFAULT_BFB=True -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/full_debug" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/full_debug
Testing '''ee4ec6634caa706730558428f10ca216b1cb4317''' for test '''full_sp_debug'''
RUN: taskset -c 52-103 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/full_sp_debug/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/full_sp_debug -DBUILD_NAME_MOD=full_sp_debug -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Debug -DEKAT_DEFAULT_BFB=True -DSCREAM_DOUBLE_PRECISION=False -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/full_sp_debug" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx/ctest-build/full_sp_debug
OVERALL STATUS: PASS
Starting analysis on weaver with cmd: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -i -m weaver
RUN: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -i -m weaver
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5524/scream/components/eamxx
Completed analysis on weaver'
- [[ 0 != 0 ]]
- [[ 1 == 0 ]]
- [[ weaver == \m\a\p\p\y ]]
- set +x
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash -le
cd $WORKSPACE/${BUILD_ID}/
./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
[SCREAM_PullRequest_Autotester_Weaver] $ /bin/bash -le /tmp/jenkins3739579300593789572.sh
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Finished: SUCCESS
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:
Pull Request Auto Testing STARTING (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5183
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5527
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Using Repos:
- Repo: SCREAM (E3SM-Project/scream)
- Branch: mahf708/spa/new-spa-file
- SHA: eb49619b659aeb194ef99a898b87f8bc2bc2993b
- Mode: TEST_REPO
Pull Request Author: mahf708
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED
Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.
Pull Request Auto Testing has FAILED (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5183
- Status: FAILED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5527
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
SCREAM_PullRequest_Autotester_Mappy # 5183 FAILED (click to see last 100 lines of console output)
Waiting for tests to finish
DIFF ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_123636_5963v3
DIFF ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240411_123636_5963v3
DIFF ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240411_123636_5963v3
DIFF ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240411_123636_5963v3
DIFF ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240411_123636_5963v3
DIFF PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_123636_5963v3
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240411_123636_5963v3
DIFF SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240411_123636_5963v3
test-scheduler took 796.7957057952881 seconds'
+ errors='Waiting for tests to finish
DIFF ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_123636_5963v3
DIFF ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240411_123636_5963v3
DIFF ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240411_123636_5963v3
DIFF ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240411_123636_5963v3
DIFF ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240411_123636_5963v3
DIFF PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_123636_5963v3
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240411_123636_5963v3
DIFF SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240411_123636_5963v3
test-scheduler took 796.7957057952881 seconds'
+ V1_FAILURES_DETAILS+='Waiting for tests to finish
DIFF ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_123636_5963v3
DIFF ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240411_123636_5963v3
DIFF ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240411_123636_5963v3
DIFF ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240411_123636_5963v3
DIFF ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240411_123636_5963v3
DIFF PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_123636_5963v3
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240411_123636_5963v3
DIFF SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240411_123636_5963v3
test-scheduler took 796.7957057952881 seconds'
+ set +x
######################################################
FAILS DETECTED:
SCREAM V1 TESTING FAILED!
Waiting for tests to finish
DIFF ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_123636_5963v3
DIFF ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240411_123636_5963v3
DIFF ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240411_123636_5963v3
DIFF ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240411_123636_5963v3
DIFF ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240411_123636_5963v3
DIFF PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_123636_5963v3
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240411_123636_5963v3
DIFF SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240411_123636_5963v3
test-scheduler took 796.7957057952881 seconds
######################################################
Build step 'Execute shell' marked build as failure
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 49947 killed;
[ssh-agent] Stopped.
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash -le
cd $WORKSPACE/${BUILD_ID}/
./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
We're having issues with some test-launcher job hanging forever. So let's make sure we clean all penting test-launcher jobs
squeue -o"%.7i %u %40j" | grep e3sm-jenkins | grep test-launcher | awk '{ print $1 }' | xargs -r scancel
[SCREAM_PullRequest_Autotester_Mappy] $ /bin/bash -le /tmp/jenkins7506274305859912481.sh
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Sending e-mails to: [email protected]
Finished: FAILURE
SCREAM_PullRequest_Autotester_Weaver # 5527 PASSED (click to see last 100 lines of console output)
Start 111: shoc_p3_nudging_glob_novert
111/128 Test #111: shoc_p3_nudging_glob_novert ............................. Passed 2.02 sec
Start 112: homme_shoc_cld_p3_rrtmgp_np1
112/128 Test #112: homme_shoc_cld_p3_rrtmgp_np1 ............................ Passed 4.62 sec
Start 113: homme_shoc_cld_p3_rrtmgp_baseline_cmp
113/128 Test #113: homme_shoc_cld_p3_rrtmgp_baseline_cmp ................... Passed 0.41 sec
Start 114: homme_shoc_cld_p3_rrtmgp_pg2_np1
114/128 Test #114: homme_shoc_cld_p3_rrtmgp_pg2_np1 ........................ Passed 4.95 sec
Start 115: homme_shoc_cld_p3_rrtmgp_pg2_baseline_cmp
115/128 Test #115: homme_shoc_cld_p3_rrtmgp_pg2_baseline_cmp ............... Passed 0.16 sec
Start 116: model_baseline
116/128 Test #116: model_baseline .......................................... Passed 5.58 sec
Start 117: model_initial
117/128 Test #117: model_initial ........................................... Passed 4.44 sec
Start 118: model_restart
118/128 Test #118: model_restart ........................................... Passed 4.93 sec
Start 119: restarted_vs_monolithic_check_np1
119/128 Test #119: restarted_vs_monolithic_check_np1 ....................... Passed 0.09 sec
Start 120: homme_shoc_cld_spa_p3_rrtmgp_np1
120/128 Test #120: homme_shoc_cld_spa_p3_rrtmgp_np1 ........................ Passed 7.74 sec
Start 121: homme_shoc_cld_spa_p3_rrtmgp_baseline_cmp
121/128 Test #121: homme_shoc_cld_spa_p3_rrtmgp_baseline_cmp ............... Passed 0.24 sec
Start 122: homme_shoc_cld_spa_p3_rrtmgp_128levels_np1
122/128 Test #122: homme_shoc_cld_spa_p3_rrtmgp_128levels_np1 .............. Passed 9.38 sec
Start 123: homme_shoc_cld_spa_p3_rrtmgp_128levels_tend_check_np1
123/128 Test #123: homme_shoc_cld_spa_p3_rrtmgp_128levels_tend_check_np1 ... Passed 1.61 sec
Start 124: homme_shoc_cld_spa_p3_rrtmgp_128levels_baseline_cmp
124/128 Test #124: homme_shoc_cld_spa_p3_rrtmgp_128levels_baseline_cmp ..... Passed 0.76 sec
Start 125: homme_shoc_cld_spa_p3_rrtmgp_dp_np1
125/128 Test #125: homme_shoc_cld_spa_p3_rrtmgp_dp_np1 ..................... Passed 20.76 sec
Start 126: homme_shoc_cld_spa_p3_rrtmgp_dp_baseline_cmp
126/128 Test #126: homme_shoc_cld_spa_p3_rrtmgp_dp_baseline_cmp ............ Passed 0.30 sec
Start 127: homme_shoc_cld_p3_mam_optics_rrtmgp_np1
127/128 Test #127: homme_shoc_cld_p3_mam_optics_rrtmgp_np1 ................. Passed 9.82 sec
Start 128: homme_shoc_cld_p3_mam_optics_rrtmgp_baseline_cmp
128/128 Test #128: homme_shoc_cld_p3_mam_optics_rrtmgp_baseline_cmp ........ Passed 0.27 sec
100% tests passed, 0 tests failed out of 128
Label Time Summary:
baseline_cmp = 97.44 secproc (16 tests)
baseline_gen = 183.17 secproc (18 tests)
bfbhash = 0.89 secproc (1 test)
check = 0.92 secproc (1 test)
cld = 13.22 secproc (3 tests)
cld_fraction = 2.18 secproc (1 test)
cxx baseline_cmp = 10.70 secproc (2 tests)
diagnostics = 33.78 secproc (19 tests)
driver = 27.67 secproc (8 tests)
dynamics = 7.38 secproc (3 tests)
fail = 54.41 secproc (4 tests)
io = 41.72 secproc (12 tests)
mam4_optics = 4.72 secproc (1 test)
nudging = 9.47 secproc (2 tests)
p3 = 142.07 secproc (7 tests)
p3_sk = 125.38 secproc (2 tests)
physics = 288.56 secproc (19 tests)
remap = 5.33 secproc (1 test)
rrtmgp = 26.37 secproc (9 tests)
shoc = 16.86 secproc (7 tests)
spa = 10.41 secproc (4 tests)
surface_coupling = 2.48 secproc (1 test)
Total Test time (real) = 687.69 sec
Testing '''6d3d3d0614d282145b6202cd87d6fc5838e08fa5''' for test '''release'''
RUN: taskset -c 104-155 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/release/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/release -DBUILD_NAME_MOD=release -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Release -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/release" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/release
Testing '''6d3d3d0614d282145b6202cd87d6fc5838e08fa5''' for test '''full_debug'''
RUN: taskset -c 0-51 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/full_debug/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/full_debug -DBUILD_NAME_MOD=full_debug -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Debug -DEKAT_DEFAULT_BFB=True -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/full_debug" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/full_debug
Testing '''6d3d3d0614d282145b6202cd87d6fc5838e08fa5''' for test '''full_sp_debug'''
RUN: taskset -c 52-103 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/full_sp_debug/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/full_sp_debug -DBUILD_NAME_MOD=full_sp_debug -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Debug -DEKAT_DEFAULT_BFB=True -DSCREAM_DOUBLE_PRECISION=False -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/full_sp_debug" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx/ctest-build/full_sp_debug
OVERALL STATUS: PASS
Starting analysis on weaver with cmd: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -i -m weaver
RUN: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -i -m weaver
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5527/scream/components/eamxx
Completed analysis on weaver'
- [[ 0 != 0 ]]
- [[ 1 == 0 ]]
- [[ weaver == \m\a\p\p\y ]]
- set +x
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash -le
cd $WORKSPACE/${BUILD_ID}/
./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
[SCREAM_PullRequest_Autotester_Weaver] $ /bin/bash -le /tmp/jenkins4434688935044942683.sh
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Finished: SUCCESS
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:
Pull Request Auto Testing STARTING (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5187
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5531
- Status: STARTED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Using Repos:
- Repo: SCREAM (E3SM-Project/scream)
- Branch: mahf708/spa/new-spa-file
- SHA: eb49619b659aeb194ef99a898b87f8bc2bc2993b
- Mode: TEST_REPO
Pull Request Author: mahf708
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED
Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.
Pull Request Auto Testing has FAILED (click to expand)
Build Information
Test Name: SCREAM_PullRequest_Autotester_Mappy
- Build Num: 5187
- Status: FAILED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
Build Information
Test Name: SCREAM_PullRequest_Autotester_Weaver
- Build Num: 5531
- Status: PASSED
Jenkins Parameters
| Parameter Name | Value |
|---|---|
| PR_LABELS | Non-B4B;bugfix;SPA;input file |
| PULLREQUESTNUM | 2786 |
| SCREAM_SOURCE_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_SOURCE_SHA | eb49619b659aeb194ef99a898b87f8bc2bc2993b |
| SCREAM_TARGET_BRANCH | master |
| SCREAM_TARGET_REPO | https://github.com/E3SM-Project/scream |
| SCREAM_TARGET_SHA | a6c73f47ede28cf44f6610a7146c5ecb96f7202f |
| TEST_REPO_ALIAS | SCREAM |
SCREAM_PullRequest_Autotester_Mappy # 5187 FAILED (click to see last 100 lines of console output)
Waiting for tests to finish
DIFF ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_154412_x6406t
DIFF ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240411_154412_x6406t
DIFF ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240411_154412_x6406t
DIFF ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240411_154412_x6406t
DIFF ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240411_154412_x6406t
DIFF PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_154412_x6406t
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240411_154412_x6406t
DIFF SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240411_154412_x6406t
test-scheduler took 796.8708872795105 seconds'
+ errors='Waiting for tests to finish
DIFF ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_154412_x6406t
DIFF ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240411_154412_x6406t
DIFF ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240411_154412_x6406t
DIFF ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240411_154412_x6406t
DIFF ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240411_154412_x6406t
DIFF PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_154412_x6406t
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240411_154412_x6406t
DIFF SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240411_154412_x6406t
test-scheduler took 796.8708872795105 seconds'
+ V1_FAILURES_DETAILS+='Waiting for tests to finish
DIFF ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_154412_x6406t
DIFF ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240411_154412_x6406t
DIFF ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240411_154412_x6406t
DIFF ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240411_154412_x6406t
DIFF ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240411_154412_x6406t
DIFF PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_154412_x6406t
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240411_154412_x6406t
DIFF SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240411_154412_x6406t
test-scheduler took 796.8708872795105 seconds'
+ set +x
######################################################
FAILS DETECTED:
SCREAM V1 TESTING FAILED!
Waiting for tests to finish
DIFF ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_154412_x6406t
DIFF ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240411_154412_x6406t
DIFF ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240411_154412_x6406t
DIFF ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240411_154412_x6406t
DIFF ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240411_154412_x6406t
DIFF PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240411_154412_x6406t
DIFF SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240411_154412_x6406t
DIFF SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 (phase BASELINE)
Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240411_154412_x6406t
test-scheduler took 796.8708872795105 seconds
######################################################
Build step 'Execute shell' marked build as failure
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 1442 killed;
[ssh-agent] Stopped.
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash -le
cd $WORKSPACE/${BUILD_ID}/
./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
We're having issues with some test-launcher job hanging forever. So let's make sure we clean all penting test-launcher jobs
squeue -o"%.7i %u %40j" | grep e3sm-jenkins | grep test-launcher | awk '{ print $1 }' | xargs -r scancel
[SCREAM_PullRequest_Autotester_Mappy] $ /bin/bash -le /tmp/jenkins879865505161742428.sh
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Sending e-mails to: [email protected]
Finished: FAILURE
SCREAM_PullRequest_Autotester_Weaver # 5531 PASSED (click to see last 100 lines of console output)
Start 111: shoc_p3_nudging_glob_novert
111/128 Test #111: shoc_p3_nudging_glob_novert ............................. Passed 1.88 sec
Start 112: homme_shoc_cld_p3_rrtmgp_np1
112/128 Test #112: homme_shoc_cld_p3_rrtmgp_np1 ............................ Passed 7.28 sec
Start 113: homme_shoc_cld_p3_rrtmgp_baseline_cmp
113/128 Test #113: homme_shoc_cld_p3_rrtmgp_baseline_cmp ................... Passed 0.25 sec
Start 114: homme_shoc_cld_p3_rrtmgp_pg2_np1
114/128 Test #114: homme_shoc_cld_p3_rrtmgp_pg2_np1 ........................ Passed 6.46 sec
Start 115: homme_shoc_cld_p3_rrtmgp_pg2_baseline_cmp
115/128 Test #115: homme_shoc_cld_p3_rrtmgp_pg2_baseline_cmp ............... Passed 0.10 sec
Start 116: model_baseline
116/128 Test #116: model_baseline .......................................... Passed 9.39 sec
Start 117: model_initial
117/128 Test #117: model_initial ........................................... Passed 4.60 sec
Start 118: model_restart
118/128 Test #118: model_restart ........................................... Passed 4.99 sec
Start 119: restarted_vs_monolithic_check_np1
119/128 Test #119: restarted_vs_monolithic_check_np1 ....................... Passed 0.08 sec
Start 120: homme_shoc_cld_spa_p3_rrtmgp_np1
120/128 Test #120: homme_shoc_cld_spa_p3_rrtmgp_np1 ........................ Passed 8.91 sec
Start 121: homme_shoc_cld_spa_p3_rrtmgp_baseline_cmp
121/128 Test #121: homme_shoc_cld_spa_p3_rrtmgp_baseline_cmp ............... Passed 0.21 sec
Start 122: homme_shoc_cld_spa_p3_rrtmgp_128levels_np1
122/128 Test #122: homme_shoc_cld_spa_p3_rrtmgp_128levels_np1 .............. Passed 9.92 sec
Start 123: homme_shoc_cld_spa_p3_rrtmgp_128levels_tend_check_np1
123/128 Test #123: homme_shoc_cld_spa_p3_rrtmgp_128levels_tend_check_np1 ... Passed 1.36 sec
Start 124: homme_shoc_cld_spa_p3_rrtmgp_128levels_baseline_cmp
124/128 Test #124: homme_shoc_cld_spa_p3_rrtmgp_128levels_baseline_cmp ..... Passed 0.93 sec
Start 125: homme_shoc_cld_spa_p3_rrtmgp_dp_np1
125/128 Test #125: homme_shoc_cld_spa_p3_rrtmgp_dp_np1 ..................... Passed 21.32 sec
Start 126: homme_shoc_cld_spa_p3_rrtmgp_dp_baseline_cmp
126/128 Test #126: homme_shoc_cld_spa_p3_rrtmgp_dp_baseline_cmp ............ Passed 0.33 sec
Start 127: homme_shoc_cld_p3_mam_optics_rrtmgp_np1
127/128 Test #127: homme_shoc_cld_p3_mam_optics_rrtmgp_np1 ................. Passed 13.58 sec
Start 128: homme_shoc_cld_p3_mam_optics_rrtmgp_baseline_cmp
128/128 Test #128: homme_shoc_cld_p3_mam_optics_rrtmgp_baseline_cmp ........ Passed 0.25 sec
100% tests passed, 0 tests failed out of 128
Label Time Summary:
baseline_cmp = 68.21 secproc (16 tests)
baseline_gen = 170.14 secproc (18 tests)
bfbhash = 1.11 secproc (1 test)
check = 0.81 secproc (1 test)
cld = 14.64 secproc (3 tests)
cld_fraction = 2.99 secproc (1 test)
cxx baseline_cmp = 6.73 secproc (2 tests)
diagnostics = 60.21 secproc (19 tests)
driver = 35.77 secproc (8 tests)
dynamics = 9.26 secproc (3 tests)
fail = 26.07 secproc (4 tests)
io = 54.68 secproc (12 tests)
mam4_optics = 7.06 secproc (1 test)
nudging = 9.37 secproc (2 tests)
p3 = 58.14 secproc (7 tests)
p3_sk = 29.25 secproc (2 tests)
physics = 118.46 secproc (19 tests)
remap = 4.69 secproc (1 test)
rrtmgp = 37.45 secproc (9 tests)
shoc = 18.35 secproc (7 tests)
spa = 12.92 secproc (4 tests)
surface_coupling = 2.25 secproc (1 test)
Total Test time (real) = 548.16 sec
Testing '''81330f077201cda8b553ccea30f2c06a64697baa''' for test '''full_sp_debug'''
RUN: taskset -c 52-103 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/full_sp_debug/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/full_sp_debug -DBUILD_NAME_MOD=full_sp_debug -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Debug -DEKAT_DEFAULT_BFB=True -DSCREAM_DOUBLE_PRECISION=False -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/full_sp_debug" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/full_sp_debug
Testing '''81330f077201cda8b553ccea30f2c06a64697baa''' for test '''full_debug'''
RUN: taskset -c 0-51 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/full_debug/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/full_debug -DBUILD_NAME_MOD=full_debug -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Debug -DEKAT_DEFAULT_BFB=True -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/full_debug" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/full_debug
Testing '''81330f077201cda8b553ccea30f2c06a64697baa''' for test '''release'''
RUN: taskset -c 104-155 sh -c '''SCREAM_BUILD_PARALLEL_LEVEL=52 CTEST_PARALLEL_LEVEL=1 ctest -V --output-on-failure --resource-spec-file /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/release/ctest_resource_file.json -DNO_SUBMIT=True -DBUILD_WORK_DIR=/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/release -DBUILD_NAME_MOD=release -S /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/cmake/ctest_script.cmake -DCTEST_SITE=weaver -DCMAKE_COMMAND="-C /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/cmake/machine-files/weaver.cmake -DNetCDF_Fortran_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-fortran/4.6.0/gcc/11.3.0/openmpi/4.1.4/5ka6asw -DNetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/netcdf-c/4.9.0/gcc/11.3.0/openmpi/4.1.4/mdd6fth -DPnetCDF_C_PATH=/projects/ppc64le-pwr9-rhel8/tpls/parallel-netcdf/1.12.3/gcc/11.3.0/openmpi/4.1.4/52dibdr -DCMAKE_BUILD_TYPE=Release -DEKAT_DISABLE_TPL_WARNINGS='''''''''ON''''''''' -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpifort -DSCREAM_DYNAMICS_DYCORE=HOMME -DSCREAM_TEST_MAX_TOTAL_THREADS=1 -DSCREAM_BASELINES_DIR=/home/projects/e3sm/scream/pr-autotester/master-baselines/weaver/release" '''
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx/ctest-build/release
OVERALL STATUS: PASS
Starting analysis on weaver with cmd: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -i -m weaver
RUN: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -i -m weaver
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5531/scream/components/eamxx
Completed analysis on weaver'
- [[ 0 != 0 ]]
- [[ 1 == 0 ]]
- [[ weaver == \m\a\p\p\y ]]
- set +x
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash -le
cd $WORKSPACE/${BUILD_ID}/
./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
[SCREAM_PullRequest_Autotester_Weaver] $ /bin/bash -le /tmp/jenkins12630636691470734787.sh
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Finished: SUCCESS
I'm retesting just because the last test was nearly a week ago. Once we see the expected DIFFs can we say this is ready to merge? @mahf708?
Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing.