ufs-weather-model icon indicating copy to clipboard operation
ufs-weather-model copied to clipboard

Error running rrfs_v1nssl_intel with -DDEBUG=ON using ufs-community:develop

Open zach1221 opened this issue 1 year ago • 10 comments

Description

Experience failure when attempting to run rrfs_v1nssl_intel on Hera, with -DDEBUG=ON, against ufs-community:develop branch.

To Reproduce:

  1. log into Hera.
  2. Clone ufs-community:dev branch
  3. edit rt.conf to add -DDEBUG=ON to compile line of case rrfs_v1nssl_intel
  4. Run test

Additional context

Example from Denise's attempt /scratch1/NCEPDEV/stmp2/Denise.Worthen/FV3_RT/rt_301756/rrfs_v1nssl_intel image

That line is for bdsnow below

image

Output

zach1221 avatar Oct 16 '23 20:10 zach1221

Clarification...I think your output above showing the failure in module_mp_nssl_2mom.F90 is the failure that @BrianCurtis-NOAA got w/ the PR #1915 branch on Acorn in non-debug mode. The failure I posted was for debug mode at develop 68050e5

DeniseWorthen avatar Oct 16 '23 20:10 DeniseWorthen

Running debug with gnu on Hercules. Some failures, but mostly due to timeouts. rrfs_v1beta_gnu seemed to be the only one to fail from another cause. I'll re-test the others that failed with an extended wall clock. working dir = /work2/noaa/stmp/zshrader/stmp/zshrader/FV3_RT/rt_2728437/
Logs: /work/noaa/nems/zshrader/hercules/rt-1915/tests/logs/RegressionTests_hercules.log

zach1221 avatar Oct 16 '23 21:10 zach1221

Clarification...I think your output above showing the failure in module_mp_nssl_2mom.F90 is the failure that @BrianCurtis-NOAA got w/ the PR #1915 branch on Acorn in non-debug mode. The failure I posted was for debug mode at develop 68050e5

Yes, that's true. I'll remove it.

zach1221 avatar Oct 16 '23 21:10 zach1221

@zach1221 I don't see the rrfs_v1nssl test in your run directory.

DeniseWorthen avatar Oct 16 '23 21:10 DeniseWorthen

@zach1221 I don't see the rrfs_v1nssl test in your run directory.

@DeniseWorthen it doesn't look like rrfs_v1nssl is part of the gnu tests in rt.conf actually. I can try to compile it with gnu.

zach1221 avatar Oct 17 '23 12:10 zach1221

It fails with gnu. Screenshot of the err log but I'm not sure it's helpful. image /work2/noaa/stmp/zshrader/stmp/zshrader/FV3_RT/rt_2824213/rrfs_v1nssl_gnu

zach1221 avatar Oct 17 '23 13:10 zach1221

It fails with gnu. Screenshot of the err log but I'm not sure it's helpful. image /work2/noaa/stmp/zshrader/stmp/zshrader/FV3_RT/rt_2824213/rrfs_v1nssl_gnu

Can't see it well in the SS, but it looks like the glacier file where we see the issue with the intel compiler?

BrianCurtis-NOAA avatar Oct 17 '23 13:10 BrianCurtis-NOAA

It fails with gnu. Screenshot of the err log but I'm not sure it's helpful. image /work2/noaa/stmp/zshrader/stmp/zshrader/FV3_RT/rt_2824213/rrfs_v1nssl_gnu

Can't see it well in the SS, but it looks like the glacier file where we see the issue with the intel compiler?

Yes, sorry I cut it off just above the glacier file portion I think. image

zach1221 avatar Oct 17 '23 13:10 zach1221

pointing to the same bdsnow line in module_sf_noahmp_glacier.F90 , that Denise posted previously.

zach1221 avatar Oct 17 '23 13:10 zach1221

Testing this issue again against ufs-wm:develop.

zach1221 avatar Oct 23 '24 16:10 zach1221