ufs-srweather-app icon indicating copy to clipboard operation
ufs-srweather-app copied to clipboard

WE2E test "grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v16" fails with segmentation fault at run_fcst step

Open mkavulich opened this issue 1 year ago • 0 comments

Original issue migrated from regional_workflow repository: https://github.com/ufs-community/regional_workflow/issues/731

Expected behavior

WE2E test "grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v16" should succeed.

Current behavior

The failure can manifest in multiple ways: occasionally it gives a segmentation fault with no helpful error messages, other times there is a line FATAL from PE 4: compute_qs: saturation vapor pressure table overflow, nbad= 1 in the output prior to segfault.

Machines affected

This failure has been observed on Hera and Orion, but likely occurs on all platforms.

Steps To Reproduce

  1. On Hera, build the latest UFS_SRW_App and run the grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v16 WE2E test
  2. observe failure at the run_fcst step

Output

The full log file for this failing test can be found on Hera at /scratch2/BMC/det/kavulich/workdir/post-merge_testing/expt_dirs/grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v16/log/run_fcst_2019052000.log

mkavulich avatar Sep 20 '22 23:09 mkavulich