Upgrading to SCORPIO v1.6.3

[jayeshkrishna]

Upgrading to SCORPIO v1.6.3 and adding support for the new rearranger, PIO_REARR_ANY.

SCORPIO v1.6.3 includes the following enhancements/fixes,

• Support for standalone EAMXX/SCREAM builds
• A new rearranger, PIO_REARR_ANY
• Fixes for ADIOS to NetCDF conversion tool hangs/crashes in certain scenarios
• Fixes for ultra high res simulations using the SUBSET rearranger
• Support for reading ADIOS BP files and updates to the BP to NetCDF conversion tool to integrate with CIME/E3SM
• Adding a SCORPIO CMake package during install
• Support for new CDF5 types in the C interface
• Support for pio_set_fill()
• Fix for performance issues while writing large output files (hang during enddef() call)
• Fix for issues writing/reading ADIOS string attributes
• Misc bug fixes

[BFB]

[github-actions[bot]]

PR Preview Action v1.4.7 :---: :rocket: Deployed preview to https://E3SM-Project.github.io/E3SM/pr-preview/pr-6329/ on branch gh-pages at 2024-05-01 17:22 UTC

[jayeshkrishna]

@dqwu noticed that the ERIO test (ERIO.ne30_g16_rx1.A.anlgce_gnu) failed with this PR on machines that support ADIOS.

[dqwu]

Error message: Exception from case_test: can't copy '/scratch/jenkins-slave/workspace/E3SM_DEVELOPER_TESTS/ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm/run/ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.hi.0001-01-12-00000.nc': doesn't exist or not a regular file

Output files:

ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.hi.0001-01-12-00000.nc -> ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.hi.0001-01-12-00000.nc.bp
ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.hi.0001-01-12-00000.nc.bp
ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.hi.0001-01-12-00000.nc.netcdf
ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.hi.0001-01-12-00000.nc.netcdf.netcdf4c
ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.hi.0001-01-12-00000.nc.netcdf.netcdf4p
ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.hi.0001-01-12-00000.nc.netcdf.pnetcdf
ERIO.ne30_g16_rx1.A.anlgce_gnu.C.20240405_004111_ocsvrm.cpl.r.0001-01-07-00000.nc

Comments:

• Not reproducible with SCORPIO v1.5.0 release.
• Only reproducible on machines configured with ADIOS2_ROOT (e.g. GCE, Frontier, Perlmutter).
• Might be related to ADIOS type and the symbolic link foo.nc to foo.nc.bp.

[jayeshkrishna]

The issue with the failing ERIO tests should be fixed now.

The ERIO test failures were not reproducible with earlier versions (<= 1.5.0) of SCORPIO because an output file created with the CLOBBER mode using the ADIOS I/O type was not being rewritten as expected (This version of SCORPIO fixes the issue and hence the error now shows up in the test). With the older versions of SCORPIO (<= 1.5.0) the ADIOS I/O type was not being tested by the ERIO tests.