From '/glade/scratch' to '/glade/derecho/scratch'

[ktwu01]

Bug Subject: As NCAR supercomputer users: From '/glade/scratch' to '/glade/derecho/scratch'

Bug Content:

Brief summary of bug

As shown in NCAR HPC Docs, /glade/scratch/username no longer exists. The new scratch path is /glade/derecho/scratch/username. However, there is a hard-coded /glade/scratch/username path in certain CTSM modules, making it difficult to run CTSM successfully.

General bug information

CTSM version you are using: ctsm5.2.010

Does this bug cause significantly incorrect results in the model's science? Yes

Configurations affected: Yes

Important details of your setup / configuration so we can reproduce the bug

Run this on NCAR Derecho/Casper supercomputer.

• Compset: I2000Clm60FatesSpRsGs
• Resolution: f45_g37
• Machine: NCAR Derecho/Casper
• Compiler: Not sure
• XML or namelist changes: None

Important output or errors that show the problem

wukoutian@casper20:~/clm_tutorial_cases/I2000_CTSM_FATESsp> ./case.setup Setting resource.RLIMIT_STACK to -1 from (8388608, -1) job is case.run USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None WALLTIME_FORMAT %H:%M:%S Creating batch scripts Writing case.run script from input template /glade/u/home/wukoutian/CTSM/ccs_config/machines/template.case.run Creating file .case.run Writing case.st_archive script from input template /glade/u/home/wukoutian/CTSM/ccs_config/machines/template.st_archive Creating file case.st_archive Creating user_nl_xxx files for components and cpl ERROR: Could not make directory '/glade/scratch/wukoutian/I2000_CTSM_FATESsp/bld/cpl/obj', error: [Errno 13] Permission denied: '/glade/scratch'

How to fix the bug

For your reference, this bug might have been noticed and partially fixed, as shown in the code search for scratch: https://github.com/search?q=repo%3AESCOMP%2FCTSM%20scratch&type=code

You may also want to also fix all the code related to the retired NCAR Cheyenne supercomputer: https://github.com/search?q=repo%3AESCOMP%2FCTSM%20cheyenne&type=code

When writing commands or scripts, you can use the environment variable $SCRATCH to represent your scratch directory. Here are examples for both a command line and a Python script.

Command Line Example

In a command line script, you can use $SCRATCH directly. For example, if you want to list the contents of your scratch directory, you can use:

ls $SCRATCH

Or, if you want to copy a file to your scratch directory:

cp myfile.txt $SCRATCH/

Python Script Example

In a Python script, you can access the $SCRATCH environment variable using the os module. Here's an example of how you can do this:

import os

# Get the path to the scratch directory from the environment variable
scratch_dir = os.getenv('SCRATCH')

# Use the scratch directory path in your script
if scratch_dir:
    print(f"Scratch directory: {scratch_dir}")

    # Example: creating a file in the scratch directory
    file_path = os.path.join(scratch_dir, 'myfile.txt')
    with open(file_path, 'w') as f:
        f.write("This is a test file in the scratch directory.")

    print(f"File created at: {file_path}")
else:
    print("SCRATCH environment variable is not set.")

This script retrieves the path of the scratch directory from the SCRATCH environment variable and then uses it to create a file in that directory.

By using the $SCRATCH variable, you ensure that your scripts and commands are flexible and can adapt to different environments where the actual path of the scratch directory might vary.

[samsrabin]

Thanks for this detailed report, @ktwu01. I've had a look, and of the parts of a CTSM checkout we control in this repo, it only seems to affect files in tools/contrib/ (plus some commented-out lines in lilac/). Should be an easy fix, but I'm not sure when we'll get to it.

[samsrabin]

I should add that I tried running your example with the latest version (ctsm5.2.026) and was able to do case.setup with no issues.

[ktwu01]

Thank you @samsrabin !

[samsrabin]

I'll just leave this open, as there are some places this does need to be fixed.