NorESMhub
Atmospheric Nudging with NorESM3 spectral element scheme
Nudging winds in NorESM3 would be useful to do shorter runs, eg in a PPEs.
The shift to the dynamical core with spectral elements is suspected to require a different solution then in NorESM 2.3 . Is that true? What needs to be done?
Any experience and suggestions on how to develop this functionality and test this? @IngoBethke @DirkOlivie @Ovewh @gold2718
An example of (the new CAM) nudging configured for both FV and SE can be found in:
<NorESM root>/components/cam/bld/namelist_files/use_cases/hist_trop_strat_nudged_cam6.xml
Is this helpful?
Nudging functionality for SE should be in place already since CESM2.1. When I find the time, I will try to test it.
thanks @IngoBethke, would be great to have an example simulation with beta01 or beta02 documented in https://github.com/NorESMhub/noresm3_dev_simulations/issues
I will give it a try and will report back in a week or so. There are two use cases to test: (1) nudging against output produced by the model itself and (2) nudging against reanalysis e.g. ERA5. The first use case is easy to test as it does not require any interpolation of the nudging input data. The second case is a bit more work. I will test (1) first and then (2).
Hi @IngoBethke what is the status? I would be happy with an example of just nudging against model output. I'm in the process of design a perturbed parameter ensemble (PPE) targeting aerosol processes. I would soon like to do some initial one at the time sensitivity tests of the parameters I'm planning to perturbe in the PPE.
Hi @Ovewh @MichaelSchulzMETNO
U,V nudging in NorESM3 works out of the box, as shown in the verification plot below (I produced some hourly output at a single location).
I performed three NF2000 simulations with noresm3_0_beta02:
NF2000_ne30pg3_ne30pg3_mtn14_default_0001-01-01: reference that produces the nudging data, cold start in 0001-01-01
NF2000_ne30pg3_ne30pg3_mtn14_nudgeUV_0001-01-10: nudged run, cold start in 0001-01-10 (i.e. ten days later than the reference)
NF2000_ne30pg3_ne30pg3_mtn14_default_0001-01-10: free run without nudging, cold start in 0001-01-10 (i.e. ten days later than the reference)
The output and content from case directories can be accessed at https://ns9039k.web.sigma2.no/datalake/ingo/noresm3_nudging/ or locally on NIRD at /nird/datalake/NS9039K/www/ingo/noresm3_nudging
On betzy, the case directories can be accessed at /cluster/projects/nn9039k/people/ingo/noresm3_0_beta02/cases (let me know if you need access to nn9039k, ns9039k)
Important notes:
Some useful info on the nudging implementation is found at https://ncar.github.io/CAM/doc/build/html/users_guide/physics-modifications-via-the-namelist.html#nudging https://noresm-docs.readthedocs.io/en/noresm2.2/Nudging.html
The nudging functionality is always compiled and available. The nudging implementation lives in components/cam/src/physics/cam/nudging.F90 (the old metdata implementation that only works for the FV core is still available components/cam/src/dynamics/fv/metdata.F90).
To use the nudging functionality, the only file that needs to be modified in the case directory is user_nl_cam. An example from my test is https://ns9039k.web.sigma2.no/datalake/ingo/noresm3_nudging/cases/NF2000_ne30pg3_ne30pg3_mtn14_nudgeUV_0001-01-10/user_nl_cam
The nudging implementation "guesses" the file names of the nudging input based on Nudge_Path and Nudge_File_Template specification in the CAM's namelist (there is no file list like in the metdata implementation). Different from the metdata implementation, the time information is given via the file names and there has to be one file per record. In-file metadata is not used, the only thing the matters is that the variable names are correct (i.e. set to "U", "V", "T", "Q", "PS").
If the model does not find a nudging file then it will simple continue without applying nudging. When testing, I recommend to check the info in the atmospheric log file if the file is found and nudging is applied. In addition, you can write extra nudging diagnostics to the diagnostic output (like the "target" U that I plotted as stippled lines).
The effect of the namelist parameters Nudge_Beg_Day, Nudge_Beg_Month, Nudge_Beg_Year, Nudge_End_Day, Nudge_End_Month, Nudge_End_Year is not exactly what I anticipated. If the model date is outside the specified period then the first/last nudging data in that period is applied (i.e. the nudged fields are kept constant). Somewhat questionable functionality if you ask me. Maybe best not using these parameters.
Happy to elaborate if you have any questions.
I will have a look at the ERA5 nudging input preparation when I find the time.
Plotting script at https://ns9039k.web.sigma2.no/datalake/ingo/noresm3_nudging/noresm3_nudging_test.py
I'm struggling to make progress on my own to generate ERA5 forcing data on the spectral element grid. While I have manage to interpolate a single field of ERA5 to the ne30pg3. Using the ESMF_RegridWeightGen to create the weighting file and then loop over all the weights and them multiplying with the ERA data. The looping becomes prohibitively slow at least in python.
This is what i did:
-
- Add longitude and latitude bounds to the ERA5 fields to make them CFGRID compliant
-
- I used a SCRIP formated grid description of the ne30pg3 grid
-
- Generate weights with ESMF_RegridWeightGen:
ESMF_RegridWeightGen -s era5_CFGRID.nc -d ne30pg3_scrip_170611.nc -w era5_to_se_ne30pg3_weights.nc --ignore_unmapped
- Generate weights with ESMF_RegridWeightGen:
-
- Then loop over all the weights:
model_nudged_raw = xr.open_dataset('NF2000_ne30pg3_ne30pg3_mtn14_nudgeUV_0001-01-10.cam.h1i.0001-01-10-43200.nc')
model_nudged = model_nudged_raw[['U','V']].isel(lev=0)
# Creating empty dataset of the same shape as the tested model generated nudging data
out_field = xr.zeros_like(model_nudged).isel(time=0)
era5_one_level = era5_raw.isel(time=0,level=0)
S = weights['S']
row = weights['row']-1
col = weights['col']-1
nlon0=len(era5_one_level['longitude'])
for i in range(0,len(S)) :
out_field['U'][row[i]] = out_field['U'][row[i]] + S[i] * era5_one_level['u'][col[i] // nlon0, col[i] % nlon0]
out_field['V'][row[i]] = out_field['V'][row[i]] + S[i] * era5_one_level['v'][col[i] // nlon0, col[i] % nlon0]
However this is very inefficient and I would presume there be more optimized approaches somewhere? ESMF have ESMF_regrid CLI tool, which according to the documentation should do both calculation of weights and multiplication, but I'm not able to get that working. Something about the grid definition files that not quite right.
There is also a set of old ncl scripts nudging, However i have limited experience with ncl and not sure'll be able to make that work.
Example of one of the interpolated ERA5 windfield at one level, that I manged to produce.
@Ovewh - I think you need to do this in parallel and I think writing a routine in fortran that leverages ESMF and PIO to do this would solve the problem. I recently wrote several modules in the BLOM NUOPC cap to map WOA multi-level initial data to the BLOM grid. This does not map the vertical levels - only the horizontal. If you checkout noresm3_0_beta03a - you can see these routines in the $SRCROOT/components/blom/drivers/nuopc
- ocn_map_woa.F90 (driver)
- ocn_pio_share.F90
- mod_io_input.F90 (maps to the blom grid)
- mod_io_output.F90 (outputs the horizontal mapped data) This uses ESMF and PIO. You should theoretically be able to leverage the code in mod_io_input.F90 and mod_io_output.F90 with your input data - but you will need to build a stand-alone executable to do this. Here is an example of this that @jedwards4b (the author of PIO) pointed me to. https://github.com/NCAR/ParallelIO/blob/main/examples/basic/testpio.F90 I am happy to help with this. Can we meet tomorrow and go over this?
An alternative is to use the xESMF python package (https://xesmf.readthedocs.io/en/stable/) and tell it to read in the weights that you generated with ESMF_RegridWeightGen.
Here is an example https://ns9039k.web.sigma2.no/datalake/ingo/people/fangxing/regrid_ne30.py
I can try to prepare an example specifically for your use case (if I find the time). This method should be very fast.
How about the vertical interpolation? Do you use the cdo python interface for that?
@IngoBethke @Ovewh - one big advantage of using the type of fortran routine I'm proposing is that you never have to generate the mapping files - the weights are generated for you at run time in parallel and can accommodate a wide variety of mappings - including vector mappings of u,v. Furthermore, they are applied in parallel as well. So this can be scaled out very efficiently with processor count and well as memory. I'm trying to prototype this now for @Ovewh - since the routines I pointed out are not quite extensible out of the box to this problem. It would be interesting to compare this to the method you are proposing @IngoBethke. I think it would be valuable to turn this into a more general tool. @DirkOlivie - what method are you using to produce your mappings? Maybe what I am proposing is not really needed?
@mvertens Nice suggestion. Thanks for sharing the code. In the long term, it could be useful to add capability to NorESM3 to perform the interpolation of nudging input online, on-the-fly (though memory constraints and mpi-parallization can pose challenges I guess). Then we would not need to generate and keep multiple pre-interpolated versions of nudging input.
@Ovewh If you can point me to your weights file, scripts and data then I can do some quick testing myself.
I suspect two things that potentially make your script slow:
- the use of "for" loop in your python script. Often the loop performance is related to cache utilization and there are several ways to improve that (see below for suggestions)
- the use of xarray to access your data (something I have encountered in the past). It can be that xarray waits with reading the data into the memory until it is used. In your case, such lazy access would be unfortunate because you access the data element by element in the for loop, potentially resulting in many individual file accesses. A simple test/workaround would be to convert the xarray to a simple numpy array prior to the loop or use the netCDF4 library to read the data into a numpy array
I believe you have a range of options to speed up your interpolation that include
- writing a standalone fortran tool following Mariana's suggestion
- try to use the command line tool ESMF_regrid (using the pre-generated weights) to remap the files
- placing the for-loop in a python function (possibly in a separate file) may allow python to optimize its performance
- replace "for" loop in your code with call to xesmf (as I suggested earlier)
- replace "for" loop in your code with sparse matrix operations (this solution from @matsbn works well for matlab) e.g. using scipy.sparse
- write c or fortran function that replaces the "for loop" and can be called from within you python script
@IngoBethke I have uploaded the grid definition and weights here. I'll try together with Mariana to make the FORTRAN implementation work and then report back to you.
@mvertens Dirk used the same method as me. To me it sound like what you are proposing would be very useful as a general tool.
@Ovewh The xESMF solution seems to work (though I did not plot the result). Please find an example below.
The example requires the xesmf and xarray python libraries. For testing you make these available with source /cluster/projects/nn9039k/people/pgchiu/conda/py3env.sh
Note that the script spends most time on creating the "regridder" object, the actual remapping is fast. So ideally you only want to create regridder object once in your script. Also, it might be best not to loop over the level but to process all levels in one go.
Let me know if this solution works for you and whether it produces the right result.
Example script:
import xesmf as xe
import xarray as xr
import numpy as np
import os
# prepare regridder
dummy = np.zeros([383,288])
dummy_b = np.zeros([384,289])
gridIn = {'lon': dummy, 'lon_b': dummy_b, 'lat': dummy, 'lat_b': dummy_b}
dummy = np.zeros([1,48602])
dummy_b = np.zeros([2,48603])
gridOut = {'lon': dummy, 'lon_b': dummy_b, 'lat': dummy, 'lat_b': dummy_b}
# use this for newer versions of xesmf
# regridder = xe.Regridder(gridIn, gridOut, 'conservative',reuse_weights=True,weights='era5_to_se_ne30pg3_weights.nc')
# use this for older versions of xesmf
os.system("ln -sf era5_to_se_ne30pg3_weights.nc conservative_383x288_1x48602.nc")
regridder = xe.Regridder(gridIn, gridOut, 'conservative',reuse_weights=True)
# read sample data
ds = xr.open_dataset('/cluster/projects/nn9039k/people/ingo/People/Ove/era5_sample_input.nc')
# regrid
ds_ne30 = regridder(ds[['u','v']])
Thanks @IngoBethke, I have been discussing with @mvertens about doing the horizontal interpolation online in the model using the exisiting CEDPS and NUOPC share code within the model. Then we could avoid having duplicate files for different spatial resolutions.
Though this is a nice backup if there too many technical hurdles to make the nudging code compatible with existing regridding tools.
@IngoBethke - just to clarify a bit more. CDEPS ((https://github.com/NorESMhub/CDEPS) has an inline API (https://escomp.github.io/CDEPS/versions/master/html/streams.html#data-model-stream-inline-api) such that data can be read in from multiple files and interpolated both spatially and in time. The regridding weights are generated in parallel at run time and do not require any mapping files. Both the regridding and the IO is done in parallel - so this is very scalable. CAM already leverages this to read in nitrogen deposition forcing data when WACCM does not produce it ($SRCROOT/components/cam/src/cpl/nuopc/atm_stream_ndep.F90).
In looking at the nudging code in CAM it seems that a lot of the functionality of the CDEPS code is duplicated but without any of the advantages (i.e. regridding in parallel). It would be great I think to have nudging capability that does not require you to create a whole set of files (one for each time interval currently). In scanning the code I think this would be fairly straightforward to implement - at least I hope so.
Sounds great. Thanks for the info, Mariana.
Some remarks regarding the nudging input data in /nird/datalake/NS9039K/inputdata/CAM_nudging/met_data/era5/1.25x0.4712:
Using certain versions of HDF5, I receive an error when trying to read the files. If this happens then using a different HDF5 version may fix it.
The "original" ERA5 data is given on a spectral grid. When downloading the ERA5 data from the Copernicus data store, however, I instructed the downloading to perform regridding to regular lonlatgrid on the server side. My intention was to regrid to the coordinates of CAM’s F09 grid. While I did not manage to instruct the downloading to do exactly F09, it worked when I used half the latitude grid spacing of F09 (therefore 1.25x04712).
Despite that the 1.25x0.4712 may be suboptimal for other grids than F09 and F19, I encourage Ove to give it a try and see how it performs. Downloading the spectral ERA5 data at 6-hourly resolution takes several weeks (if not months), uses a lot of storage space, and anyway, one has to interpolate the data to a regular physical grid before being able to remap it to SE grid. In the long term, I would suggest that we download and store nudging input regridded to a 0.25x0.25 regular grid typically used by satellite products.
Any comments/opinions regard the above are welcome.
@Ovewh An example shell-script for the vertical interpolation using cdo:
#!/bin/sh -evx
source /cluster/projects/nn9039k/people/pgchiu/conda/py3env.sh
# create vertical grid file
cat <<EOF> create_vct.py
from netCDF4 import Dataset
fpath= '/nird/datalake/NS9039K/www/ingo/noresm3_nudging/archive/NF2000_ne30pg3_ne30pg3_mtn14_default_0001-01-01/atm/hist/NF2000_ne30pg3_ne30pg3_mtn14_default_0001-01-01.cam.h1i.0001-01-01-21600.nc'
nc = Dataset(fpath,'r')
hyai = nc.variables['hyai'][:]
hybi = nc.variables['hybi'][:]
nc.close()
for i in range(len(hyai)):
print('{:5d} {:25.17f} {:25.17f}'.format(i,hyai[i]*100000,hybi[i]))
EOF
python create_vct.py > vct_L58.txt
# prepare merged input file (pick only first record of file, exclude t and level variables)
ncks -O -C -x -v t,level -d time,0,0 -o in.nc /nird/datalake/NS9039K/inputdata/CAM_nudging/met_data/era5/1.25x0.4712/era5_UVT_1.25x0.4712_6hr_r1_202001.nc
ncks -A -v sp -d time,0,0 -o in.nc /nird/datalake/NS9039K/inputdata/CAM_nudging/met_data/era5/1.25x0.4712/era5_PS_1.25x0.4712_6hr_r0_202001.nc
ncrename -d level,lev in.nc
ncks -A -v lev,hyai,hybi,hyam,hybm -o in.nc /nird/datalake/NS9039K/inputdata/CAM_nudging/met_data/auxillary/grid_era5_1.25x0.4712.nc
# vertical remapping
cdo -O remapeta,vct_L58.txt in.nc out.nc
An easy alternative to using cdo for the vertical remapping is to use NCL's hyi2hyo command (https://www.ncl.ucar.edu/Document/Functions/Built-in/hyi2hyo.shtml). The command should be available in python through the PyNGL python interface. I will test it and prepare an example.
UPDATE: PyNGL does unfortunately not feature the NCL command hyi2hyo. So, one would need to use ncl directly.
@IngoBethke I have now made a 1 year NorESM3 simulation with Nudged ERA5 (2017) simulation using the branch of @mvertens mvertens:feature/refactor_nudging . Model vs obs (climatatology) : https://ns2345k.web.sigma2.no/datalake/diagnostics/ADF/ovewh/NFHISTfsst_ne16pg3_ne16pg3_mtn14_ne16pg3_ne16pg3_mtn.NFLTHIST.Nudging.PPE_Baseline_20251106_2017_2017_vs_Obs/website/html_img/plot_page_V_ANN_Meridional_Mean.html
CaseDirectory: /nird/datalake/NS2345K/ovewh/PPE-BASELINE/NFHISTfsst_ne16pg3_ne16pg3_mtn14_ne16pg3_ne16pg3_mtn.NFLTHIST.Nudging.PPE_Baseline_20251106
It looks like the vertical profile of the temperature field is inverted. Do you nudge also temperature, or only winds? Or are the vertical coordinates inverted in the nudging fields?
@Ovewh The result for the zonal mean zonal wind could hint that there is something wrong with the vertical interpolation as the extratropical jet extends way too far down. Maybe that can cause the alternate temperature patterns close to the surface. https://ns2345k.web.sigma2.no/datalake/diagnostics/ADF/ovewh/NFHISTfsst_ne16pg3_ne16pg3_mtn14_ne16pg3_ne16pg3_mtn.NFLTHIST.Nudging.PPE_Baseline_20251106_2017_2017_vs_Obs/website/html_img/plot_page_U_ANN_Zonal_Mean.html
I will have try to have a look at the cdo-based vertical interpolation, maybe there is something wrong with the ancillary files that I provided to you or that cdo got the vertical units wrong.
Also, I have started work on updating my ncl-script for preparing nudging input so that it can be used for SE grids (using the ncl functions to perform the esmf regridding). Once ready, I will share it.
@IngoBethke - with the branch I have provided @Ovewh - we no longer need to do any horizontal interpolation on the input nudging fields and we no longer require to have one file per time sample of nudging. The code does not do any vertical interpolation at this point - so @Ovewh provided me with fields that were vertically interpolated. I can add vertical interpolation as well - but we decided to hold off at this point and validate what is in the draft PR now (see https://github.com/NorESMhub/CAM/pull/243) This type of horizontal interpolation will work for any time of model grid - including regionally refined grids and will require much less preprocessing of input files. I am happy to have a chat about this and point you to the branch if you want to give it a try.
@mvertens That's great news. I am looking forward to using that new capability.
In any case, I will try to verify Ove's vertical offline-interpolation. I kind of hope that the problem is there.
@IngoBethke
Here is the code i used for the offline vertical interpolation:
#!/bin/sh -evx
source /conda/miniconda3/etc/profile.d/conda.sh
conda activate
conda activate /diagnostics/esmvaltool-env/2.12.0/
# create vertical grid file
cat <<EOF> create_vct.py
from netCDF4 import Dataset
fpath= '/nird/datalake/NS9039K/www/ingo/noresm3_nudging/archive/NF2000_ne30pg3_ne30pg3_mtn14_default_0001-01-01/atm/hist/NF2000_ne30pg3_ne30pg3_mtn14_default_0001-01-01.cam.h1i.0001-01-01-21600.nc'
nc = Dataset(fpath,'r')
hyai = nc.variables['hyai'][:]
hybi = nc.variables['hybi'][:]
nc.close()
for i in range(len(hyai)):
print('{:5d} {:25.17f} {:25.17f}'.format(i,hyai[i]*100000,hybi[i]))
EOF
python create_vct.py > vct_L58.txt
# loop over all files for 2016 and vertically remap them
for month in $(seq -w 1 12); do
# prepare merged input file (pick only first record of file, exclude t and level variables)
ncks -O -C -x -v level,t -o in.nc /nird/datalake/NS9039K/inputdata/CAM_nudging/met_data/era5/1.25x0.4712/era5_UVT_1.25x0.4712_6hr_r0_2016${month}.nc
ncks -A -v sp -o in.nc /nird/datalake/NS9039K/inputdata/CAM_nudging/met_data/era5/1.25x0.4712/era5_PS_1.25x0.4712_6hr_r0_2016${month}.nc
ncrename -d level,lev -v u,U -v v,V -v sp,PS in.nc
ncks -A -v lev,hyai,hybi,hyam,hybm -o in.nc /nird/datalake/NS9039K/inputdata/CAM_nudging/met_data/auxillary/grid_era5_1.25x0.4712.nc
# vertical remapping
mkdir -p /nird/datalake/NS2345K/ovewh/era5_UVPS_58levels_2016
cdo -O remapeta,vct_L58.txt in.nc out.nc
ncap2 -O -s 'T[$time,$lev,$latitude,$longitude]=0.0f' -s 'Q[$time,$lev,$latitude,$longitude]=0.0f' out.nc /nird/datalake/NS2345K/ovewh/era5_UVPS_58levels_2016/era5_UVPS_58levels_2016${month}.nc
done
@MichaelSchulzMETNO I nudge U and V. I used the CMIP7 forcings which has not been validated by dirk yet, so there might be some issues there aswell.
My build script
#!/bin/bash
paths_noresm="/cluster/projects/nn2345k/ovewh/"
perror(){
if [ $1 -ne 0 ]; then
echo "ERROR: $2"
exit $1
fi
}
export PATH=${paths_noresm}/NorESM3/cime/scripts:${PATH}
nyears=13
resubmit=0
project="nn2345k"
runStartDate="2016-12-01"
res="ne16pg3_ne16pg3_mtn14"
compset="NFLTHIST"
wall_clock_time="24:00:00"
queue="normal"
tag="ne16pg3_ne16pg3_mtn.NFLTHIST.Nudging.PPE_Baseline"
case_dir="/cluster/projects/nn2345k/ovewh/Nudged_baseline/cases"
# set up the directory structure
mkdir -p ${case_dir}
current_date=$(date +%Y%m%d)
setup_case() {
case_name=$1
echo "Creating case: ${case_name}"
echo "create_newcase --mach betzy --case \"${case_dir}/${case_name}\" --compset \"${compset}\" --res \"${res}\" --project \"${project}\" --compiler \"intel\" --driver nuopc --run-unsupported"
create_newcase --mach betzy --case "${case_dir}/${case_name}" --compset "${compset}" --res "${res}" --project "${project}" --compiler "intel" --driver nuopc --run-unsupported
perror $? "Problem with creating new case"
}
case_name="NFHISTfsst_${res}_${tag}_${current_date}"
if [ -e "${case_dir}/${case_name}" ]; then
echo "${case_name} already exists, skipping"
continue
fi
echo "Setting up case: ${case_name}"
setup_case "${case_name}"
cd ${case_dir}/${case_name}
./xmlchange SSTICE_DATA_FILENAME="/cluster/shared/noresm/inputdata/atm/cam/sst/sst_HadOIBl_bc_1.9x2.5_1850_2021_c120422.nc"
./xmlchange NTASKS=-8
./xmlchange STOP_OPTION="nmonths"
./xmlchange RESUBMIT="${resubmit}"
./xmlchange --subgroup case.st_archive JOB_WALLCLOCK_TIME='04:00:00'
./xmlchange STOP_N="${nyears}"
./xmlchange RUN_STARTDATE="${runStartDate}"
./xmlchange SSTICE_YEAR_END="2021"
./xmlchange RUN_TYPE=startup
./xmlchange CALENDAR=GREGORIAN
./xmlchange JOB_WALLCLOCK_TIME="${wall_clock_time}" --subgroup case.run
./xmlchange JOB_QUEUE="${queue}" --subgroup case.run
./xmlchange --subgroup case.compress JOB_WALLCLOCK_TIME=3:00:00
./case.setup
perror $? "Problem with case.setup."
echo "Setting up user namelists"
cat > user_nl_cam << EOF
! Users should add all user specific namelist changes below in the form of
! namelist_var = new_namelist_value
Nudge_model = .true.
Nudge_Filenames = 'era5_UVPS_58levels_201611.nc', 'era5_UVPS_58levels_201612.nc', 'era5_UVPS_58levels_201701.nc', 'era5_UVPS_58levels_201702.nc', 'era5_UVPS_58levels_201703.nc', 'era5_UVPS_58levels_201704.nc', 'era5_UVPS_58levels_201705.nc', 'era5_UVPS_58levels_201706.nc', 'era5_UVPS_58levels_201707.nc', 'era5_UVPS_58levels_201708.nc', 'era5_UVPS_58levels_201709.nc', 'era5_UVPS_58levels_201710.nc', 'era5_UVPS_58levels_201711.nc', 'era5_UVPS_58levels_201712.nc'
Nudge_Datapath = '/cluster/shared/noresm/inputdata/noresm-only/inputForNudging/era5_UVPS_58levels_2016-2017/'
Nudge_Meshfile = '/cluster/shared/noresm/inputdata/noresm-only/inputForNudging/era5_UVPS_ESMF_Mesh_cdf5.nc'
Nudge_beg_day = 1
Nudge_beg_month = 1
Nudge_beg_year = 2016
Nudge_end_day = 31
Nudge_end_month = 12
Nudge_end_year = 2017
Nudge_file_times_per_day = 4
Model_update_times_per_day = 48
Nudge_Force_Opt = 1
Nudge_Uprof = 1
Nudge_Ucoef = 1.0
Nudge_Vprof = 1
Nudge_Vcoef = 1.0
Nudge_Tprof = 0
Nudge_Tcoef = 0.0
Nudge_PSprof = 0
Nudge_PScoef = 0.0
interpolate_nlat = 96
interpolate_nlon = 144
interpolate_output = .true.
nhtfrq = -1,-1
mfilt = 1,1
ndens = 2,2
interpolate_nlat = 96,96
interpolate_nlon = 144,144
mfilt = 1,120
ndens = 2,2
nhtfrq = 0,-3
fincl2 = 'Nudge_U','Nudge_V','Target_U','Target_V',
interpolate_output = .true., .true.
prescribed_ozone_file = "ozone_strataero_WACCM_L70_zm5day_18500101-21010201_CMIP6histEnsAvg_SSP245_c190403.nc"
flbc_file = '/cluster/shared/noresm/inputdata/atm/waccm/lb/LBC_17500116-25001216_CMIP6_SSP585_0p5degLat_h2-ch4-lbc-hyway_c20200824.nc'
use_aerocom = .true.
dust_emis_fact = 6.1D0
rafsip_on = .true.
&chem_inparm
ext_frc_specifier = 'H2O -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/elev/H2OemissionCH4oxidationx2_3D_L70_1849-2101_CMIP6ensAvg_SSP2-4.5_c190403.nc',
'BC_AX -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_BC_AX_airALL_vertical_1995-2025_1.9x2.5_version20250620.nc',
'BC_AX -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_BC_AX_anthroprofENEIND_vertical_1995-2025_1.9x2.5_version20250620.nc',
'BC_N -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_BC_N_airALL_vertical_1995-2025_1.9x2.5_version20250620.nc',
'BC_N -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_BC_N_anthroprofENEIND_vertical_1995-2025_1.9x2.5_version20250620.nc',
'BC_NI -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_BC_NI_bbAGRIBORFDEFOPEATSAVATEMF_vertical_1995-2025_1.9x2.5_version20250620.nc',
'OM_NI -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_OM_NI_airALL_vertical_1995-2025_1.9x2.5_version20250620.nc',
'OM_NI -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_OM_NI_anthroprofENEIND_vertical_1995-2025_1.9x2.5_version20250620.nc',
'OM_NI -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_OM_NI_bbAGRIBORFDEFOPEATSAVATEMF_vertical_1995-2025_1.9x2.5_version20250620.nc',
'SO2 -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO2_airALL_vertical_1995-2025_1.9x2.5_version20250620.nc',
'SO2 -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO2_anthroprofENEIND_vertical_1995-2025_1.9x2.5_version20250620.nc',
'SO2 -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO2_bbAGRIBORFDEFOPEATSAVATEMF_vertical_1995-2025_1.9x2.5_version20250620.nc',
'SO2 -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO2_volcCONTEXPL_vertical_1995-2025_1.9x2.5_version20250620.nc',
'SO4_PR -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO4_PR_airALL_vertical_1995-2025_1.9x2.5_version20250620.nc',
'SO4_PR -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO4_PR_anthroprofENEIND_vertical_1995-2025_1.9x2.5_version20250620.nc',
'SO4_PR -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO4_PR_bbAGRIBORFDEFOPEATSAVATEMF_vertical_1995-2025_1.9x2.5_version20250620.nc',
'SO4_PR -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO4_PR_volcCONTEXPL_vertical_1995-2025_1.9x2.5_version20250620.nc'
srf_emis_specifier = 'BC_AX -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_BC_AX_anthrosurfAGRTRADOMSOLWSTSHP_surface_1995-2025_1.9x2.5_version20250620.nc',
'BC_N -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_BC_N_anthrosurfAGRTRADOMSOLWSTSHP_surface_1995-2025_1.9x2.5_version20250620.nc',
'OM_NI -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_OM_NI_anthrosurfAGRTRADOMSOLWSTSHP_surface_1995-2025_1.9x2.5_version20250620.nc',
'SO2 -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO2_anthrosurfAGRTRADOMSOLWSTSHP_surface_1995-2025_1.9x2.5_version20250620.nc',
'SO4_PR -> /cluster/shared/noresm/inputdata/atm/cam/chem/emis/cmip7_emissions_version20250620/emissions_cmip7_noresm3_SO4_PR_anthrosurfAGRTRADOMSOLWSTSHP_surface_1995-2025_1.9x2.5_version20250620.nc'
tracer_cnst_file = "tracer_cnst_halons_3D_L70_1849-2101_CMIP6ensAvg_SSP2-4.5_c190403.nc"
/
ubc_file_input_type = 'CYCLICAL'
ubc_file_cycle_yr = 2010
ubc_file_path = '/cluster/shared/noresm/inputdata/atm/cam/chem/ubc/b.e21.BWHIST.f09_g17.CMIP6-historical-WACCM.ensAvg123.cam.h0zm.H2O.185001-201412_c230509cdf5.nc'
EOF
cd ${case_dir}/${case_name}
./case.build
perror $? "Problem with case.build."
@Ovewh Thanks for the code for offline vertical interpolation. I will try to have a closer look at it and do some tests when I find the time.
Concerning the line
Nudge_Filenames = 'era5_UVPS_58levels_201611.nc', 'era5_UVPS_58levels_201612.nc', 'era5_UVPS_58levels_201701.nc', 'era5_UVPS_58levels_201702.nc', 'era5_UVPS_58levels_201703.nc', 'era5_UVPS_58levels_201704.nc', 'era5_UVPS_58levels_201705.nc', 'era5_UVPS_58levels_201706.nc', 'era5_UVPS_58levels_201707.nc', 'era5_UVPS_58levels_201708.nc', 'era5_UVPS_58levels_201709.nc', 'era5_UVPS_58levels_201710.nc', 'era5_UVPS_58levels_201711.nc', 'era5_UVPS_58levels_201712.nc'
Have you verified that CAM picks the right file and record? I am just asking because I am not familiar with this functionality. So far, I always created one input file per record where the file name uniquely identifies the associated time.
Have you verified that CAM picks the right file and record? I am just asking because I am not familiar with this functionality. So far, I always created one input file per record where the file name uniquely identifies the associated time.
@IngoBethke Yes, when @mvertens refactored the code, we changed how we provide the file names. I believe it use the information in the file to determine the date, instead of relying on the filename.
I looked at the log file from my run and model seems to move seamlessly to next file in the list after the previous file ended.
WSHIST: writing time sample 12 to instantaneous h-file 2 DATE=2017/01/31 NCSEC= 54000
WSHIST: writing time sample 12 to accumulated h-file 2 DATE=2017/01/31 NCSEC= 48600
nstep, te 2959 0.26223455435106368E+10 0.26223445971088161E+10 -0.52465247000023380E-04 0.98477598543494561E+05 0.20548077300190923E+03
nstep, te 2960 0.26223588724626837E+10 0.26223575048617616E+10 -0.75815044294024318E-04 0.98477631719174475E+05 0.20548077300190923E+03
nstep, te 2961 0.26223659199232798E+10 0.26223654364464169E+10 -0.26802271580290102E-04 0.98477654370212287E+05 0.20548077300190923E+03
nstep, te 2962 0.26223743953345656E+10 0.26223738841664886E+10 -0.28337368870663632E-04 0.98477678887766073E+05 0.20548077300190923E+03
nstep, te 2963 0.26223782793764606E+10 0.26223784783659797E+10 0.11031280920033593E-04 0.98477691959022472E+05 0.20548077300190923E+03
(shr_strdata_readstrm) close : /cluster/shared/noresm/inputdata/noresm-only/inputForNudging/era5_UVPS_58levels_2016-2017/era5_UVPS_58levels_201701.nc
(shr_strdata_readstrm) opening : /cluster/shared/noresm/inputdata/noresm-only/inputForNudging/era5_UVPS_58levels_2016-2017/era5_UVPS_58levels_201702.nc
(shr_strdata_readstrm) reading file ub: /cluster/shared/noresm/inputdata/noresm-only/inputForNudging/era5_UVPS_58levels_2016-2017/era5_UVPS_58levels_201702.nc 1
(shr_strdata_readstrm) close : /cluster/shared/noresm/inputdata/noresm-only/inputForNudging/era5_UVPS_58levels_2016-2017/era5_UVPS_58levels_201701.nc
(shr_strdata_readstrm) opening : /cluster/shared/noresm/inputdata/noresm-only/inputForNudging/era5_UVPS_58levels_2016-2017/era5_UVPS_58levels_201702.nc
(shr_strdata_readstrm) reading file ub: /cluster/shared/noresm/inputdata/noresm-only/inputForNudging/era5_UVPS_58levels_2016-2017/era5_UVPS_58levels_201702.nc 1
nstep, te 2964 0.26223833587633405E+10 0.26223832177010012E+10 -0.78200000213207693E-05 0.98477707644008900E+05 0.20548077300190923E+03
WSHIST: writing time sample 13 to instantaneous h-file 2 DATE=2017/01/31 NCSEC= 64800
WSHIST: writing time sample 13 to accumulated h-file 2 DATE=2017/01/31 NCSEC= 59400