HEMCO ERROR: ModelLev_Interpolate was called but MESSy should have been used

[jinlx]

Name and Institution (Required)

Name: Lixu Jin Institution: The University of Montana

Description of your issue or question

Hi there,

I was trying to run the GC14.2.3 nested simultions out of box. I The only thing I editted is turned NEI for the US anthropogenic emissions. However, here goes the error infomartion related to vertical regridding (turning off NEI works well in the nested run).

HEMCO: Opening /projects/atmoschem/geos-chem/ExtData/HEMCO/AnnualScalar/v2014-07/AnnualScalar.geos.1x1.nc
HEMCO: Opening /projects/atmoschem/geos-chem/ExtData/HEMCO/NEI2016/v2021-06/2016fh_16j_afdust_adj_0pt1degree_month_06.ncf

HEMCO ERROR:  ModelLev_Interpolate was called but MESSy should have been used: EPA16_BCPI__afdustPEC

HEMCO ERROR: ERROR 0
 --> LOCATION: ModelLev_Interpolate (hco_interp_mod.F90)

HEMCO ERROR: ERROR 16
 --> LOCATION: HCOIO_READ (HCOIO_READ_STD_MOD.F90)

HEMCO ERROR: Error encountered in routine HCOIO_Read!

HEMCO ERROR: Error in HCOIO_DATAREAD called from HEMCO ReadList_Fill: EPA16_BCPI__afdustPEC
 --> LOCATION: ReadList_Fill (HCO_ReadList_Mod.F90)

HEMCO ERROR: Error in ReadList_Fill (3) called from HEMCO ReadList_Read
 --> LOCATION: ReadList_Read (HCO_ReadList_Mod.F90)
 Error in ReadList_Read called from hco_run
===============================================================================
GEOS-Chem ERROR: Error encountered in "HCO_Run"!
 -> at HCOI_GC_Run (in module GeosCore/hco_interface_gc_mod.F90)

THIS ERROR ORIGINATED IN HEMCO!  Please check the HEMCO log file for
additional error messages!

I was trying to search through the github and only found this #1904 . I was wondering if the error caused by this edits or any other potential settings.

[nicholasbalasus]

Hi @jinlx - this is fixed in GEOS-Chem 14.3.1 / HEMCO 3.8.1. There is some discussion here.

[jinlx]

Thanks @nicholasbalasus ! I should have noticed the discussion earlier, but changing both the HEMCO_Config and src codes fixes the bug through my test. I will go ahead and close the issues for now.

[jinlx]

Sorry for openning it again. It seems that the GC get killed within 1-hour running when NEI is turned on. Below goes the error information. I have tried increasing the mem from 100G to 300G but it does not help. Any insights that we can fix the bug?

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x14b7ce5e1b4f in ???
#1  0x4a8638 in __hco_utilities_gc_mod_MOD_get_boundary_conditions
        at /projects/atmoschem/lixujin/GC_output/GC14.2.3/GEOSFP_025_MSO_NEI/CodeDir/src/GEOS-Chem/GeosCore/hco_utilities_gc_mod.F90:2561
#2  0x48e83a in __hco_interface_gc_mod_MOD_hcoi_gc_run
        at /projects/atmoschem/lixujin/GC_output/GC14.2.3/GEOSFP_025_MSO_NEI/CodeDir/src/GEOS-Chem/GeosCore/hco_interface_gc_mod.F90:1055
#3  0x4529c9 in __emissions_mod_MOD_emissions_run
        at /projects/atmoschem/lixujin/GC_output/GC14.2.3/GEOSFP_025_MSO_NEI/CodeDir/src/GEOS-Chem/GeosCore/emissions_mod.F90:185
#4  0x406ac8 in geos_chem
        at /projects/atmoschem/lixujin/GC_output/GC14.2.3/GEOSFP_025_MSO_NEI/CodeDir/src/GEOS-Chem/Interfaces/GCClassic/main.F90:948
#5  0x405496 in main
        at /projects/atmoschem/lixujin/GC_output/GC14.2.3/GEOSFP_025_MSO_NEI/CodeDir/src/GEOS-Chem/Interfaces/GCClassic/main.F90:32
/var/spool/slurmd/job625458/slurm_script: line 34: 2129976 Segmentation fault      (core dumped) ./gcclassic > GC.log

[nicholasbalasus]

@yantosca @lizziel @msulprizio might have better advice, but there are some guidelines for that kind of error here.

I also see that you might be using a modified version of 14.2.3. It could be worth it to start fresh with the current release (14.3.1), which should work out of the box for NEI with a script like this. I was able to run a month-long simulation.

#!/bin/bash

# Source env file
source /n/home06/nbalasus/envs/gcclassic.rocky+gnu10.minimal.env

# Create run dir
dir="/n/holyscratch01/jacob_lab/nbalasus/test-nei/"
mkdir -p "${dir}"
cd "${dir}"
git clone https://github.com/geoschem/GCClassic.git
cd GCClassic
git checkout 14.3.1
git submodule update --init --recursive
cd run

# Compile GC for Full-Chem
run_dir="gc_run"
c="1\n1\n1\n1\n1\n${dir}\n${run_dir}\nn\n"
printf ${c} | ./createRunDir.sh
cd "${dir}/${run_dir}/build"
cmake ../CodeDir -DRUNDIR=.. -DMECH=fullchem
make -j
make install
cd "${dir}/${run_dir}"

# Modify config files
sed -i -e "s|NEI2016_MONMEAN        :       false|NEI2016_MONMEAN        :       true|g" HEMCO_Config.rc

# Modify and submit run script
cp runScriptSamples/operational_examples/harvard_cannon/geoschem.run .
sed -i -e "s|-c 8|-c 24|g" \
    -e "s|15000|64000|g" geoschem.run
sbatch geoschem.run

[jinlx]

Thanks Nicholas! Sounds like using the latest version would be the quick fix! I will go ahead running the model with GC14.3.1 and update the status when I am there.

[yantosca]

This is a duplicate issue of https://github.com/geoschem/HEMCO/issues/264.

[yantosca]

Closing out this issue