Failure with iris.Constraint and extract

[catherinehardacre]

🐛 Bug Report

iris.Constraint failing when constraining on 'year' coordinate

Code snippet: season_constraint = iris.Constraint(clim_season=season) year_constraint = iris.Constraint(season_year=year) season_data = obs_cube[:, istn].extract(season_constraint & year_constraint)

How To Reproduce

Steps to reproduce the behaviour:

1. Check out ESMValTool branch "aeronet_aod_clim_new_diagnostic" at commit 1382e36
2. Note that a specific package is needed for this diagnostic: $ mamba install pys2index
3. Run esmval tool: $ esmvaltool run /path/to/recipe_aod_aeronet_assess.yml

Expected behaviour

A cube constrained on climate season and year should be returned

Screenshots

log output from the diagnostic script:

Traceback (most recent call last): File "/home/users/chardacr/ESMValTool/esmvaltool/diag_scripts/aerosols/aod_aeronet_assess.py", line 486, in main(CONFIG) File "/home/users/chardacr/ESMValTool/esmvaltool/diag_scripts/aerosols/aod_aeronet_assess.py", line 439, in main obs_cube = preprocess_aod_observational_dataset(obs_dataset) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/users/chardacr/ESMValTool/esmvaltool/diag_scripts/aerosols/aod_aeronet_assess.py", line 345, in preprocess_aod_observational_dataset season_data = obs_cube[:, istn].extract( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/users/chardacr/miniforge3/envs/esmvaltool_aod/lib/python3.11/site-packages/iris/cube.py", line 2765, in extract return constraint.extract(self) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/users/chardacr/miniforge3/envs/esmvaltool_aod/lib/python3.11/site-packages/iris/_constraints.py", line 199, in extract result = cube[slice_tuple] ~~~~^^^^^^^^^^^^^ File "/home/users/chardacr/miniforge3/envs/esmvaltool_aod/lib/python3.11/site-packages/iris/cube.py", line 2693, in getitem dims = self.ancillary_variable_dims(ancvar) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/users/chardacr/miniforge3/envs/esmvaltool_aod/lib/python3.11/site-packages/iris/cube.py", line 1654, in ancillary_variable_dims ancillary_variable = self.ancillary_variable(ancillary_variable) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/users/chardacr/miniforge3/envs/esmvaltool_aod/lib/python3.11/site-packages/iris/cube.py", line 2342, in ancillary_variable raise iris.exceptions.AncillaryVariableNotFoundError(emsg) iris.exceptions.AncillaryVariableNotFoundError: "Expected to find exactly 1 ancillary_variable matching the given 'Number of days' ancillary_variable's metadata, but found none."

Environment

• OS & Version: [e.g., Ubuntu 20.04 LTS]
• Iris Version: [e.g., From the command line run python -c "import iris; print(iris.__version__)"] Iris version: 3.7.0 :q

Additional context

This problem occurred in my ESMValTool environment which I am running on JASMIN. The iris.Constraint and extract steps worked when tested outside of ESMValTool using the same input NetCDF file.

Click to expand this section...

On JASMIN the following works (aod_aeronet_assess_local_test.py is below) $ module load jaspy $ python aod_aeronet_assess_local_test.py

import iris
import iris.plot as iplt
import iris.coord_categorisation
import matplotlib.cm as mpl_cm
import matplotlib.lines as mlines
import matplotlib.pyplot as plt
import numpy as np
import scipy
from aero_utils import AeroAnsError, add_bounds, extract_pt
from matplotlib import colors, gridspec
from numpy import ma

import pdb

fontsizedict = {"title": 25, "axis": 20, "legend": 18, "ticklabel": 18}

# Load obs data
dir_obs =/gws/nopw/j04/esmeval/example_data/esmvaltool/OBS6_AERONET_atmos_20231021_AERmon_od440aer_1994-2014.nc
filename_obs = dir_obs + "OBS6_AERONET_atmos_20231021_AERmon_od440aer_1994-2014.nc"
obs_cube = iris.load_cube(filename_obs)

# Add the clim_season and season_year coordinates.
#iris.coord_categorisation.add_year(obs_cube, 'time', name='year')

iris.coord_categorisation.add_season(
    obs_cube, 'time', name='clim_season')

iris.coord_categorisation.add_season_year(
    obs_cube, 'time', name='season_year')

# Aggregate by seasonal and climate-year
annual_seasonal_mean = obs_cube.aggregated_by(['clim_season', 'season_year'],
                                              iris.analysis.MEAN)

multi_annual_seasonal_mean = annual_seasonal_mean.aggregated_by('clim_season',
                                              iris.analysis.MEAN)

# Loop over stations
#for istn, station in enumerate(obs_cube.slices_over('platform_name')):
platform_names = obs_cube.coord('platform_name').points

for istn, station in enumerate(platform_names):
    # Set up an array to hold flags for valid mean data
    valid_asm = annual_seasonal_mean.data * 0.0

    # Loop over the years in the time series
    for isy, (season, year) in enumerate(zip(
        annual_seasonal_mean.coord('clim_season').points,
        annual_seasonal_mean.coord('season_year').points)):

        # Extract data for climate season and year
        season_constraint = iris.Constraint(clim_season=season)
        year_constraint = iris.Constraint(season_year=year)
        season_data = obs_cube[:,istn].extract(season_constraint & year_constraint)

        pdb.set_trace()

This DOES NOT work: On JASMIN $ module load esmvaltool $ esmvaltool run /path/to/recipe/recipe_aod_aeronet_assess.yml

[rcomer]

Thanks for the report @catherinehardacre. Since you say that the steps you require work outside of ESMValTool, that suggests that the problem is with ESMValTool itself rather than with Iris. So it should be reported over there.