datacube-core icon indicating copy to clipboard operation
datacube-core copied to clipboard

Issue loading virtual product using `like=ds.geobox`

Open robbibt opened this issue 2 years ago • 0 comments

Expected behaviour

I would like to load a Landsat virtual product into the same spatial grid/geobox as a previously loaded Sentinel-2 virtual product. To do this using dc.load, I would pass in like=s2_ds.geobox and my data would be loaded into the same grid.

Actual behaviour

However, when I attempt to use like=s2_ds.geobox on my Landsat virtual product, I get the following error. For a large data load, this code also runs slowly, which is unexpected given that it uses Dask so should load lazily/quickly.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_127/2986533797.py in <module>
     44 
     45 # Load Sentinel-2 product
---> 46 ls_ds = catalog["ls8_nbart"].load(
     47     dc,
     48     time=("2020-01", "2020-02"),

/env/lib/python3.8/site-packages/datacube/virtual/impl.py in load(self, dc, **query)
    307         datasets = self.query(dc, **query)
    308         grouped = self.group(datasets, **query)
--> 309         return self.fetch(grouped, **query)
    310 
    311 

/env/lib/python3.8/site-packages/datacube/virtual/impl.py in fetch(self, grouped, **load_settings)
    397         if grouped.load_natively:
    398             canonical_names = [product.canonical_measurement(measurement) for measurement in measurement_dicts]
--> 399             dataset_geobox = geobox_union_conservative([native_geobox(ds,
    400                                                                       measurements=canonical_names,
    401                                                                       basis=merged.get('like'))

/env/lib/python3.8/site-packages/datacube/virtual/impl.py in <listcomp>(.0)
    397         if grouped.load_natively:
    398             canonical_names = [product.canonical_measurement(measurement) for measurement in measurement_dicts]
--> 399             dataset_geobox = geobox_union_conservative([native_geobox(ds,
    400                                                                       measurements=canonical_names,
    401                                                                       basis=merged.get('like'))

/env/lib/python3.8/site-packages/datacube/testutils/io.py in native_geobox(ds, measurements, basis)
    103     if is_doc_eo3(ds.metadata_doc):
    104         if basis is not None:
--> 105             return eo3_geobox(ds, basis)
    106 
    107         gboxes = [eo3_geobox(ds, band) for band in measurements]

/env/lib/python3.8/site-packages/datacube/testutils/io.py in eo3_geobox(ds, band)
     61 
     62 def eo3_geobox(ds: Dataset, band: str) -> GeoBox:
---> 63     mm = ds.measurements.get(ds.type.canonical_measurement(band),
     64                              None)
     65     if mm is None:

/env/lib/python3.8/site-packages/datacube/model/__init__.py in canonical_measurement(self, measurement)
    585         m = self._resolve_aliases().get(measurement, None)
    586         if m is None:
--> 587             raise ValueError(f"No such band/alias {measurement}")
    588 
    589         return m.canonical_name

ValueError: No such band/alias GeoBox(Geometry({'type': 'Polygon', 'coordinates': (((136.65975034783972, -15.829968303813862), (136.65991574403617, -15.850210580671073), (136.68017401916399, -15.85005442568298), (136.68000660670677, -15.829812358392235), (136.65975034783972, -15.829968303813862)),)}, CRS('EPSG:4326')))

Steps to reproduce the behaviour

from datacube.virtual import catalog_from_yaml
import datacube

dc = datacube.Datacube()

catalog_yaml = """
about: Test catalogue
products:

    ls8_nbart:
        recipe:
            &ls8_nbart
            product: ga_ls8c_ard_3
            measurements: [nbart_blue, nbart_green, nbart_red, nbart_nir, 
                           nbart_swir_1, nbart_swir_2, 
                           oa_nbart_contiguity, oa_fmask]
            gqa_iterative_mean_xy: [0, 1]
            group_by: solar_day

    s2a_nbart:
        recipe:
            &s2a_nbart
            product: ga_s2am_ard_3
            measurements: [nbart_blue, nbart_green, nbart_red, nbart_nir_1, 
                           nbart_nir_2, nbart_swir_2, nbart_swir_3, 
                           oa_nbart_contiguity, oa_s2cloudless_mask]
            gqa_iterative_mean_xy: [0, 1]
            group_by: solar_day
"""

# Load catalogue
catalog = catalog_from_yaml(catalog_yaml)

# Load Sentinel-2 product
s2_ds = catalog["s2a_nbart"].load(
    dc,
    time=("2020-01", "2020-02"),
    dask_chunks={"time": 1},
    x=(136.66, 136.68),
    y=(-15.83, -15.85),
    resolution=(-10, 10),
    output_crs="epsg:32753",
)

# Load Landsat product
ls_ds = catalog["ls8_nbart"].load(
    dc,
    time=("2020-01", "2020-02"),
    dask_chunks={"time": 1},
    like=s2_ds.geobox,
)

For comparison, this works fine:

ls_ds = dc.load(
    product="ga_ls8c_ard_3",
    time=("2020-01", "2020-02"),
    dask_chunks={"time": 1},
    like=s2_ds.geobox,
)

Environment information

  • Which datacube --version are you using? 1.8.7
  • What datacube deployment/enviornment are you running against? DEA Dev Sandbox

Note: Stale issues will be automatically closed after a period of six months with no activity. To ensure critical issues are not closed, tag them with the Github pinned tag. If you are a community member and not a maintainer please escalate this issue to maintainers via GIS StackExchange or Slack.

robbibt avatar Jun 15 '22 03:06 robbibt