datacube-core icon indicating copy to clipboard operation
datacube-core copied to clipboard

datacube dataset add should fail if center_dt or datetime is out of start and end time range

Open pindge opened this issue 2 years ago • 0 comments

Expected behaviour

datacube dataset add should error and prevent dataset from indexing if datetime or center_dt is out of start_datetime and end_datetime time range.

for example

datetime: 2019-05-15T00:00:00Z
dtr:end_datetime: 2019-05-31T23:59:59Z
dtr:start_datetime: 2019-05-31T00:00:00Z 

Actual behaviour

pqsl metadata query returns the following

                                                                            metadata                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 {"id": "35cbccee-cb07-51cf-85d2-6d2948957545", "crs": "epsg:4326", "grids": {"default": {"shape": [1600, 1500], "transform": [0.05000000074505806, 0.0, -20.0, 0.0, -0.05000000074505806, 40.0, 0.0, 0.0, 1.0]}}, "extent": {"lat": {"end": 40.0, "begin": -40.000001192092896}, "lon": {"end": 55.00000111758709, "begin": -20.0}}, "$schema": "https://schemas.opendatacube.org/dataset", "lineage": {"source_datasets": {}}, "product": {"name": "rainfall_chirps_daily"}, "geometry": {"type": "Polygon", "coordinates": [[[-20.0, 40.0], [-20.0, -40.000001192092896], [55.00000111758709, -40.000001192092896], [55.00000111758709, 40.0], [-20.0, 40.0]]]}, "location": "s3://deafrica-input-datasets/rainfall_chirps_daily/2019/05/chirps-v2.0_2019.05.31.stac-item.json", "properties": {"datetime": "2019-05-15T00:00:00Z", "proj:bbox": [-20.0, -40.000001192092896, 55.00000111758709, 40.0], "proj:epsg": 4326, "proj:shape": [1600, 1500], "odc:product": "rainfall_chirps_daily", "proj:geometry": {"type": "Polygon", "coordinates": [[[-20.0, 40.0], [-20.0, -40.000001192092896], [55.00000111758709, -40.000001192092896], [55.00000111758709, 40.0], [-20.0, 40.0]]]}, "proj:transform": [0.05000000074505806, 0.0, -20.0, 0.0, -0.05000000074505806, 40.0, 0.0, 0.0, 1.0], "odc:file_format": "GeoTIFF", "dtr:end_datetime": "2019-05-31T23:59:59Z", "dtr:start_datetime": "2019-05-31T00:00:00Z", "odc:processing_datetime": "2021-12-17T02:30:32.725683Z"}, "accessories": {}, "grid_spatial": {"projection": {"valid_data": {"type": "Polygon", "coordinates": [[[-20.0, 40.0], [-20.0, -40.000001192092896], [55.00000111758709, -40.000001192092896], [55.00000111758709, 40.0], [-20.0, 40.0]]]}, "geo_ref_points": {"ll": {"x": -20.0, "y": -40.000001192092896}, "lr": {"x": 55.00000111758709, "y": -40.000001192092896}, "ul": {"x": -20.0, "y": 40.0}, "ur": {"x": 55.00000111758709, "y": 40.0}}, "spatial_reference": "epsg:4326"}}, "measurements": {"rainfall": {"path": "chirps-v2.0_2019.05.31.tif"}}}
(1 row)

datacube dataset search returns the following

root@e2ea6f2165ae:/code# datacube dataset search id = '35cbccee-cb07-51cf-85d2-6d2948957545'
id: 35cbccee-cb07-51cf-85d2-6d2948957545
product: rainfall_chirps_daily
status: active
locations:
- file://example.com/test_dataset/
fields:
    cloud_cover: null
    creation_time: 2021-12-17 02:30:32.725683+00:00
    crs_raw: epsg:4326
    dataset_maturity: null
    format: GeoTIFF
    instrument: null
    label: null
    lat: {begin: -40.000001192092896, end: 40.0}
    lon: {begin: -20.0, end: 55.00000111758709}
    platform: null
    product_family: null
    region_code: null
    time: {begin: '2019-05-15T00:00:00+00:00', end: '2019-05-31T23:59:59+00:00'}

Note, the begin time showing in console output time begin is being adjusted to 2019-05-15 whereas in metadata it is being stored as

"dtr:start_datetime": "2019-05-31T00:00:00Z"

Steps to reproduce the behaviour

index a dataset with datetime out of start and end daterange.

Environment information

  • Which datacube --version are you using?
root@e2ea6f2165ae:/code# datacube --version
Open Data Cube core, version 1.8.6
  • What datacube deployment/enviornment are you running against? Test

pindge avatar Jun 03 '22 01:06 pindge