eodag
eodag copied to clipboard
Published
20 hours ago •
CS-SI
CS-SI
Unable to access CAMS_GLOBAL_EMISSIONS data after 2021
Describe the bug
I've tried to access the CAMS_GLOBAL_EMISSIONS through the cop_ads provider but I'm getting the following error:
DownloadError: the request you have submitted is not valid. Value '2021' not valid for parameter 'year', valid values are: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, ....
However, should exist as the dag.search returns a EOProduct
The error raises from 2021 on.
Code To Reproduce CLI commands or Python code snippet to reproduce the bug. Please use maximum verbosity using:
from eodag import EODataAccessGateway, setup_logging
setup_logging(3) # 0: nothing, 1: only progress bars, 2: INFO, 3: DEBUG
dag = EODataAccessGateway()
geometry = "POLYGON ((-70.889282 -33.560064, -70.889282 -33.284979, -70.493774 -33.284979, -70.493774 -33.560064, -70.889282 -33.560064))"
search_results, total_count = dag.search(
provider="cop_ads",
productType="CAMS_GLOBAL_EMISSIONS",
geom=geometry,
start="2021-02-15",
end="2021-02-28",
)
product = search_results[0]
product_path = product.download()
Output
2024-02-29 17:44:20,772 eodag.core [INFO ] (tid=140253333770240) Searching product type 'CAMS_GLOBAL_EMISSIONS' on provider: cop_ads
2024-02-29 17:44:20,773 eodag.search.base [DEBUG ] (tid=140253333770240) Mapping eodag product type to provider product type
2024-02-29 17:44:20,774 eodag.search.base [DEBUG ] (tid=140253333770240) Getting provider product type definition parameters for CAMS_GLOBAL_EMISSIONS
2024-02-29 17:44:20,774 eodag.search.qssearch [DEBUG ] (tid=140253333770240) Building the query string that will be used for search
2024-02-29 17:44:20,775 eodag.product.metadata_mapping [DEBUG ] (tid=140253333770240) Retrieving queryable metadata from metadata_mapping
2024-02-29 17:44:20,780 eodag.core [INFO ] (tid=140253333770240) Found 1 result(s) on provider 'cop_ads'
CAMS_GLOBAL_EMISSIONS_20210215_775429dfba2ed71b2bfe0ead9bac3e9fa1f45f44:
0.00/? [00:00<?, ?B/s]
2024-02-29 17:44:20,789 eodag.download.base [INFO ] (tid=140253333770240) Download url: https://ads.atmosphere.copernicus.eu/cdsapp#!/dataset/cams-global-emission-inventories?{"area": "-33.284979/-70.889282/-33.560064/-70.493774", "dataset": "cams-global-emission-inventories", "day": ["23", "25", "24", "17", "19", "16", "21", "26", "20", "15", "18", "27", "22"], "format": "zip", "month": "02", "source": "anthropogenic", "variable": "acids", "version": "latest", "year": "2021"}
2024-02-29 17:44:21,221 eodag.apis.cds [DEBUG ] (tid=140253333770240) Connection checked on CDS API
2024-02-29 17:44:21,223 eodag.apis.cds [INFO ] (tid=140253333770240) Request download on CDS API: dataset=cams-global-emission-inventories, request={'area': '-33.284979/-70.889282/-33.560064/-70.493774', 'day': ['23', '25', '24', '17', '19', '16', '21', '26', '20', '15', '18', '27', '22'], 'format': 'zip', 'month': '02', 'source': 'anthropogenic', 'variable': 'acids', 'version': 'latest', 'year': '2021'}
2024-02-29 17:44:21,225 cdsapi [DEBUG ] (tid=140253333770240) CDSAPI {'url': 'https://ads.atmosphere.copernicus.eu/api/v2', 'key': '18663:382eca88-1560-42b3-bbe3-8e7a85414020', 'quiet': True, 'verify': True, 'timeout': 60, 'progress': False, 'sleep_max': 120, 'retry_max': 500, 'full_stack': False, 'delete': True, 'metadata': None, 'forget': False}
2024-02-29 17:44:21,360 cdsapi [INFO ] (tid=140253333770240) Welcome to the CDS
2024-02-29 17:44:21,362 cdsapi [INFO ] (tid=140253333770240) Sending request to https://ads.atmosphere.copernicus.eu/api/v2/resources/cams-global-emission-inventories
2024-02-29 17:44:21,363 cdsapi [DEBUG ] (tid=140253333770240) POST https://ads.atmosphere.copernicus.eu/api/v2/resources/cams-global-emission-inventories {"area": "-33.284979/-70.889282/-33.560064/-70.493774", "day": ["23", "25", "24", "17", "19", "16", "21", "26", "20", "15", "18", "27", "22"], "format": "zip", "month": "02", "source": "anthropogenic", "variable": "acids", "version": "latest", "year": "2021"}
2024-02-29 17:44:21,493 cdsapi [DEBUG ] (tid=140253333770240) REPLY {'state': 'queued', 'request_id': '46da30f4-e0d0-468c-a28a-fe8c0b1b6fea', 'specific_metadata_json': {'top_request_origin': 'api'}}
2024-02-29 17:44:21,494 cdsapi [INFO ] (tid=140253333770240) Request is queued
2024-02-29 17:44:21,495 cdsapi [DEBUG ] (tid=140253333770240) Request ID is 46da30f4-e0d0-468c-a28a-fe8c0b1b6fea, sleep 1
2024-02-29 17:44:22,497 cdsapi [DEBUG ] (tid=140253333770240) GET https://ads.atmosphere.copernicus.eu/api/v2/tasks/46da30f4-e0d0-468c-a28a-fe8c0b1b6fea
2024-02-29 17:44:22,632 cdsapi [DEBUG ] (tid=140253333770240) REPLY {'state': 'failed', 'request_id': '46da30f4-e0d0-468c-a28a-fe8c0b1b6fea', 'error': {'message': 'the request you have submitted is not valid', 'url': 'http://copernicus-climate.eu/exc/constraints-bad-params', 'reason': "Value '2021' not valid for parameter 'year', valid values are: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, ...", 'context': {'traceback': 'Traceback (most recent call last):\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume\n result = handle_locally()\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>\n lambda: self.handle_exception(context, e),\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 383, in handle_exception\n raise exception\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume\n result = handle_locally()\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>\n lambda: self.handle_exception(context, e),\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 365, in handle_exception\n raise exception\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume\n result = handle_locally()\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>\n lambda: self.handle_exception(context, e),\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 406, in handle_exception\n raise exception\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume\n result = handle_locally()\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>\n lambda: self.handle_exception(context, e),\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 132, in handle_exception\n raise exception\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume\n result = handle_locally()\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>\n lambda: self.handle_exception(context, e),\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 132, in handle_exception\n raise exception\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume\n result = handle_locally()\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>\n lambda: self.handle_exception(context, e),\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 132, in handle_exception\n raise exception\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume\n result = handle_locally()\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>\n lambda: self.handle_exception(context, e),\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 132, in handle_exception\n raise exception\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume\n result = handle_locally()\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 210, in <lambda>\n lambda: self.handle_request(context, request_data),\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/constraintsprocessor.py", line 46, in handle_request\n constraint_type_strict)\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/constraintsengines/constraintsengine.py", line 66, in run\n self._check_request_validity(cdsinf, request_params, constraints, constraint_type_strict)\n File "/opt/cds/cdsinf/python/lib/cdsinf/runner/constraintsengines/constraintsengine.py", line 162, in _check_request_validity\n "http://copernicus-climate.eu/exc/constraints-bad-params")\ncdsinf.exceptions.BadRequestException: Value \'2021\' not valid for parameter \'year\', valid values are: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, ...\n'}, 'permanent': True, 'who': 'client'}, 'sent_to_rmq_at': '2024-02-29T16:44:21.365Z', 'specific_metadata_json': {'top_request_origin': 'api'}}
2024-02-29 17:44:22,634 cdsapi [INFO ] (tid=140253333770240) Request is failed
2024-02-29 17:44:22,635 cdsapi [ERROR ] (tid=140253333770240) Message: the request you have submitted is not valid
2024-02-29 17:44:22,636 cdsapi [ERROR ] (tid=140253333770240) Reason: Value '2021' not valid for parameter 'year', valid values are: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, ...
2024-02-29 17:44:22,638 cdsapi [ERROR ] (tid=140253333770240) Traceback (most recent call last):
2024-02-29 17:44:22,638 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume
2024-02-29 17:44:22,639 cdsapi [ERROR ] (tid=140253333770240) result = handle_locally()
2024-02-29 17:44:22,640 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>
2024-02-29 17:44:22,641 cdsapi [ERROR ] (tid=140253333770240) lambda: self.handle_exception(context, e),
2024-02-29 17:44:22,642 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 383, in handle_exception
2024-02-29 17:44:22,642 cdsapi [ERROR ] (tid=140253333770240) raise exception
2024-02-29 17:44:22,643 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume
2024-02-29 17:44:22,644 cdsapi [ERROR ] (tid=140253333770240) result = handle_locally()
2024-02-29 17:44:22,644 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>
2024-02-29 17:44:22,645 cdsapi [ERROR ] (tid=140253333770240) lambda: self.handle_exception(context, e),
2024-02-29 17:44:22,645 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 365, in handle_exception
2024-02-29 17:44:22,645 cdsapi [ERROR ] (tid=140253333770240) raise exception
2024-02-29 17:44:22,646 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume
2024-02-29 17:44:22,646 cdsapi [ERROR ] (tid=140253333770240) result = handle_locally()
2024-02-29 17:44:22,646 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>
2024-02-29 17:44:22,646 cdsapi [ERROR ] (tid=140253333770240) lambda: self.handle_exception(context, e),
2024-02-29 17:44:22,647 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 406, in handle_exception
2024-02-29 17:44:22,647 cdsapi [ERROR ] (tid=140253333770240) raise exception
2024-02-29 17:44:22,647 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume
2024-02-29 17:44:22,647 cdsapi [ERROR ] (tid=140253333770240) result = handle_locally()
2024-02-29 17:44:22,647 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>
2024-02-29 17:44:22,648 cdsapi [ERROR ] (tid=140253333770240) lambda: self.handle_exception(context, e),
2024-02-29 17:44:22,648 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 132, in handle_exception
2024-02-29 17:44:22,648 cdsapi [ERROR ] (tid=140253333770240) raise exception
2024-02-29 17:44:22,648 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume
2024-02-29 17:44:22,648 cdsapi [ERROR ] (tid=140253333770240) result = handle_locally()
2024-02-29 17:44:22,649 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>
2024-02-29 17:44:22,649 cdsapi [ERROR ] (tid=140253333770240) lambda: self.handle_exception(context, e),
2024-02-29 17:44:22,649 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 132, in handle_exception
2024-02-29 17:44:22,649 cdsapi [ERROR ] (tid=140253333770240) raise exception
2024-02-29 17:44:22,649 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume
2024-02-29 17:44:22,649 cdsapi [ERROR ] (tid=140253333770240) result = handle_locally()
2024-02-29 17:44:22,650 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>
2024-02-29 17:44:22,650 cdsapi [ERROR ] (tid=140253333770240) lambda: self.handle_exception(context, e),
2024-02-29 17:44:22,650 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 132, in handle_exception
2024-02-29 17:44:22,650 cdsapi [ERROR ] (tid=140253333770240) raise exception
2024-02-29 17:44:22,650 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume
2024-02-29 17:44:22,651 cdsapi [ERROR ] (tid=140253333770240) result = handle_locally()
2024-02-29 17:44:22,651 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 252, in <lambda>
2024-02-29 17:44:22,651 cdsapi [ERROR ] (tid=140253333770240) lambda: self.handle_exception(context, e),
2024-02-29 17:44:22,652 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 132, in handle_exception
2024-02-29 17:44:22,652 cdsapi [ERROR ] (tid=140253333770240) raise exception
2024-02-29 17:44:22,652 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 163, in _consume
2024-02-29 17:44:22,653 cdsapi [ERROR ] (tid=140253333770240) result = handle_locally()
2024-02-29 17:44:22,653 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/dispatcher.py", line 210, in <lambda>
2024-02-29 17:44:22,654 cdsapi [ERROR ] (tid=140253333770240) lambda: self.handle_request(context, request_data),
2024-02-29 17:44:22,654 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/constraintsprocessor.py", line 46, in handle_request
2024-02-29 17:44:22,654 cdsapi [ERROR ] (tid=140253333770240) constraint_type_strict)
2024-02-29 17:44:22,654 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/constraintsengines/constraintsengine.py", line 66, in run
2024-02-29 17:44:22,660 cdsapi [ERROR ] (tid=140253333770240) self._check_request_validity(cdsinf, request_params, constraints, constraint_type_strict)
2024-02-29 17:44:22,660 cdsapi [ERROR ] (tid=140253333770240) File "/opt/cds/cdsinf/python/lib/cdsinf/runner/constraintsengines/constraintsengine.py", line 162, in _check_request_validity
2024-02-29 17:44:22,661 cdsapi [ERROR ] (tid=140253333770240) "http://copernicus-climate.eu/exc/constraints-bad-params")
2024-02-29 17:44:22,661 cdsapi [ERROR ] (tid=140253333770240) cdsinf.exceptions.BadRequestException: Value '2021' not valid for parameter 'year', valid values are: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, ...
2024-02-29 17:44:22,662 eodag.apis.cds [ERROR ] (tid=140253333770240) the request you have submitted is not valid. Value '2021' not valid for parameter 'year', valid values are: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, ....
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
File ~/envs/coplac/lib/python3.10/site-packages/eodag/plugins/apis/cds.py:351, in CdsApi._prepare_download_link(self, product)
350 client = self._get_cds_client(**auth_dict)
--> 351 result = client._api(
352 "%s/resources/%s" % (client.url, dataset_name), download_request, "POST"
353 )
354 # update product download link through a new asset
File ~/envs/coplac/lib/python3.10/site-packages/cdsapi/api.py:519, in Client._api(self, url, request, method)
518 self.error(" %s", n)
--> 519 raise Exception(
520 "%s. %s."
521 % (reply["error"].get("message"), reply["error"].get("reason"))
522 )
524 raise Exception("Unknown API state [%s]" % (reply["state"],))
Exception: the request you have submitted is not valid. Value '2021' not valid for parameter 'year', valid values are: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, ....
During handling of the above exception, another exception occurred:
DownloadError Traceback (most recent call last)
Cell In[28], line 15
6 search_results, total_count = dag.search(
7 provider="cop_ads",
8 productType="CAMS_GLOBAL_EMISSIONS",
(...)
11 end="2021-02-28",
12 )
14 product = search_results[0]
---> 15 product_path = product.download()
File ~/envs/coplac/lib/python3.10/site-packages/eodag/api/product/_product.py:365, in EOProduct.download(self, progress_callback, wait, timeout, **kwargs)
359 self.location = urllib.parse.unquote(self.location)
361 progress_callback, close_progress_callback = self._init_progress_bar(
362 progress_callback
363 )
--> 365 fs_path = self.downloader.download(
366 self,
367 auth=auth,
368 progress_callback=progress_callback,
369 wait=wait,
370 timeout=timeout,
371 **kwargs,
372 )
374 # close progress bar if needed
375 if close_progress_callback:
File ~/envs/coplac/lib/python3.10/site-packages/eodag/plugins/apis/cds.py:386, in CdsApi.download(self, product, auth, progress_callback, wait, timeout, **kwargs)
383 product.location = path_to_uri(fs_path)
384 return fs_path
--> 386 self._prepare_download_link(product)
388 try:
389 return super(CdsApi, self).download(
390 product,
391 progress_callback=progress_callback,
392 **kwargs,
393 )
File ~/envs/coplac/lib/python3.10/site-packages/eodag/plugins/apis/cds.py:358, in CdsApi._prepare_download_link(self, product)
356 except Exception as e:
357 logger.error(e)
--> 358 raise DownloadError(e)
DownloadError: the request you have submitted is not valid. Value '2021' not valid for parameter 'year', valid values are: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, ....
Environment:
- Python version: Python 3.10.12
- EODAG version: eodag (Earth Observation Data Access Gateway): version 2.12.0
- eodag-cube 0.4.0
Hello @InigoReizabal , this provider does not provide a search API, so in eodag we use the search method to build a ready-to-download EO Product. See CDS API tuturial or CdsApi plugin developer documentation.
However, now that queryables using provider's constraints are implemented, we can plan to add a check during the search step. A feature request is already implemented for that: https://github.com/CS-SI/eodag/issues/942
Thanks for the answer. In that case why data before 2020 can be downloaded?
Thanks for the answer. In that case why data before 2020 can be downloaded?
This limitation comes from the the provider itself, see https://ads.atmosphere.copernicus.eu/cdsapp#!/dataset/cams-global-emission-inventories?tab=form