cdsapi
cdsapi copied to clipboard
ecmwf
401 Client Error: Unauthorized for url: ...
I'm trying to retrieve a dataset from a Google Collab notebook, unsuccessfully so far, although I did apply all instructions.
import cdsapi
uploaded = files.upload() #to upload the .cdsapirc
!cp .cdsapirc ../root/
c = cdsapi.Client()
c.retrieve(
'insitu-glaciers-elevation-mass',
{
'product_type': [
'elevation_change', 'mass_balance',
],
'file_version': '20181103',
'variable': 'all',
'format': 'zip',
},
'download.zip')
insitu_glaciers_elevation_mass = pd.read_csv('download.zip', compression='zip')
This yields the following error:
2020-04-15 13:18:23,712 INFO Welcome to the CDS
2020-04-15 13:18:23,713 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/insitu-glaciers-elevation-mass
---------------------------------------------------------------------------
HTTPError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/cdsapi/api.py in _api(self, url, request, method)
388 try:
--> 389 result.raise_for_status()
390 reply = result.json()
3 frames
/usr/local/lib/python3.6/dist-packages/requests/models.py in raise_for_status(self)
939 if http_error_msg:
--> 940 raise HTTPError(http_error_msg, response=self)
941
HTTPError: 401 Client Error: Unauthorized for url: https://cds.climate.copernicus.eu/api/v2/resources/insitu-glaciers-elevation-mass
During handling of the above exception, another exception occurred:
Exception Traceback (most recent call last)
<ipython-input-12-ceb78a244bb3> in <module>()
16 'format': 'zip',
17 },
---> 18 'download.zip')
19
20 insitu_glaciers_elevation_mass = pd.read_csv('download.zip', compression='zip')
/usr/local/lib/python3.6/dist-packages/cdsapi/api.py in retrieve(self, name, request, target)
315
316 def retrieve(self, name, request, target=None):
--> 317 result = self._api('%s/resources/%s' % (self.url, name), request, 'POST')
318 if target is not None:
319 result.download(target)
/usr/local/lib/python3.6/dist-packages/cdsapi/api.py in _api(self, url, request, method)
408 "of '%s' at %s" % (t['title'], t['url']))
409 error = '. '.join(e)
--> 410 raise Exception(error)
411 else:
412 raise
Exception: <html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx/1.16.1</center>
</body>
</html>
I did first accept the terms & conditions:
What am I missing?
Can confirm that I am getting the same issue even after T&Cs accepted. I can also download the data by making manual requests at https://ads.atmosphere.copernicus.eu/cdsapp#!/dataset/cams-europe-air-quality-forecasts?tab=form so it seems I am recognised as an authenticated user.
Same problem. Inspecting the request being made, the auth header looks correct. Manually doing it via the website doesn't seem to use the same basic-auth header, so I guess the problem's somewhere in there?
Before I was having the following settings in my $HOME/.cdsapirc url: https://cds.climate.copernicus.eu/api/v2 key: {xxxxx}:{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}
After getting this error I changed to the following (Removal of curly brackets) and error got resolved. url: https://cds.climate.copernicus.eu/api/v2 key: xxxxx:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
The CDS format is as @patel-zeel stated.
url: https://cds.climate.copernicus.eu/api/v2 key: 126595:6223f93f-8f44-4f94-a369-a478f26d4926
https://cds.climate.copernicus.eu/api-how-to
I am getting the same error but my $HOME/.cdsapirc looks exactly like specified. I aceepted the terms & conditions and I can download files via the web interface just fine. My system accesses the internet via a custom proxy configuration. Is there any way I can debug the issue to find what exactly the problem is?
Here's the debug log from the client (proxy URL removed):
2022-04-08 14:49:40,580 DEBUG https://cds.climate.copernicus.eu:443 "GET /api/v2/status.json HTTP/1.1" 200 None
2022-04-08 14:49:40,581 INFO Welcome to the CDS
2022-04-08 14:49:40,581 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/cems-glofas-forecast
2022-04-08 14:49:40,581 DEBUG POST https://cds.climate.copernicus.eu/api/v2/resources/cems-glofas-forecast {"variable": "river_discharge_in_the_last_24_hours", "format": "netcdf", "system_version": "operational", "hydrological_model": "lisflood", "product_type": "control_forecast", "year": "2021", "month": "06", "day": "01", "leadtime_hour": "24"}
/home/localadmin/anaconda3/lib/python3.9/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host '<my-proxy>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
2022-04-08 14:49:40,818 DEBUG https://cds.climate.copernicus.eu:443 "POST /api/v2/resources/cems-glofas-forecast HTTP/1.1" 401 179
2022-04-08 14:49:40,819 DEBUG {"message": "<html>\r\n<head><title>401 Authorization Required</title></head>\r\n<body>\r\n<center><h1>401 Authorization Required</h1></center>\r\n<hr><center>nginx/1.20.2</center>\r\n</body>\r\n</html>\r\n"}
Traceback (most recent call last):
File "/home/localadmin/anaconda3/lib/python3.9/site-packages/cdsapi/api.py", line 427, in _api
result.raise_for_status()
File "/home/localadmin/anaconda3/lib/python3.9/site-packages/requests/models.py", line 953, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://cds.climate.copernicus.eu/api/v2/resources/cems-glofas-forecast
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/localadmin/Desktop/glofas_request_simple.py", line 7, in <module>
c.retrieve(
File "/home/localadmin/anaconda3/lib/python3.9/site-packages/cdsapi/api.py", line 348, in retrieve
result = self._api("%s/resources/%s" % (self.url, name), request, "POST")
File "/home/localadmin/anaconda3/lib/python3.9/site-packages/cdsapi/api.py", line 450, in _api
raise Exception(error)
Exception: <html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx/1.20.2</center>
</body>
</html>
Here the same, I deleted the blank space between UID and API, but i'm getting the same error.
try to delete the blank after the colon between your UID and API key
ohhh many thanks!!! It solves the problem which cost me almost 2 hours!
Hi,
This looks to be a user-issue regarding incorrect configuration of user keys. To summarise the useful hints and tips were provided here:
The key in the .cdsapirc should be of the format:
UID:APIKEY
where UID is numeric and APIKEY is alphanumeric with hyphens (-). There should be no spaces or any form of brackets. For exampe:
12345:123abc-de45f-ghi6j-7kl89
If you are still having Authentication errors after checking your key, I recommend you contact C3S user support via one of the channels available here: https://cds.climate.copernicus.eu/cdsapp#!/usersupport
Thanks, Eddy
