cdsapi
cdsapi copied to clipboard
Load into memory without saving to disk
Is there any way to load the requested data into memory without saving to disk? I am trying to load the data as netcdf and do something like this:
import cdsapi
c = cdsapi.Client()
params = {
'product_type': 'reanalysis',
'variable': 'total_precipitation',
'year': '2019',
'month': '01',
'day': '01',
'time': '00:00',
'format': 'netcdf',
'grid':[1.0, 1.0],
}
data = c.retrieve('reanalysis-era5-single-levels', params)
ds = data.load() # doesn't exist, but could load data into memory
# process the data below here
I know .download()
will save to disk. but I am curious if there is any way to implement something like a .load()
method which would load the data into memory as an xarray dataset. This way one could load the data and process it in one script. The current solution is to download the data and just throw away the original dataset after processing, but I feel like this process could be streamlined.
I looked at the download method but couldn't figure out a way to implement some type of load method. Any thoughts?
Thanks for making this package. This has saved me so much time!
I figured out a solution using urlopen
. Here is a small working example
import cdsapi
import xarray as xr
from urllib.request import urlopen
c = cdsapi.Client()
# API request
params = {'format': 'netcdf',
'product_type': 'monthly_averaged_reanalysis',
'variable': '2m_temperature',
'year': list(map(str, range(2000, 2022))),
'month': list(map("{:02d}".format, range(1,13))),
'time': '00:00',
'grid': [1.0, 1.0],}
# retrieve the location of the file
fl = c.retrieve('reanalysis-era5-single-levels-monthly-means', params)
# load into memory
with urlopen(fl.location) as f:
ds = xr.open_dataset(f.read())
Hi, Thanks @lgloege for providing a solution. Eddy