earthkit-data
earthkit-data copied to clipboard
Add option to retrieve data into user specified target files without any parsing
Is your feature request related to a problem? Please describe.
Currently we cannot retrieve data directly into a user defined file, because we cannot pass a file target to from_source
, but we have to use e.g.
ds = eathkit.data.from_source("cds", ....)
ds.save("my_file")
Here, the data is first downloaded into a tmp/cache, unpacked when needed, then parsed/interpreted. This all happens before calling save()
, which actually performs a file copy operation.
Describe the solution you'd like
A possible API:
ds = eathkit.data.from_source("cds", ...., target="my_file")
where the data will be save into file my_file
.
Some questions immediately arise:
- what would
ds
be? (it has to be a source) - what if multiple files are retrieved? (with the
cds
source it is possible) - what if the user wants to preserve the remote file name?
- sometimes the type of the retrieved file is not know in advance e.g. it can be a "zip" file. It would be nice to have an option to automatically add the right suffix to the target file. E.g. when this option is used we just need to specify
target="my_file"
and the data would be saved intomy_file.zip
.
Describe alternatives you've considered
No response
Additional context
No response
Organisation
ECMWF