netcdf4-python
netcdf4-python copied to clipboard
Unidata
Timeout of Dataset constructor
For our operational service it would be useful with possibility to specify a timeout on calling netcdf4.Dataset on remote datasets (Thredds/OPeNDAP), e.g.:
f = Dataset(OPeNDAP_URL, 'r', timeout=10)
This might be possible using the signals module (https://docs.python.org/3/library/signal.html#example).
If anyone wants to tackle this, a pull request would be welcome.
I think this can be done now by setting the TIMEOUT variable in the .daprc file that the C library uses.
https://www.unidata.ucar.edu/software/netcdf/docs/auth.html
You are right, it helps to add e.g. HTTP.TIMEOUT=5 to .dodsrc (probably same with .daprc).
This is an ok solution, although it would still be nice to be able to control it explicitly through Python.
Same problem here, I had some cases when some scripts got frozen following some gateway problems on a server.
Do you think that it would be possible to make this into a 'native' feature of the Dataset constructor, and / or to timeout / raise an exception by default when this happens?
The C API (which the python interface uses) does not provide an explicit way to set the timeout.
Ok. That is a bit sad, but guess not much to be done there from Python then. Do you think it may be possible to try something else as a workaround, for example, checking for a valid ping in the case of an url, or something like that? Could that help for the cases when the server may be completely down?
Perhaps a Dataset wrapper or subclass could be created for working with remote OpenDAP datasets, with additional features like this. A PR would be welcome.