netcdf4-python icon indicating copy to clipboard operation
netcdf4-python copied to clipboard

bug reading netcdf via opendap

Open jhlasheras opened this issue 7 years ago • 7 comments

I found, what i guess is a bug in the netcdf4 library.

I am using anaconda with python 3.6.3 and Ipython 6.1.0

I tried to read a file via opendap when just starting my ipython, which works:

import netCDF4 ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')

The problem arises when I load some other libraries and try to load the same file. Doing the following:

import numpy as np import pandas as pd import matplotlib.pyplot as plt

ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')

OSError: [Errno -36] NetCDF: Invalid argument: b'http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc'

However, I can read local netcdf files and also files via opendap with a shorter name:

ds2 = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/operational_models/oceanographical/hydrodynamics/wmop_surface/latest.nc')

This works fine! It also works fine when executing the same code in jupyter notebook. Any explanation? Cheers

jhlasheras avatar Jan 18 '18 16:01 jhlasheras

Sorry, but I have no idea. Does it work if you don't import pandas, or if you import netCDF4 last?

jswhit avatar Jan 18 '18 17:01 jswhit

The provided example:

import netCDF4
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')

works fine for me, produces no errors.

My NetCDF version is 1.3.1

rsignell-usgs avatar Jan 18 '18 17:01 rsignell-usgs

Works here too.

jswhit avatar Jan 18 '18 17:01 jswhit

That's rigth. I have tried it in a macbook and it works fine. However in my ubuntu 14.04 it continuosly give an error. Both of them using netCDF4 version 1.3.1. Also in other mates computer with similar configuration the same error happened.

May anaconda netcdf library be interfering with some other netcdf library?

jhlasheras avatar Jan 18 '18 18:01 jhlasheras

This might depend on the version of the netCDF-C library being used. IIRC, there were some versions that got a bit more picky about some of the opendap parsing.

dopplershift avatar Jan 18 '18 23:01 dopplershift

At least on Windows, this issue is fixed by https://github.com/Unidata/netcdf-c/pull/366 (for me)

cgohlke avatar Feb 18 '18 06:02 cgohlke

I found the problem may be due to the matplotlib backend. When initializing with Tk backend the problem dissaspears.

Adding this at the begining of the script solves the problem:

import matplotlib matplotlib.use('TkAgg') import matplotlib.pyplot as plt

everything works fine untill I execute %matplotlib

jhlasheras avatar Feb 20 '18 12:02 jhlasheras