can't run in celery/django

[wsf1990]

Hello， It works ok as normal, But when I run the netcdf in celery task/django,It can't work,and show me this error:

File "...", line 53, in ...
    with Dataset(path, format='NETCDF4') as dataset:
  File "netCDF4/_netCDF4.pyx", line 2291, in netCDF4._netCDF4.Dataset.__init__
  File "netCDF4/_netCDF4.pyx", line 1855, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -101] NetCDF: HDF error: b'/my/nc/path'

What happened when I use celery/django?

[wsf1990]

I checked the source code, May be the question is at the 2232 line in file netCDF4/_netCDF4.pyx: https://github.com/Unidata/netcdf4-python/blob/master/netCDF4/_netCDF4.pyx#L2232

ierr = nc_open(path, NC_NOWRITE, &grpid)

But I don't know what's wrong with this code.

[jswhit]

Is '/my/nc/path' a real file?

[wsf1990]

@jswhit Yes, and it can be opened in shell mode

[wsf1990]

And I use ubyte.nc which is your test file,It works well.

[jswhit]

A permissions issues then?

[wsf1990]

Maybe no, the file in same position.Can I transfor my file to you?

[wsf1990]

test.zip

[wsf1990]

works well in shell mode. Can't work in django.

[jswhit]

Looks like the traceback is coming from MFDataset, which won't work with this file (it can only open NETCDF3 or NETCDF4_CLASSIC formatted files).

[wsf1990]

@jswhit I had checked both Dataset and MFDataset,It's all show the same error.

[jswhit]

haven't ever used django or celery myself, so I'm afraid I won't be of much help. Somehow the environment in which your are running the script is different than when you run in the shell - perhaps you're picking up a different version of the python module of C libs, or the user that the script is running under doesn't have permission to read the file?

[jswhit]

It must be possible, since this project uses netcdf4-python and django:

https://ncdjango.readthedocs.io/en/latest/

[wsf1990]

Thanks, I'll have a try.

[wsf1990]

I'd try set the dir's permission to 777: chmod -R 777 /my/dir/path. And I found that the 'ubyte.nc' and some other's which you provide can work well,but other's can't.

[MiCurry]

This project uses both Django, Celery and NetCDF4 Python: https://github.com/seacast/SharkEyes.

You are most likely not handling your environment correctly.