Cannot reader a gridfile with croco output file

[sebainzunza]

trafficstars

Hi, I'm new user of Opendrift. I try to read a gridfile with a croco output fil because lat and lot variables is not in this file. I'm have this error:

18:20:08 DEBUG opendrift.models.basemodel: Adding 17 config items from basemodel 18:20:08 DEBUG opendrift.models.basemodel: Adding 4 config items from basemodel 18:20:08 DEBUG opendrift.models.basemodel: Adding 34 config items from basemodel 18:20:08 INFO opendrift.models.basemodel: OpenDriftSimulation initialised (version 1.6.0 / v1.6.0-176-g151ae59-dirty) 18:20:08 DEBUG opendrift.models.basemodel: Adding 13 config items from oceandrift 18:20:08 DEBUG opendrift.models.basemodel: Overwriting config item seed:z 18:20:08 INFO opendrift.readers.reader_ROMS_native: Opening dataset: /home/seba/opendrift/Tutorial_opendrift/croco_his.00720.nc 18:20:08 INFO opendrift.readers.reader_ROMS_native: Opening file with Dataset 18:20:08 WARNING opendrift.readers.reader_ROMS_native: Vtransform not found, using 1 Traceback (most recent call last): File "/home/seba/opendrift/Tutorial_opendrift/./sur_chile.py", line 21, in mosa_native = reader_ROMS_native.Reader('/home/seba/opendrift/Tutorial_opendrift/' + filename,gridfile=gridfile) File "/home/seba/opendrift/opendrift/readers/reader_ROMS_native.py", line 168, in init gf = Dataset(gridfile) NameError: name 'Dataset' is not defined

[knutfrode]

Hi,

Yes, it seems that this line still uses netCDF-python, whereas the rest of the code is ported to Xarray. Can you try if it works by changing line 168 in /home/seba/opendrift/opendrift/readers/reader_ROMS_native.py to

gf = xr.open_dataset(gridfile)

[sebainzunza]

I have this now: 12:11:42 DEBUG opendrift.models.basemodel: Adding 17 config items from basemodel 12:11:42 DEBUG opendrift.models.basemodel: Adding 4 config items from basemodel 12:11:42 DEBUG opendrift.models.basemodel: Adding 34 config items from basemodel 12:11:42 INFO opendrift.models.basemodel: OpenDriftSimulation initialised (version 1.6.0 / v1.6.0-176-g151ae59-dirty) 12:11:42 DEBUG opendrift.models.basemodel: Adding 13 config items from oceandrift 12:11:42 DEBUG opendrift.models.basemodel: Overwriting config item seed:z 12:11:42 INFO opendrift.readers.reader_ROMS_native: Opening dataset: /home/seba/opendrift/Tutorial_opendrift/croco_his.00720.nc 12:11:42 INFO opendrift.readers.reader_ROMS_native: Opening file with Dataset 12:11:43 WARNING opendrift.readers.reader_ROMS_native: Vtransform not found, using 1 12:11:43 INFO opendrift.readers.reader_ROMS_native: 'gls_cmu0' 12:11:43 INFO opendrift.readers.reader_ROMS_native: Did not find complete set of GLS parameters 12:11:43 INFO opendrift.readers.reader_ROMS_native: Ocean time given as seconds relative to start Setting artifical start time of 1 Jan 2000. 12:11:43 WARNING opendrift.readers.basereader.structured: No proj string or projection could be derived, using 'fakeproj'. This assumes that the variables are structured and gridded approximately equidistantly on the surface (i.e. in meters). This must be guaranteed by the user. You can get rid of this warning by suppling a valid projection to the reader. 12:11:43 INFO opendrift.readers.basereader.structured: Making interpolator for lon,lat to x,y conversion... Traceback (most recent call last): File "/home/seba/opendrift/Tutorial_opendrift/./sur_chile.py", line 21, in mosa_native = reader_ROMS_native.Reader('/home/seba/opendrift/Tutorial_opendrift/' + filename,gridfile=gridfile) File "/home/seba/opendrift/opendrift/readers/reader_ROMS_native.py", line 212, in init super(Reader, self).init() File "/home/seba/opendrift/opendrift/readers/basereader/init.py", line 94, in init super().init() File "/home/seba/opendrift/opendrift/readers/basereader/structured.py", line 73, in init (self.lon.ravel(), self.lat.ravel()), AttributeError: 'Variable' object has no attribute 'ravel'

[AndresSepulveda]

Hi, we got the same error (in version 1.5.6 it seems) and after the change we get

(opendrift) tomas@debian:/data1/tomas/opendrift$ python3 1pto.py 14:43:54 DEBUG opendrift.models.basemodel: Adding 17 config items from basemodel 14:43:54 DEBUG opendrift.models.basemodel: Adding 4 config items from basemodel 14:43:54 DEBUG opendrift.models.basemodel: Adding 34 config items from basemodel 14:43:54 INFO opendrift.models.basemodel: OpenDriftSimulation initialised (version 1.5.6 / v1.5.6-111-g21e544e-dirty) 14:43:54 DEBUG opendrift.models.basemodel: Adding 13 config items from oceandrift 14:43:54 DEBUG opendrift.models.basemodel: Overwriting config item seed:z 14:43:54 INFO opendrift.readers.reader_ROMS_native: Opening dataset: croco_his_hija1.00720.nc 14:43:54 INFO opendrift.readers.reader_ROMS_native: Opening file with Dataset 14:43:54 WARNING opendrift.readers.reader_ROMS_native: Vtransform not found, using 1 14:43:54 INFO opendrift.readers.reader_ROMS_native: 'gls_cmu0' 14:43:54 INFO opendrift.readers.reader_ROMS_native: Did not find complete set of GLS parameters 14:43:54 INFO opendrift.readers.reader_ROMS_native: Ocean time given as seconds relative to start Setting artifical start time of 1 Jan 2000. Traceback (most recent call last): File "/data1/tomas/opendrift/1pto.py", line 24, in mosa_native = reader_ROMS_native.Reader(filename=filename_nc,gridfile="croco_grd_his_hija.nc") File "/data1/tomas/opendrift/opendrift/readers/reader_ROMS_native.py", line 196, in init self.sigma = self.sigma.data AttributeError: 'Reader' object has no attribute 'sigma'

And, indeed, there is no sigma variable in the GRD file.

[knutfrode]

@sebainzunza You can try to also add two more lines, so that the block looks like this:

gf = xr.open_dataset(gridfile)
self.lat = gf.variables['lat_rho'][:]
self.lon = gf.variables['lon_rho'][:]
self.lon = self.lon.data
self.lat = self.lat.data

@AndresSepulveda The import of Sigma seems to have changed in reader_ROMS_native from v1.5.6 to present (v1.7.1). Now the reading is like this Thus, if sigma is not found in the ROMS file, it is constructed from the number of layers (not sure how meaningful this is?). But there is presently no option to import sigma from gridfile - and I have also not seen cases of that(?)

Unfortunately I have no croco- or gridfiles available to test these things myself.

[sebainzunza]

Thanks Knut! but I have this error now: 13:16:57 DEBUG opendrift.models.basemodel: Traceback (most recent call last): File "/home/seba/opendrift/opendrift/models/basemodel.py", line 1021, in get_environment reader.get_variables_interpolated( File "/home/seba/opendrift/opendrift/readers/basereader/variables.py", line 799, in get_variables_interpolated env, env_profiles = self.get_variables_interpolated_xy( File "/home/seba/opendrift/opendrift/readers/basereader/variables.py", line 667, in get_variables_interpolated_xy env, env_profiles = self.get_variables_interpolated( File "/home/seba/opendrift/opendrift/readers/basereader/structured.py", line 228, in get_variables_interpolated self.get_variables(blockvariables_before, time_before, File "/home/seba/opendrift/opendrift/readers/reader_ROMS_native.py", line 352, in get_variables self.mask_rho = self.Dataset.variables['mask_rho'][:] File "/home/seba/anaconda3/envs/opendrift/lib/python3.9/site-packages/xarray/core/utils.py", line 461, in getitem return self.mapping[key] KeyError: 'mask_rho'

[knutfrode]

Ok, is there a variable "mask_rho" in your file, or in your gridfile?

[sebainzunza]

Yes, It is in my gridfile.

[gauteh]

Is there some changes to cfgrib on whether vars are available as a dictionary? Maybe it is a bug in that library.

tor. 9. sep. 2021, 19:13 skrev Sebastian Inzunza @.***>:

Yes, It is in my gridfile.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/OpenDrift/opendrift/issues/695#issuecomment-916284083, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAN36YXDDTW5PAK6MXSSH3UBDTJXANCNFSM5DTHNTAQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[knutfrode]

This is not a grib-file, but a "grid-file" :-) Sometimes lon and lat are not stored in the ROMS netCDF-file, but are then in a separate grid-file In this specific case, it seems that sigma-coordinates are also in the separate grid-file. So the ROMS reader must be updated to also read sigma from the grid-file, for such cases. I will probably not be able to look at this today, or the next few days. It could help if you could in the meantime provide the actual grid-file, and possibly a small corresponding ROMS-file.

[gauteh]

Oh sorry!

fre. 10. sep. 2021 kl. 09:23 skrev Knut-Frode Dagestad < @.***>:

This is not a grib-file, but a "grid-file" :-) Sometimes lon and lat are not stored in the ROMS netCDF-file, but are then in a separate grid-file In this specific case, it seems that sigma-coordinates are also in the separate grid-file. So the ROMS reader must be updated to also read sigma from the grid-file, for such cases. I will probably not be able to look at this today, or the next few days. It could help if you could in the meantime provide the actual grid-file, and possibly a small corresponding ROMS-file.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/OpenDrift/opendrift/issues/695#issuecomment-916690381, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAN367BC6ZXABBB5BDBN4LUBGW5HANCNFSM5DTHNTAQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[sebainzunza]

This is not a grib-file, but a "grid-file" :-) Sometimes lon and lat are not stored in the ROMS netCDF-file, but are then in a separate grid-file In this specific case, it seems that sigma-coordinates are also in the separate grid-file. So the ROMS reader must be updated to also read sigma from the grid-file, for such cases. I will probably not be able to look at this today, or the next few days. It could help if you could in the meantime provide the actual grid-file, and possibly a small corresponding ROMS-file.

I hope I can help with this: https://drive.google.com/file/d/1icK5A51MfWctzS-RgCTwcIUPHZYtzy72/view?usp=sharing

[TomValderrama]

Hi! I am having the same error, this only happens with ROMS outputs recorded in netCDF-3, other outputs in netCDF-4 do not generate that problem for me. I'm not sure if it is because of that.

[TomValderrama]

Hi, I finally tried to transform the files from netCDF-3 to netCDF-4 and got positive results. The sequence is as follows:

1. Transform the his file (or avg, etc.) and the grid file from netCDF-3 to netCDF-4 nccopy -k 3 in_gridfile.nc out_gridfile.nc nccopy -k 3 in_hisfile.nc out_hisfile.nc
2. Then I attached the grid file to the his file ncks -A out_gridfile.nc out_hisfile.nc Finally, add it to the codes you are using, and it should work. I hope it is a useful solution.