ValueError: too many values to unpack (expected 2)

[rodebiet]

trafficstars

Am i doing something wrong?

$ makesurface vectorize gfs.t00z.pgrb2.0p25.f000 p.shp -g Traceback (most recent call last): File "/usr/local/bin/makesurface", line 9, in load_entry_point('makesurface==0.0.8dev', 'console_scripts', 'makesurface')() File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 610, in call return self.main(_args, *_kwargs) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 590, in main rv = self.invoke(ctx) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 936, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 782, in invoke return ctx.invoke(self.callback, *_ctx.params) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 416, in invoke return callback(_args, **kwargs) File "/disk/src/make-surface/makesurface/scripts/cli.py", line 57, in vectorize makesurface.vectorize(infile, outfile, classes, classfile, weight, nodata, smoothing, band, carto, globewrap, axonometrize, nosimple, setnodata, nibble, rapfix) File "/disk/src/make-surface/makesurface/init.py", line 4, in vectorize vectorize_raster.vectorizeRaster(infile, outfile, classes, classfile, weight, nodata, smoothing, band, cartoCSS, globeWrap, axonometrize, nosimple, setNoData, nibbleMask, rapFix) File "/disk/src/make-surface/makesurface/scripts/vectorize_raster.py", line 84, in vectorizeRaster band = filter(lambda i: src.tags(i)['GRIB_ELEMENT'] == band, src.indexes) File "/disk/src/make-surface/makesurface/scripts/vectorize_raster.py", line 84, in band = filter(lambda i: src.tags(i)['GRIB_ELEMENT'] == band, src.indexes) File "rasterio/_base.pyx", line 501, in rasterio._base.DatasetReader.tags (rasterio/_base.c:8978) ValueError: too many values to unpack (expected 2)

[dnomadb]

@rodebiet yeah, I have seen that happen with certain many-band gribs (and have not worked out exactly why as of yet). Sorry about that!

A related side note - I have been moving the -g / --globewrap functionality out of makesurface into its own module, gribdoctor. This is an attempt to better modularize the work, as well as reduce issues like the above) by cleaning up the grib and saving to a geotiff before vectorizing.

Try converting that grib (note - be cautious about converting full gribs, as a 300 + band grib can end up very large as a tiff), and re-running.

[sgillies]

Here's the troubled code in Rasterio:

            key, value = item.split('=')

Rasterio is expecting metadata items from GDAL like KEY=VALUE but if there was another = in the value... two names on the left hand side and three values on the right would produce that unpacking error.

[dnomadb]

@rodebiet any chance you could gist the results / paste the results of gdalinfo {that file}? It might point out which band tag is the culprit in the above issue.

[sgillies]

@dnomadb @rodebiet try Rasterio 0.17.1 (just released). It's got a bug fix that might help.

[rodebiet]

Thanks for your help!

The grib file can be downloaded from: http://www.ftp.ncep.noaa.gov/data/nccf/com/gfs/prod/gfs.2015011800/gfs.t00z.pgrb2.0p25.f000

@dnomadb gdalinfo gist: https://gist.github.com/rodebiet/11be40470ad7a47cdaa6

@sgillies I updated to rasterio 0.17.1

I now have another error:

$ makesurface vectorize gfs.t00z.pgrb2.0p25.f000 p.shp -g Traceback (most recent call last): File "/usr/local/bin/makesurface", line 9, in load_entry_point('makesurface==0.0.8dev', 'console_scripts', 'makesurface')() File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 610, in call return self.main(_args, *_kwargs) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 590, in main rv = self.invoke(ctx) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 936, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 782, in invoke return ctx.invoke(self.callback, *_ctx.params) File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 416, in invoke return callback(_args, **kwargs) File "/disk/src/make-surface/makesurface/scripts/cli.py", line 57, in vectorize makesurface.vectorize(infile, outfile, classes, classfile, weight, nodata, smoothing, band, carto, globewrap, axonometrize, nosimple, setnodata, nibble, rapfix) File "/disk/src/make-surface/makesurface/init.py", line 4, in vectorize vectorize_raster.vectorizeRaster(infile, outfile, classes, classfile, weight, nodata, smoothing, band, cartoCSS, globeWrap, axonometrize, nosimple, setNoData, nibbleMask, rapFix) File "/disk/src/make-surface/makesurface/scripts/vectorize_raster.py", line 88, in vectorizeRaster inarr = src.read_band(band) File "rasterio/_io.pyx", line 548, in rasterio._io.RasterReader.read_band (rasterio/_io.c:6656) File "rasterio/_io.pyx", line 601, in rasterio._io.RasterReader.read (rasterio/_io.c:7005) ValueError: No indexes to read

[dnomadb]

@rodebiet Sorry for the long delay, but I think that I may have fixed the root cause of your error. I've updated quite a few things - make sure to take a look at the readme. One big thing is that it is no longer writing to shapefiles, but rather, to GeoJSON, or as streaming GeoJSON. The latter is useful as one can directly make into vector mbtiles using tippecanoe by direct piping.