mapbox-vector-tile icon indicating copy to clipboard operation
mapbox-vector-tile copied to clipboard

Add command line utils: mvt_decode and mvt_encode

Open ptpt opened this issue 6 years ago • 4 comments

This PR adds two executables mvt_encode and mvt_decode.

Still WIP. Would nice to have your options on adding these utils.

ptpt avatar Nov 01 '17 09:11 ptpt

Sure, if you'd like to add that we'd accept a pr for it.

I'd suggest making a single module to house both commands, maybe call it command.py? Then, just add entry points to it from setup.py, something like:

      entry_points=dict(
          console_scripts=[
              'mvt_encode = mapbox_vector_tile.command:mvt_encode',
              'mvt_decode = mapbox_vector_tile.command:mvt_decode',
          ]
      )

I'd expect the decoder to take either stdin or the path of the file to read as an argument. The other variable here is whether the input is gzipped or not. Maybe try to gunzip and default to assuming pbf binary if it's not gzip?

What would the encode command take as input? Would it assume a geojson file just to be symmetric with decode? Where keys are layer names, and values are individual geojson for that key?

rmarianski avatar Nov 01 '17 15:11 rmarianski

The commands are just used for debugging purpose, so I'd keep them simple. Here is my proposal:

The decoder will accept mvt binary stream from stdin and print out what the function mvt.encode returns as json. If the mvt binary is compressed, the user should uncompress it before passing to the decoder.

The encoder will take geojson stream from stdin, a layer name, and a tile coordinate, and print out the mvt binary.

mvt_encode layer_name z/x/y < cities.geojson

The geometries in the geojson could be in any coordinate system. By default it's WGS84.

Maybe we can add an option --format to specify the input format other than geojson, e.g.

mvt_decode < 1/2/3.mvt | mvt_encode --format mvt layer_name 1/2/3 > copy/1/2/3.mvt

1/2/3.mvt and copy/1/2/3.mvt will be the same.

ptpt avatar Nov 02 '17 14:11 ptpt

All sounds fine to me. And given that it's intended to be more of a debugging tool, I can see it being reasonable to skip having the encoder have the flexibility to accept input in multiple formats. But if you're keen on adding that, great! :)

rmarianski avatar Nov 03 '17 18:11 rmarianski

This is done. Two slight changes:

  1. only support geojson as input
  2. alteratively, you can encode all tiles in a range of zoom levels. The usage:
mvt_encode --directory tiles layer_name 0-5  < /tmp/s.json
2018-01-01 20:23:37,009 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/5/17/11.mvt
2018-01-01 20:23:37,014 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/5/16/11.mvt
2018-01-01 20:23:37,019 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/5/17/10.mvt
2018-01-01 20:23:37,024 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/5/16/10.mvt
2018-01-01 20:23:37,028 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/4/8/5.mvt
2018-01-01 20:23:37,033 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/3/4/2.mvt
2018-01-01 20:23:37,035 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/2/2/1.mvt
2018-01-01 20:23:37,037 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/1/1/0.mvt
2018-01-01 20:23:37,039 - INFO     - mapbox_vector_tile.command:301 - Wrote 1 feature(s) in tiles/0/0/0.mvt

ptpt avatar Jan 01 '18 12:01 ptpt