feature: patch header/metadata from JSON

[bdon]

define a JSON schema like

{
  'tile_type':0,
  'metadata':{
    'my_key':'my_value'
  }
}

to allow modification of pmtiles by writing a new file.

pmtiles show --header-metadata-json would also be useful to output this.

[dvd3v]

Is it always necessary to write to a new file? Given that PMTiles can become quite large, updating the metadata in place would be beneficial.

[bdon]

Yes, same as flatgeobuf, parquet or video files, there is no extra space left in the file.

[spatialillusions]

@bdon Since you sort of mentioned it in https://github.com/protomaps/basemaps/issues/160#issuecomment-1784156935, it would be really handy if I just could push all my extra data into the metadata json in a simple way.

[bdon]

the command would be something like pmtiles metadata FILE.pmtiles --write metadata.json where you provide whatever metadata.json you need.

pmtiles show --metadata FILE.pmtiles > metadata.json
pmtiles metadata FILE.pmtiles --write metadata.json # this is a noop

It would be fast+atomic but require temporary disk space as large as the new file, since it writes a completely new file.

[bdon]

an open question is whether we ever want to expose the ability to modify the header (non-metadata parts) - I guess if you accidentally set the wrong bounds or tile type, or want to change the center position/zoom?

[spatialillusions]

I vote for just metadata for now. Maybe a pmtiles header XXXX command in the future for other parts of the header.

[bdon]

Step one is making pmtiles show --metadata FILE write just the raw JSON to stdout:

https://github.com/protomaps/go-pmtiles/pull/98

[AndrewJDR]

I have a question related to this -- Is it currently possible to add additional metadata to the output pmtiles file when running pmtiles convert input.mbtiles output.pmtiles?

If this is not currently possible, I assume this would be a bit easier than an in-place method that works against existing pmtiles files?

[AndrewJDR]

(~~Incidentally, I'm not aware of any command line tool that can just patch the metadata of an existing mbtiles file, which would also solve this problem. If anyone is aware of something like this, please do let me know~~) Edit: I suppose the answer is to just use sqlite directly. Sorry for the slight thread hijacking. Disregard :D

[bdon]

Yeah I would say use the sqlite3 CLI to modify the MBTiles. there's no equivalent with the pmtiles CLI yet.

[bdon]

Needing this now, currently plan:

pmtiles show foo.pmtiles --header-json > header.json

Writes a JSON representation of the non-offset parts of the header to a file.

pmtiles show foo.pmtiles --metadata > metadata.json

Writes the JSON metadata to a file.

pmtiles write-header old.pmtiles header.json new.pmtiles (or maybe new.pmtiles first?)

Write the JSON representation of the header into a new archive. (maybe we need --inplace where it atomically deletes and renames?) All fields must be specified.

pmtiles write-metadata old.pmtiles metadata.json new.pmtiles (or maybe new.pmtiles first?)

Write the JSON representation of the header into a new archive. (maybe --inplace) ?