pbf2json icon indicating copy to clipboard operation
pbf2json copied to clipboard

rewrite node encoding for clarity and ease of extension

Open missinglink opened this issue 3 years ago • 0 comments

this PR rewrites some older code in a clearer way, which opens it up for extension. in particular issue https://github.com/pelias/pbf2json/issues/96 could benefit from this refactor

some notes on what this part of the code does:

  • when 'node' elements from OSM are stored in leveldb they must be encoded/decoded from a slice of bytes []byte{}
  • we use a custom encoding scheme to save disk space
  • for the lat/lon values, we convert them to their IEEE 754 binary representation and truncate them to 6 bytes each, which is a nice tradeoff of precision vs. size (this consumes 12 bytes total)
  • we optionally use a 13th byte to store metadata about entrances and accessibility
  • there are still 4 bits reserved in that 13th byte for extension (such as storing information about the node role in a relation)

the subject of this PR is mainly to make the operations on the metadata byte more obvious and open them up for extension.

I didn't add new tests since https://github.com/pelias/pbf2json/blob/master/encoding_test.go seems to cover this sufficiently

Hey @paulmach if you get a second could you please do a quick review?

missinglink avatar Jul 21 '21 08:07 missinglink