tegola icon indicating copy to clipboard operation
tegola copied to clipboard

Rounding/encoding issue with numeric fields in postgis provider

Open russss opened this issue 3 years ago • 3 comments

I have been putting off investigating this for a while, and it appears I've found a fix for my use case, but I thought I should report this as an issue.

Consider the following test case:

tegola_rounding_test=# select * from test;
 id |                        geom                        |  output
----+----------------------------------------------------+----------
  1 | 0101000020110F0000014630D227F215C14908E08BD9C15C41 | 0.045080
(1 row)
Full test config Schema:
create table test(id serial, geom geometry(Geometry,3857), output decimal(10, 6));
insert into test(id, geom, output) values(1, ST_Transform(ST_SetSRID(ST_GeomFromText('POINT(-3.23 55.9)'), 4326), 3857), 0.045080);

config.toml:

[[maps]]
name = "map"
[[maps.layers]]
min_zoom = 2
max_zoom = 17
provider_layer = "postgis.test"

[[providers]]
max_connections = 20
host = "localhost"
name = "postgis"
database = "tegola_rounding_test"
srid = 3857
type = "postgis"
port = 5432
user = "russ"
password = ""

[[providers.layers]]
name = "test"
id_fieldname="id"
geometry_type = "Point"
sql = "SELECT id, ST_AsBinary(geom) AS geom, output AS output FROM test WHERE geom && !BBOX!"

[webserver]
port = ":8081"
cors_allowed_origin = "*"

When rendered (using the postgis provider), the "output" field shows an incorrect, long repeating fractional part (this is especially annoying because Mapbox/Maplibre GL doesn't have configurable rounding): image

This does not happen with the mvt_postgis provider, so my suspicion is that it's either an issue with Tegola itself or with the golang protobuf library.

When I last looked at this a while back (there is a little more info in openinframap/styles#34), I came to the conclusion that there was a single-bit error in the protobuf encoding of the value. This doesn't happen with all decimal values, just a few of them.

I think my solution to this will be to switch to mvt_postgis, but I thought I should file an issue in case anyone else hits this annoying bug.

russss avatar Mar 13 '21 12:03 russss

@russss thanks for the report. I agree with your assessment. This is probably an issue in the Postgres driver or the photobuff library. The Postgres driver is due for an update soon, which may address this problem. I appreciate you documenting it.

ARolek avatar Mar 15 '21 22:03 ARolek

An update to this: it turns out that this does also happen with the mvt_postgis provider.

I thought this was fixed because my columns were of the NUMERIC type in Postgres, and the mvt_postgis provider outputs NUMERIC values as strings in the resulting MVT. (Either the native Tegola MVT support outputs NUMERIC as MVT floats, or I changed the data type around the same time.)

So it seems unlikely this is a Tegola bug after all, although that does make me wonder what is causing it...

russss avatar Sep 12 '21 18:09 russss

@russss are the results the same with mvt_postgis then? It's strange (and annoying) to have the precision getting messed up like this.

ARolek avatar Sep 14 '21 20:09 ARolek