Option to not cache empty tile respones

[sheinbergon]

Hi all

We are using Tegola in front of PostGIS, and encounter empty tiles, from time to time.

While it can be debated whether or not empty tiles should be cached or not in principle (personally, I don't think an app should cache not found responses), we did encounter scenarios where the cache contained empty stale entries (tiles containing empty responses) and purging it manually was the only way to reacquire valid tiles

Looking at the configuration, I couldn't find an option to avoid caching for empty responses.

We tried using both the postgis and mvt_postgis provider (with latter supposedly bypassing tegola's MVT generation layer, to the best of my understanding), but we're still caching empty tiles.

Is there a way to configure/achieve this behavior?

[ARolek]

@sheinbergon I would need to look into what PostGIS returns when a tile is empty. I'm not sure if they:

1. return NULL1
2. return 0 bytes
3. return the tile layer encoded but with 0 geometries

Option 1 and 2 could have a check added and we could make this configurable. I'm not sure how we would know if a tile is empty if scenario 3 is the case.

While it can be debated whether or not empty tiles should be cached or not in principle (personally, I don't think an app should cache not found responses),

This also depends on how you're using tegola. If you pre-seed all your tiles you might want the empty tiles so things play nice with client side libraries. IIRC, the reason tegola handles "not found" tiles the way it does is because client side libraries were not expecting 404 responses from the server. Are you encountering a different scenario? What UI library are you using?

[sheinbergon]

@ARolek thank you for responding.

At least when using mvt_postgres PostGIS returns an empty response,Looking at what the query does, it results in empty response, so it should be 2

I believe Tegola just takes the empty results and transforms it to an empty FeatureCollection. By the time it reaches the caching middleware response it's just a byte array, so we should probably add some sort of an empty tile indication and avoid caching based on it

As for your second question, we are using (for this particular use-case) mapbox. Just to clarify, I didn't say we should return null responses to the client, only that I want to have the option to not cache that in case it represents an empty vector tile ( no features)

[iwpnd]

I believe Tegola just takes the empty results and transforms it to an empty FeatureCollection.

tegola does not transform anything here. it stores the the tile data it receives from postgis. we could check the length of it and optionally avoid caching but still service those empty ones to avoid errors. That however would mean that consecutive calls will perform database lookups. I would like to avoid that, so it would definitely have to be optional/default:false.

[sheinbergon]

@iwpnd That sounds like a great approach. Opt out by default is also good.

[ARolek]

That however would mean that consecutive calls will perform database lookups

This is a key point. By non caching the "empty" tile, you're increasing the database load.

@sheinbergon can you elaborate a bit more on your situation? Did tegola cache an "empty" tile, but then later your dataset had data in that area and it wasn't be refetched?

[sheinbergon]

@ARolek we're well aware of it.

What you mentioned is exactly our use case - we have a completely static database we want to serve MVT tiles from, and we encountered some scenarios of stale cache, empty tiles on different zoom levels contained no data, and only deleting from our Redis cache resolved the issue.

While we could have very long discussion about debugging the cause of such behaviors/issues, it'd be much simpler to just ignore them empty tiles

[ARolek]

@sheinbergon you mentioned redis cache. Have you considered just using the TTL in redis to expire the tiles? https://github.com/go-spatial/tegola/tree/master/cache/redis#properties

[sheinbergon]

@ARolek, yes we know Redis has this option. However:

• We'll be increasing the database load to a much greater degree, depnding on the actual TTL (longer times, means users my experience absent tiles longers).
• We may want to move away from Redis, as it's costly
• Area of intersets usually display contigues populated tiles blocks. We have no intreset in expiring those, only cases we get zero feature tiles, so database load is not expected to increase by much.

Honestly, going that path kind of feels like using a 5 KG hammer to drive a nail through a wooden board, that's why we avoided it thus far

[iwpnd]

we have a completely static database we want to serve MVT tiles from, and we encountered some scenarios of stale cache

this is odd, or I don't understand you correctly. if the dataset is static, why would you have issues with empty tiles? either they contain the data of your static dataset, or your dataset does not provide data for them. in that case there's an issue with tegola i would rather before anything else. :)

[sheinbergon]

While I agree with you, I was just looking for a simple fix to work around these Tegola issues. We wound up disabling cache all together

[ARolek]

We wound up disabling cache all together

Nice! How's this working out so far? Are you using a CDN at all for short lived caching?