rally icon indicating copy to clipboard operation
rally copied to clipboard

Add support for vector tiles API

Open iverase opened this issue 2 years ago • 6 comments

It would be great if we could add operations in rally that points to the vector tiles API. It currently fails because the result of the query is a protbuffer that the system does not understand.

iverase avatar Jan 13 '23 08:01 iverase

The main blocker for this issue is migrating to version 8.x of the Python client in https://github.com/elastic/rally/issues/1350 as version 7.14 of the Elasticsearch Python client does not support that API. Support was added in 7.15.0: https://www.elastic.co/guide/en/elasticsearch/client/python-api/7.17/release-notes.html#rn-7-15-0. This API is quite specific in that it returns binary data, so it could make sense to wait for the upgrade, which hopefully should be done in March.

pquentin avatar Jan 18 '23 12:01 pquentin

@craigtaverner @iverase We released Rally 2.8.0 last month with the 8.x client. Are you still interested in doing that?

pquentin avatar Jul 07 '23 07:07 pquentin

Yes we are super interested, need to find the time (and in my case, some python skills :))

iverase avatar Jul 07 '23 08:07 iverase

Can you maybe write how such an operation would look in a Rally track, so that we can see how difficult it would be to implement? Some runners are only a few lines of code.

pquentin avatar Jul 07 '23 09:07 pquentin

A mvt request is very similar to a search request except it requires 4 mandatory parameters. One is the field where the geo data is, and the other three is the actual tile to be processed by providing the z/x/y values.

Therefore a simple example would look like:

 {
      "name": "mvt-search",
      "operation-type": "mvt",
       "z" : 11,
       "x" : 23,
       "y" : 145,
       "field": "my-geo-field",
      "body": {
           -- extra configuration here 
      }
    }

iverase avatar Jul 11 '23 00:07 iverase

An example of how this would look like in the geoshape track is:

{
      "name": "mvt-search-hits",
      "operation-type": "mvt",
      "index": "osm*",
       "field": "shape",
       "z" : 2,
       "x" : 2,
       "y" : 1,
       "field": "my-geo-field",
      "body": {
           "grid_precision" : 0,
           "sort": [
               {
                 "size": {
                  "order": "desc"
                }
              }
           ]
      }
    }

or another example:

{
      "name": "mvt-search-grid",
      "operation-type": "mvt",
      "index": "osm*",
       "field": "shape",
       "z" : 2,
       "x" : 2,
       "y" : 1,
      "body": {
           "size" : 0,
            "grid_agg" : "geo_hex"
      }
    }

iverase avatar Jul 12 '23 03:07 iverase