interpolation
interpolation copied to clipboard
WIP: Use Libpostal service
Looking for early review, not 100% production ready yet
Use microservice-wrapper to avoid having to load libpostal locally.
~Note: this now requires a new configuration section in pelias.json
, a
top-level services
key with the usual properties. Here's an example
full pelias.json
:~
Update: The code will fall back to api.services.libpostal
if services.libpostal
is not defined. This helps ensure backwards compatibility.
{
"api": {
"textAnalyzer": "libpostal"
},
"services": {
"libpostal": {
"url": "http://libpostal-service-url:8080",
"timeout": 4000
}
}
}
Notes
This is a breaking change, since a new configuration option is required.
~The deasync module was chosen to give the libpostal service a synchronous-like interface. We should carefully evaluate whether there are any issues with this. We can avoid using it, however it would be quite a bit of work, as many sections of the code, including lots of unit tests, would have to be re-written in a callback style.~
See update below, this code has now been rewritten in an asynchronous style.
Fixes: https://github.com/pelias/interpolation/issues/106
This has now been rewritten to use Promises, async
/await
, and asynchronous interfaces in general. It's a lot of changed code, and I'm actually not an expert in using the new asynchronous tools in newer Javascript in production. In practice I think we'll need more try/catch blocks or general error handling for asynchronous code failures. I think even something like a single intermittent network glitch could crash the service right now.
However this should be a big change as the interpolation service will take only a few MB of RAM, and is thus much easier and cheaper to deploy.
Turns out there is some more work to do:
- We will probably want to add some logging of structured data regarding each request to libpostal, including the query that was sent and how long it took to get a response
- In to record response time information,
pelias-microservice-wrapper
exposes ametadata
object as a 3rd parameter returned to callback functions. This makes it a bit harder to useutil.promisify
, so sticking with a standard callback architecture where possible might be best - Finally, while doing some more testing of the interpolation service it turns out that the demo is broken because the
/extract/geojson
endpoint was not converted to use the libpostal service. It's a challenging endpoint that needs to normalize multiple street names and therefore can generate multiple calls to the libpostal service, so this will take some work.