geocoder
geocoder copied to clipboard
alexreisner
Add support for batch geocoding
Some providers (such as Esri) support an API method for batch geocoding. In the case of Esri, very different rate limits apply, so we can use this to geocode a large number of records without worrying about hitting HTTP rate limits.
I plan on using the Esri batch geocoding API, and thought it would be a good thing to add to this gem. Any thoughts on whether a PR adding support for that would be accepted?
Generally speaking, I'm open to adding this feature. Would it be added through a rake task? It seems like it might be neat if the existing batch geocoding rake task was modified to detect whether the configured lookup supports batch geocoding, and took advantage of the feature if available.
well I plan on using this without any of the model support, so I will need to either add support for it to Geocoder.search or make a new method like Geocoder.batch. I do like the idea of the Rake task using batch geocoding if it's configured, although that isn't something I will be using.
I would prefer it to be Geocoder.batch then adding complexity to .search
On Monday, May 16, 2016, Aaron Parecki [email protected] wrote:
well I plan on using this without any of the model support, so I will need to either add support for it to Geocoder.search or make a new method like Geocoder.batch. I do like the idea of the Rake task using batch geocoding if it's configured, although that isn't something I will be using.
— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/alexreisner/geocoder/issues/1044#issuecomment-219547696
I've made some progress on this here: #1046 I'd appreciate a quick 👍 if I'm on the right track with how others imagine this feature working.
@alexreisner I just discovered that - at least for Google Maps -, batch geocoding already works by passing an array of street addresses: Geocoder.search(["1 abc street, foo-town", "2 xyz stree, bar-town", "..."]). It works because the method accepts an array, in order to support an array of lat/lon strings (-> a lat/lon-pair). It then concats these strings and makes the query for multiple street addresses in one request. Judging from the comments in the code, this was not intentional (?). But we actually appreciate that feature, as it improves our code performance significantly. Is there any way to keep the existing functionality of the code (such that you can continue using the search method with an array), but improve the structure such that it becomes apparent that batch searching is possible (e.g. using the options)? Hope this doesn't sound like a bunch of nonsense...
Haha! @henry-p you're right that wasn't intentional, but I'm glad it's working for you. However, I can't reproduce it on my end, and I wasn't aware Google provided that service. Could you post a sample response? (I get a 400 Bad Request when I try it.)
Any updates on this feature? I noticed that Bing Maps has a batch option too and want to access it. https://docs.microsoft.com/en-us/bingmaps/spatial-data-services/geocode-dataflow-api/?toc=%2Fen-us%2Fbingmaps%2Fspatial-data-services%2FTOC.json&bc=%2Fen-us%2FBingMaps%2Fbreadcrumb%2Ftoc.json
It's a bit tricky as it seems to be an async thing where you have to create a geocoding batch job, then poll for the status of completion of the job. What are peoples thought on this being in this gem?
Google reference this: https://developers.google.com/maps/documentation/javascript/geocoding
and states:
The per-session rate limit prevents the use of client-side services for batch requests, such as batch geocoding. For batch requests, use the Geocoding API web service.
However, when you go to the Geocoding API webservices page, I see no reference to batch processing. The above sentence infers that you can to batch processing. Any ideas?
@joshm1204 it's a good question. I haven't been able to find any more specific info. But as @henry-p mentions above, it seems to already work if you pass an array of addresses.
When I pass an array I get
Geocoding API error: 400 Bad Request Google API error: invalid request (Invalid request. Invalid 'latlng' parameter.).
So if it did work, it no longer works