api
api copied to clipboard
Increase `focus.point` weight
Hi there, this is the PR for #1567
I kept in mind what @orangejulius says in https://github.com/pelias/api/issues/1567#issuecomment-944391612:
Funnily enough, right now we can have two problems that are almost opposites, at the same time:
- nearby, exact text matches can score below a far away populated/popular place that is a poor text match
- distant, exact text matches for popular places can score below a nearby place with a good text match
I did several tests to find out what would be the best configuration.
Results with population and popularity boost
I started by decreasing population and popularity boosts as I said in my issue. I tried with three values, 8, 10 and 12 and ran acceptance tests (issues are from autocomplete_acushnet_antiques.json
and autocomplete_focus.json
).
With boost decreased to 10 for popularity and population I had 2 improvements, 1 regression and 2 changes.
Improvements
/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antiq
before | after |
---|---|
Antique, Philippines (14020.113km) | Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :heavy_check_mark: |
Province of Antique, AQ, Philippines (14008.427km) | Antiques, Tiverton, RI, USA (23.22km) |
Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :x: | Seaview Antiques, Plymouth, MA, USA (40.358km) |
Antiques, Tiverton, RI, USA (23.22km) | Larchmere Antique District, Cleveland, OH, USA (888.459km) |
Seaview Antiques, Plymouth, MA, USA (40.358km) | Province of Antique, AQ, Philippines (14008.427km) |
Larchmere Antique District, Cleveland, OH, USA (888.459km) | Antique, Philippines (14020.113km) |
Armory Antique Marketplace, Newport, RI, USA (37.73km) | Armory Antique Marketplace, Newport, RI, USA (37.73km) |
Antiquarian House, Plymouth, MA, USA (40.241km) | Antiquarian House, Plymouth, MA, USA (40.241km) |
Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) | Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) |
Bridge Antiques, Weymouth Town, MA, USA (65.747km) | Bridge Antiques, Weymouth Town, MA, USA (65.747km) |
/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antique
before | after |
---|---|
Antique, Philippines (14020.113km) | Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :heavy_check_mark: |
Province of Antique, AQ, Philippines (14008.427km) | Antiques, Tiverton, RI, USA (23.22km) |
Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :x: | Seaview Antiques, Plymouth, MA, USA (40.358km) |
Antiques, Tiverton, RI, USA (23.22km) | Larchmere Antique District, Cleveland, OH, USA (888.459km) |
Seaview Antiques, Plymouth, MA, USA (40.358km) | Province of Antique, AQ, Philippines (14008.427km) |
Larchmere Antique District, Cleveland, OH, USA (888.459km) | Antique, Philippines (14020.113km) |
Armory Antique Marketplace, Newport, RI, USA (37.73km) | Armory Antique Marketplace, Newport, RI, USA (37.73km) |
Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) | Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) |
Bridge Antiques, Weymouth Town, MA, USA (65.747km) | Bridge Antiques, Weymouth Town, MA, USA (65.747km) |
Genuine Antique Lighting, Boston, MA, USA (77.334km) | Genuine Antique Lighting, Boston, MA, USA (77.334km) |
There is no doubt, this is an improvement
Regressions
/v1/autocomplete?focus.point.lat=40.744131&focus.point.lon=-73.990424&text=San Francisco
before | after |
---|---|
San Francisco, CA, USA (4135.156km) :heavy_check_mark: | South San Francisco, CA, USA (4132.782km) |
South San Francisco, CA, USA (4132.782km) | San Francisco, QZ, Philippines (13779.823km) |
City of San Francisco, San Francisco, CA, USA (4136.075km) | South Beach, San Francisco, CA, USA (4131.75km) |
San Francisco County, San Francisco, CA, USA (4136.075km) | San Francisco Gotera, MO, El Salvador (3308.524km) |
Cuautitlán Izcalli, MEX, Mexico (3340.891km) | San Francisco Cahuacúa, OAX, Mexico (3474.854km) |
San Francisco de Macorís, DU, Dominican Republic (2414.237km) | San Francisco del Valle, OC, Honduras (3270.844km) |
Campeche, Mexico (2807.209km) | San Fco. Del Norte, CI, Nicaragua (3308.078km) |
S. Francisco del Rincon, GUA, Mexico (3424.249km) | San Francisco, CA, USA (4135.156km) :x: |
San Francisco, AS, Philippines (14187.026km) | San Francisco La Unión, QZ, Guatemala (3338.114km) |
San Francisco, QZ, Philippines (13779.823km) | San Francisco Zapotitlán, SU, Guatemala (3371.286km) |
This one is very weird, South San Francisco
took the place of San Francisco
...
Change
/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New York
before | after |
---|---|
New York County, NY, USA (47.631km) | New Castle, NY, USA (24.692km) |
New York City, Manhattan, New York City, NY, USA (54.514km) | New Rochelle, NY, USA (25.49km) |
City of New York, Brooklyn, NY, USA (54.878km) | New Salem, Port Washington, NY, USA (28.565km) |
New Rochelle, NY, USA (25.49km) | New City, NY, USA (39.498km) |
New York, NY, USA (54.989km) :o: | New Cassel, NY, USA (32.589km) |
City of New Rochelle, NY, USA (24.928km) | New Square, NY, USA (42.21km) |
New Castle, NY, USA (24.692km) | Town of New Castle, NY, USA (24.424km) |
New City, NY, USA (39.498km) | City of New Rochelle, NY, USA (24.928km) |
East New York, Brooklyn, New York, NY, USA (51.821km) | New York County, NY, USA (47.631km) |
Town of New Castle, NY, USA (24.424km) | North New Hyde Park, NY, USA (36.402km) |
Here the new position of New York City is 42 ! This may be considered as a regression, that's why I will not chose this configuration.
/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New York, NY
before | after |
---|---|
New York County, NY, USA (47.631km) | New York County, NY, USA (47.631km) |
New York, NY, USA (54.989km) :o: | East New York, Brooklyn, New York, NY, USA (51.821km) |
New York City, Manhattan, New York City, NY, USA (54.514km) | New York, NY, USA (54.989km) :x: |
City of New York, Brooklyn, NY, USA (54.878km) | New York City, Manhattan, New York City, NY, USA (54.514km) |
East New York, Brooklyn, New York, NY, USA (51.821km) | City of New York, Brooklyn, NY, USA (54.878km) |
Deutsche Schule New York, White Plains, NY, USA (16.891km) | Deutsche Schule New York, White Plains, NY, USA (16.891km) |
Keio Academy of New York, Purchase, NY, USA (15.338km) | Keio Academy of New York, Purchase, NY, USA (15.338km) |
Port Chester, New York 10573, Port Chester, NY, USA (12.209km) | Port Chester, New York 10573, Port Chester, NY, USA (12.209km) |
NYLO New York City, New York, NY, USA (47.886km) | NYLO New York City, New York, NY, USA (47.886km) |
New York Mills, NY, USA (270.079km) | New York University, New York, NY, USA (52.843km) |
Here New York is loosing one place, but still good enough. At this point I decided to change my mind and increase focus point instead of decreasing population/popularity.
Results with focus point increase weight
So now I want to have the same scale between focus point boosts and population/popularity, so I set the weight of focust point to 20. With this configuration I had 2 improvements, 0 regression :tada: and 1 change.
Improvements
/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antiq
before | after |
---|---|
Antique, Philippines (14020.113km) | Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :heavy_check_mark: |
Province of Antique, AQ, Philippines (14008.427km) | Antique, Philippines (14020.113km) |
Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :x: | Province of Antique, AQ, Philippines (14008.427km) |
Antiques, Tiverton, RI, USA (23.22km) | Antiques, Tiverton, RI, USA (23.22km) |
Seaview Antiques, Plymouth, MA, USA (40.358km) | Seaview Antiques, Plymouth, MA, USA (40.358km) |
Larchmere Antique District, Cleveland, OH, USA (888.459km) | Armory Antique Marketplace, Newport, RI, USA (37.73km) |
Armory Antique Marketplace, Newport, RI, USA (37.73km) | Antiquarian House, Plymouth, MA, USA (40.241km) |
Antiquarian House, Plymouth, MA, USA (40.241km) | Larchmere Antique District, Cleveland, OH, USA (888.459km) |
Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) | Bridge Antiques, Weymouth Town, MA, USA (65.747km) |
Bridge Antiques, Weymouth Town, MA, USA (65.747km) | Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) |
/v1/autocomplete?focus.point.lat=41.652889&focus.point.lon=-70.922898&text=antique
before | after |
---|---|
Antique, Philippines (14020.113km) | Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :heavy_check_mark: |
Province of Antique, AQ, Philippines (14008.427km) | Antique, Philippines (14020.113km) |
Acushnet River Antiques Showroom, New Bedford, MA, USA (0km) :x: | Province of Antique, AQ, Philippines (14008.427km) |
Antiques, Tiverton, RI, USA (23.22km) | Antiques, Tiverton, RI, USA (23.22km) |
Seaview Antiques, Plymouth, MA, USA (40.358km) | Seaview Antiques, Plymouth, MA, USA (40.358km) |
Larchmere Antique District, Cleveland, OH, USA (888.459km) | Armory Antique Marketplace, Newport, RI, USA (37.73km) |
Armory Antique Marketplace, Newport, RI, USA (37.73km) | Larchmere Antique District, Cleveland, OH, USA (888.459km) |
Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) | Bridge Antiques, Weymouth Town, MA, USA (65.747km) |
Bridge Antiques, Weymouth Town, MA, USA (65.747km) | Cobwebs Antiques & Jewelry, Boston, MA, USA (75.016km) |
Genuine Antique Lighting, Boston, MA, USA (77.334km) | Genuine Antique Lighting, Boston, MA, USA (77.334km) |
There is no doubt, this is still an improvement
Regressions
No regressions :heavy_check_mark:
Change
/v1/autocomplete?focus.point.lat=41.05343&focus.point.lon=-73.53873&text=New York
before | after |
---|---|
New York County, NY, USA (47.631km) | New York County, NY, USA (47.631km) |
New York City, Manhattan, New York City, NY, USA (54.514km) | New Rochelle, NY, USA (25.49km) |
City of New York, Brooklyn, NY, USA (54.878km) | New York City, Manhattan, New York City, NY, USA (54.514km) |
New Rochelle, NY, USA (25.49km) | City of New York, Brooklyn, NY, USA (54.878km) |
New York, NY, USA (54.989km) :o: | City of New Rochelle, NY, USA (24.928km) |
City of New Rochelle, NY, USA (24.928km) | New York, NY, USA (54.989km) :x: |
New Castle, NY, USA (24.692km) | New Castle, NY, USA (24.692km) |
New City, NY, USA (39.498km) | New City, NY, USA (39.498km) |
East New York, Brooklyn, New York, NY, USA (51.821km) | East New York, Brooklyn, New York, NY, USA (51.821km) |
Town of New Castle, NY, USA (24.424km) | Town of New Castle, NY, USA (24.424km) |
Here New York is loosing one place, but still good enough.
Resume
So instead of decreasing population and popularity boosts, I chose to increase focus point weight. The overall result seems good and close to current responses.
Results are also better in my initial test with Vars
(the lang is important here).
/v1/autocomplete?lang=fr&focus.point.lat=48.03661925338169&focus.point.lon=6.580299229512&text=vars
before | after |
---|---|
Varsovie, MZ, Pologne (1127.93km) | Varsovie, MZ, Pologne (1127.93km) |
Finlande propre, Turku, Finlande propre, Finlande (1711.878km) | Finlande propre, Turku, Finlande propre, Finlande (1711.878km) |
Varsovie, Johns Creek, GA, USA (7334.865km) | Ham-sous-Varsberg, France (127.305km) |
Ham-sous-Varsberg, France (127.305km) | Varsovie, Johns Creek, GA, USA (7334.865km) |
Varsity Center, Lehigh Acres, FL, USA (7732km) | Varsity Center, Lehigh Acres, FL, USA (7732km) |
Varsity Lakes, QLD, Australie (16424.897km) | Varsity Lakes, QLD, Australie (16424.897km) |
Varsovie, Throop, PA, USA (6225.986km) | Varsberg, France (126.335km) |
Varchets, MT, Bulgarie (1404.336km) | Vars, France (96.218km) :o: |
Obshtina Varshets, MT, Bulgarie (1406.591km) | Varsovie, Throop, PA, USA (6225.986km) |
Varsberg, France (126.335km) | Varchets, MT, Bulgarie (1404.336km) |
:memo: TODO: write acceptance test for Vars
related #1567
Wow, quite comprehensive testing. We'll run this through our own tests as well and report back. Thanks for taking the time to try out lots of options.
The South San Francisco
situation is something I've looked at in the past, and unfortunately it's a result of the issue with scoring multiple alt names. San Francisco (as expected) has more alt-names than South San Francisco.
- https://github.com/pelias/pelias/issues/862
So hopefully we can improve things even more someday when we fix scoring issues.
Hey this looks pretty good!
We tested it out and there don't really look to be any unexpected effects. Queries with a focus point do now prefer things closer to the focus point by a tiny bit. The acceptance tests provide the best examples, especially with the Union Square tests that have always been challenging to make pass:
The flip side is that some city/locality/high population results come up lower. Both the following examples are only slight regressions but worth looking more closely at.
Also NYC and SF are hard queries for various reasons (aliases, duplicate names, etc), so while they're obviously very prominent examples, if they are the only things affected then that's not so bad.