GmsCore
GmsCore copied to clipboard
microg
Mozilla Location Service is retiring
As was announced at https://github.com/mozilla/ichnaea/issues/2065, Mozilla will retire MLS soon. The final deadline for third parties seems to be set to June 12.
Apple does not allow us to access their location service. Also, in contrast to most Google APIs we're using, usage is not strictly required for interoperability, so we don't have a legal basis for using their API in most jurisdictions.
Unfortunately there is no natural successor I'm aware of. However there are also other open source projects (e.g. geoclue, the location service of Linux desktop) affected, so I'm hoping for some coordinated effort to bring a similar open source service to life.
Can we just continue to run MLS in some jurisdiction that isn't concerned about patents?
This would sort of kill microG for me, can we just use Apple's NLP? It was always more accurate anyway?
As I mentioned in the /e/ OS Community I hope something better than MLS is used because ever since microG went to 0.3.0 every app but maps (Magic Earth) is not accurate at all and is off by 50-100 miles. I agree with using Apple's NLP. Unless a good accurate open source NLP is found.
Even if we have to put Apple's NLP behind a radio button with a big disclaimer "Apple doesn't want you to do this, you're on your own"
-------- Original Message -------- On 14 Mar 2024, 12:53 pm, ToughDBlue wrote:
As I mentioned in the /e/ OS Community I hope something better than MLS is used because ever since microG went to 0.3.0 every app but maps (Magic Earth) is not accurate at all and is off by 50-100 miles. I agree with using Apple's NLP. Unless a good accurate open source NLP is found.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>
mentioned in the /e/ OS Community I hope something better than MLS is used because ever since microG went to 0.3.0 every app but maps (Magic Earth) is not accurate at all and is off by 50-100 miles. I agree with using Apple's NLP. Unless a good accurate open source NLP is found.
It is all about the data. The more users contribute data to the service, the more precise the results become.
mentioned in the /e/ OS Community I hope something better than MLS is used because ever since microG went to 0.3.0 every app but maps (Magic Earth) is not accurate at all and is off by 50-100 miles. I agree with using Apple's NLP. Unless a good accurate open source NLP is found.
It is all about the data. The more users contribute data to the service, the more precise the results become.
I understand but when I had microG 0.2.27 it all worked great. How about going back to that NLP method?
As I wrote above:
Apple does not allow us to access their location service. Also, in contrast to most Google APIs we're using, usage is not strictly required for interoperability, so we don't have a legal basis for using their API in most jurisdictions.
As a last resort option we could of course have that included and put a big warning that using it is a breach in ToS and Apple can sue your for financial compensation if you used that feature when trying to enable it, but if we can figure out somehting else I would definitely prefer that.
If you have problems with location in current versions of microG I suggest to enable local location learning in settings and then use an app that requests both GPS and network based location. As soon as you get a GPS fix, it would then remember the correct location of nearby wifi networks and use it instead of MLS, so MLS' data quality doesn't matter anymore.
As I wrote above:
Apple does not allow us to access their location service. Also, in contrast to most Google APIs we're using, usage is not strictly required for interoperability, so we don't have a legal basis for using their API in most jurisdictions.
As a last resort option we could of course have that included and put a big warning that using it is a breach in ToS and Apple can sue your for financial compensation if you used that feature when trying to enable it, but if we can figure out somehting else I would definitely prefer that.
If you have problems with location in current versions of microG I suggest to enable local location learning in settings and then use an app that requests both GPS and network based location. As soon as you get a GPS fix, it would then remember the correct location of nearby wifi networks and use it instead of MLS, so MLS' data quality doesn't matter anymore.
Understandable, don't want to breach any TOS. Didn't know that. I'd rather something else be figured out as well. I don't see where to enable local location learning in the microG settings. Can you recommend apps that request both GPS and network based location?
SatStat would do the trick. Also apps that use Google Maps API of microG often do request both locations through microG's API.
SatStat would do the trick. Also apps that use Google Maps API of microG often do request both locations through microG's API.
That works! Although my weather app says "Current Location" and not the actual city and state. The map in the weather is correct but again just says "Current Location" and not city and state. I have Nominatim enabled. Is there a way to reset Nominatim?
I feel that the loss of UnifiedNLP together with this makes for an unacceptable step back. @mar-v-in I know there were technical reasons behind dropping UnifiedNLP but consider that it was working fine or well enough not to notice the issues for many, including on newer versions of Android, and that there was a small but relatively healthy "ecosystem" of backends and not just Apple, for instance I was an avid user of offline backends, and the GSM Location backend wasn't limited to "learning" but could download cell databases for local use. That was amazing and given I use mostly open source applications, I do lose a lot of the usefulness of microG when network location goes away.
but could download cell databases for local use.
This was intended to be a feature in the new implementation as well (+ importing wifi and cell databases from file). With the learning code, there already is the logic to handle location data once it's in the database, we "just" lack the downloading/importing.
working fine or well enough not to notice the issues for many
I think there might be a bias here. As you said, you've been mostly working with offline backends, which are faster in processing and less likely to run into timeouts or similar issues than online backends.
I am convinced that the single process approach and centralized code will in the long run improve the results, but I do recognize that short term there are downsides as long as we lack some important features (with download/import of data being one of the major lacking features).
The way that I see it, one of the primary purposes for which people use their smartphones is navigation. Personally, I'd say navigating is a more common use case for a smartphone than voice calls.
If they don't have a fast, accurate location provider (ie if all they have is GPS), they can't navigate with their phone. The lock-on time is too high and it's not usable effectively indoors or in dense cities.
If people can't navigate with their phone on MicroG, they won't use MicroG on their phone. Losing the ability to provide navigation - via whatever means - isn't just a feature regression, it's an existential threat to MicroG usage.
It's important to do things the right way, but it's also important to make sure the project has a healthy enough base of users that it survives.
Does anyone know how accurate OpenCellID and other downloadable databases actually are? Maybe if their TOS allows we could add a menu to download their data (maybe on a country by country basis).
we "just" lack the downloading/importing.
@mar-v-in
Downloading can be worked around for some (I e.g. do that on my Linux machine, using FastLacellsGenerator, which includes MLS and OpenCellID, and then push the database to all my devices – also saves me from downloading and generating it multiple times). Easiest way would probably be supporting that SQLite database directly. Not sure if the corresponding Backend could be used stand-alone to download and generate the database on-device for those who need it – but having it supported would be a good first step IMHO.
Btw: I saw that TowerCollector now supports custom MLS URLs for upload, so there might be some "aggregators" for that, too. Not sure if they could be accessed the way microG currently accesses MLS, though. But an offline database is what some of us would definitely like to have anyway :stuck_out_tongue_winking_eye:
If people can't navigate with their phone on MicroG, they won't use MicroG on their phone.
Yeah I've been using microG now for a while, however having to mess about whenever attempting to order an Uber is becoming too much.
Sandboxed GMS is looking a lot better in terms of quality of life.
I really don't want to have to deal with downloading local towers etc, I just want a decent enough NLP provider (Apple or Google)
I vote for return UnifiedNLP.
This doesn't really solve the privacy issues; this would push people to Google and Apple for Location Services, weakening the privacy/FOSS projects.
I think there are some values in trying to create new tech for Location Services; whether it's open or closed.
One thing I suggested is hashing access point details to use as IDs instead of raw data. (for example, hash(mac_address, SSID, ...))
That way you can create a public database but have it be relatively private; someone revealing their SSID won't have the same effect on their location privacy as it usually has; of course there's other public databases like Wigle that can make this an issue, but they seem very bad and outdated that accuracy is... eh. As you would only really know the mac and SSID if you know where the network is anyway; of course you wouldn't hash individual properties as that doesn't really give you many benefits I think Signal's username implementation would be a good read for anyone implementing something like it. This would automatically deal with some issues like when someone changes SSIDs and location but not the router, etc.
The issue with such an approach rather than just publishing raw data, is that protocol ossification can be an issue. One solution to this is allow users to backup their contribution data and never delete it; so in the case of an upgrade to the network, one can re-upload new data.
I think something like this can also have value in a closed ecosystem, which would have better privacy, as in a map it is still very possible to do some analysis of contributed points and dates and things; I personally think that a closed system that uses a private protocol is better than a completely open, or than closed but not private to the operator; with a closed system with a private protocol, we could potentially move the data in a private way to other projects; and data analysis is harder.
TowerCollector is an interesting project, the maintainer is busy ATM and it's not his focus, so a fork to include WiFi, Bluetooth data, etc and be compatible with such a protocol would be needed.
This would be a lot of work and research; so it's very likely something like this isn't actually developed; it's still interesting to debate this, especially what improvements can be made to location services overall :P
Or someone self-hosted database server.
/e/foundation seems to be offering to do that (thankfully!)
This doesn't really solve the privacy issues; this would push people to Google and Apple for Location Services, weakening the privacy/FOSS projects.
It would push me to using the local databases I was using before UnifiedNLP was retired. Can't speak for others of course.
I also wonder: how many independent developers are probably going to want to be contributing to this repository, with PRs that must rightly not break things and be approved, as opposed to tinkering by writing their own module in their own repository, maybe using a simple existing module as a template, and publishing it there? I think that was a powerful aspect of the approach.
FYI, my plan for the next version is to add:
- Option for selecting a custom Ichnaea-compatible server. Given the interest of the community (microG, but also Linux Desktop and Linux Mobile), I consider it very likely that there will be a viable replacement soon. For people that don't care about privacy for this specific usecase, this means they can also use the Google Geolocation API, as it is compatible with Ichnaea.
- Functionality to contribute to the Ichnaea server being used. Essentially the data that is already used for the local database learning can also be submitted to the Ichnaea server at little additional cost. Of course this is Opt-In.
- Import and Export of the local database, for WiFi and Cells. This means you can use your previous databases, import dumps of cell databases and (given the necessary external tooling) you can upload the collected data to arbitrary services and thus have it survive a server change.
Import and Export of the local database
Will that include the cell database as used by LocalGSM – and maybe auto-detect if a fresh copy has been placed in a defined location (as LocalGSM does)? Idea is to easily distribute an existing database to multiple devices for offline use (in addition to or instead of online checks like Ichnaea).
Or someone self-hosted database server.
/e/foundation seems to be offering to do that (thankfully!)
And @rtsisyk offered the same on Mozilla's thread on this.
There doesn't seem to be any shortage of people/organizations willing to do the hosting.
FYI, all of us and a bunch of others are in communication with each other. We're aiming to ensure there is a single successor of MLS endorsed and supported by all the libre software projects involved, including microG
Relatedly, one issue I've had myself and seen many reports of with the internal UNLP implementation in recent GmsCore versions beyond the Mozilla db staleness:
Just enabling any of the Mozilla UNLP sources at all seems to "pollute" GPS results to the point that even with a good GPS lock geolocation is flaky or completely anomalous.
You can see this with SatStat on the map page where the blue UNLP "circle" and/or pin is constantly turning grey ("stale data") and/or flickering on/off constantly even when the device is stationary. Seems to me that a good GPS lock should override UNLP inputs especially if the UNLP inputs are extremely unstable.
I can open a separate issue on this if necessary. Just seems that this should be fixed along with enabling new MLS hosts, or else I think it will undermine the utility of those as well.
The network location provider is (and should be) independent of GPS and having a GPS fix and the results from GPS are not considered in it. The blue circle in SatStat displays the network location provider, the red circle shows the GPS location.
There is also the fused location, merged from both. This should use GPS once GPS is available and network location if not.
The flickering of the blue location is probably caused by the update interval. MicroG only provides an update if updated data exists, which due to rate limiting might be less often then SatStat's requested interval.
Perhaps my analysis of the problem was not correct.
But the observation that any enabling of the Mozilla UNLP sources often results in completely anomalous geolocation results is consistently demonstrable. I have talked to various people from all around the world that were getting results like placing them in the middle of the Atlantic ocean or something, until they disabled the Mozilla UNLP data sources in the microG utility.
After disabling the Mozilla switches, they got good geolocation results.
I actually just saw the Fused location provider in SatStat for the first time after acquiring a Pixel7Pro I installed CalyxOS on.
Prior to that, I had never seen that data source displayed in SatStat on my older devices at all. (Some of which are running older versions of LOS, some CarbonROM. All using Qualcomm SoCs.)
The fused provider is not available on all devices, that's why there is also a fused location API in Play Services that is also provided by microG; that's why some apps need microG to display locations, they use the Play Services fused location provider.
If erroneous network location data claims to be more accurate then GPS, it might be preferred over GPS data. I've never seen this as a relevant issue in practice, but it could explain the behavior you described. Indeed this could be improved in microG's fused provider by making low accuracy GPS still override high accuracy network location if the network location is off by too far from the GPS.