UnifiedNlp
UnifiedNlp copied to clipboard
microg
Provider Priority?
Hi, I do not know if this is already implemented and it is hard to test, so I just write my suggestion here. Ignore it if you wish. What I would like is to have some less accurate providers still be able to cancel more accurate ones if the data differs.
I had the problem with the Google provider, that someone in the Netherlands used the same MAC as my router. Google would always place me in the Netherlands when I was at home and had no GPS signal. But I am in Austria and had GSM reception. There just is no way to be connected to an Austrian GPS network and be in the Netherlands.
So in my opinion, while the exact location of the GSM provider is less accurate, at least the country is more accurate and it would be great to use this to filter wrong results. This wrong data can not only happen with cloned MAC but also when wifi aps are moved which is not uncommon today,
I'm not sure if backend priorities would solve the problem
- GSM can be used to pin down the country (MCC, low resolution, perfect confidence)
- GSM can also be used to triangulate (high accuracy, good confidence)
- GSM can use your current tower to estimate your location (medium accuracy, medium confidence)
- GSM has the notion of local area codes, which can be used to estimate your location (low accuracy, good confidence)
- WIFI can use your current AP to estimate your location (very high accuracy, low confidence)
- WIFI can triangulate your location based on a scan (perfect accuracy, high confidence)
- IP addresses can be used to pin down the country (low resolution, high confidence)
- IP addresses can be used to pin down your city (medium accuracy, medium confidence)
- GPS, once you have a fix, is perfect.
As one can see accuracy and confidence are to different things. And not related to a backend but rather to the exact algorithm used within the backend....
The confidence should be posted with every location report. It might be enough to have 2-4 fixed confidence levels. Updates would be assumed to be as accurate as possible, but of lowest confidence (if not specified otherwise). Reports and updates may not contradict reports of higher confidence. A backend would fire reports for every confidence level it reaches.
Alternately one could export multiple backends (E.g. GSMCountry, GSMArea, GSMTower, Wifi, NetworkCountry, NetworkCitiy, ...). This would simplify the receive / send, but it would still require some notion of confidence.
Just my 0.02¢
PS: There would be some more open question, e.g. what to do if 2 providers contradict each other at the same confidence level. I hope this will never happen, but I'm sure it will....
Currently the more up-to-date and more accurate backend is used.
I planned to add a more fine-grained priority system, but i'm still not sure how and what, because (as @rtreffer pointed out) one can't only go for accuracy. Adding something like confidence to a reported location could help out here too, but is not solving the problem, because there might be several other things users might be interested (eg. if you have two possible backends for wifi based locations using different online databases, a user might want to only ask the second backend, if the accuracy of the first is worse then 100m).
Another interesting feature could be to allow each backend to see the last reported locations of the other backends. This way a wifi backend can look at gsm backends to know if there location is possible.
Just a final note to @J0s3f's problem with moving/changed wifi macs: It is a de-facto industry standard to only assume a wifi-based location to be valid if it is based on atleast two wifis. This solves your problem in more then 90% of cases, but not always (sometimes multiple wifis change location)
I think before 1.1.2, with the previous settings US, there was a possibility to rearrange the orders in which backends were listed (and, I assumed, used). This seems to be gone now; is that intentional? Was rearranging them never really meaningful?
Rearranging backends was indeed possible in UI but had no influence. Sorry if this caused confusion.
I would like to second this enhancement. I would like to see a feature where the online providers are only asked if the offline providers find no location. This is more from the privacy point of view, but I think it fits here. I guess it should be pretty to implement, but it would change the API.
I agree with @dicer concerning online providers – especially I'd like to keep Google location service out of the game entirely when running on a "stock device" with GApps on. If I understand correctly (and I see no other options in settings or the app), to use UnifiedNLP I have to turn Network location on in Android settings. And I can chose which UnifiedNLP backends I wish to use. But nowhere I find an option whether I wish to use Google here. Is it even possible to (optionally) turn it off? I somehow have the feeling that if it were, I'd find a switch in the configs …
I think using UnifiedNLP (and other components from the microG project) is an either/or situation: Either you use Google or you use microG. I guess someone could write a Google backend for UnifiedNLP but it may not have a huge positive benefit as there are already equivalent backends that use either Apple or Mozilla's servers.
In my case, I use the microG project's components including BlankStore. It means that I don't have easy access to paid apps. But I've had good luck finding general purpose apps I need in F-Droid and the apps for various airlines, banks, etc. are usually free in the PlayStore so I have everything I need with very little downside. There are some apps that really, really want access to the Google Play Services and they don't run well but it seems that mar-v-in is making good progress there (thanks!).
Once microG's Play Services replacement is ready then I suspect that you need not install gapps at all to get at the official Google Play Apps you desire as things like the YouTube app should be available for free from the store/market and be able to run.
@n76 you got me slightly wrong. I have a device with stock ROM where GApps were pre-installed. There I want to get rid of Google location altogether and use µg instead. So it's not a question of installing both – but about not being able to uninstall the pre-installed one (this will resolve later when I install a custom ROM of course).
I have a second device running CM, with no GApps on, which still has the "old NOGAPPS" variant of NetworkLocation running, and of course the NOGAPPS mapsAPI and BlankStore (plus F-Droid of course – so basically the same you have). The only thing I miss there is GCM, which I need for several notifications; that's currently the only show-stopper to switch over with the first device.
Well I still think the confidence in GSM is so high, it should cancel obvious invalid Wifi locations. If I am in a GSM network in country X I am either in this country or near the border in a neighbouring country, no matter what wifi says. Also if the location/size of a GSM area code is known, I am there with some tolerance. No matter what wifi says.
Maybe also some time based filter can work. If I have a GPS signal, go into a building and lose GPS my wifi location just can't be hundreds or thousands of kilometres away in less than a minute.
All just some thoughts. Probably the time needed to implement it is too much for the benefit.
Agree with @J0s3f here: WiFi should have a lower prio than GSM. WiFi routers are sometimes relocated (when their owners move, or think of accidentally collected Mobile Hotspots). Never heard that for GSM cells. Not sure whether IDs of cells are never reused – but I'd expect re-usage following rules similar to those of phone numbers: if re-used, then not without giving them a reasonable "down-time" in between. Which hopefully has the ID purged from the CellID databases before it goes live again.
Maybe a (temporary) way around this could be integration for Tasker's API, so the end-user could at least define rules like "if I have a good GSM signal, turn off the WiFi location provider"? This would also be a work-around for a second issue: "if Internet is available, turn off NetworkLocation" (useful for devices with GApps pre-installed and for some reason not having XposedUnifiedNlp available, to make sure GoogleLocation doesn't permanently "phone home with my current location"). And probably several other use-cases :stuck_out_tongue_winking_eye:
@IzzySoft : I your stock device is rooted, have you tried removing all the GApps apk files, then (optionnaly) installing only the ones you really need (if any) using http://forum.xda-developers.com/paranoid-android/general/5-1-x-5-0-x-4-4-x-pa-gapps-mirror-t3069879 ? And afterwards, install XposedUnifiedNlp.
Didn't rfteffer (check his/her 2014-03-23 post above) make a list of accuracy/confidence of GPS/GSM/wifi methods of location ?
Why not sort location methods by (descending) confidence THEN accuracy ?
@breversa If I wanted to uninstall them, I of course had tried that (or I'd rather installed a custom ROM without installing GApps altogether). But until GCM is part of µg, I need one device with GApps – as for some things I need those notifications (e.g. StackExchange app uses GCM). So removing GApps is not an option on that device. Which unfortunately means: The one app responsible for GCM uses the same package name as … exactly.
Er… I don't wanna look silly or anything, but what do GCM means, in that context ? :-/
Anyway, here's what I did with my Samsung Galaxy Nexus :
- rooted it
- installed CyanogenMod (for the privacy options, but you might give XPrivacy a shot if you want/must stay as AOSP as possible). I included system apps in the "ask for permission" list.
- installed GApps from the above link (I actually only need the Play Store, so I took the "nano" package and removed the contacts/calendar syncing apps)
- started any GPS-using app. When Google Services (or whatever its name is) asks to access my location, I refuse !
- installed XposedUnifiedNlp and whatever backends I want
- started any GPS-using app. When XposedUnifiedNlp backends ask to access my location, I accept.
In the end, I have the Play Store enabled, XposedUnifiedNlp enabled, and (hopefully) no Google Location Service spying on my location. :-)
Is that what you're trying to achieve ?
GCM stands for Google Cloud Messaging. On Android, it's part of GSF (oops, sorry: Google Services Framework) – which has the very same package name the main NLP app for non-GApps devices is using. This is no coincidence: µg aims to "replace" the GSF transparently (i.e. in a way other apps accept it as GSF, without the need of any modifications), hence it must use the same package name. Many apps (according to my stats on ~11,500 apps: exactly 10%, told by the permission "Receive data from Internet") are using GCM for communication with their servers. Idea behind it is reducing network traffic in-device: apps only register with GSF on the device, which in turn registers with the GSF server, which then registers as listener for all affected sites. So instead of "n" apps permanently listening or polling, it's just one → less battery drain, less network traffic. Amazon has a similar service (with its own permission).
So: No, what you describe is not exactly what I aim. I'd like the first two steps (well, number one is already achieved), but then no GAPPS (instead, NOGAPPS – i.e. UnifiedNLP, BlankStore, MapsAPI, a combination working pretty well on one of my other devices), plus GCM. But currently, the last step (GCM) is not possible without GApps. I am aware of the work-around with that Xposed module, which is probably what I will revert to (after having the bootloader unlocked and a custom recovery installed – pre-conditions for Xposed if one wants to be on the safe side). I know @mar-v-in wishes to "have the gaps closed", so one day I won't need GApps for GCM anymore hopefully – but as magic as he is, things need time even for magicians. He's got my full understanding on that – I have the very same problems with time (check out my projects here at Github, and you can tell).
Regarding relative priority GSM and WiFi based locations: Some phones, like my Galaxy Nexus and from what I read many other Samsung models, does not report adjacent cells nor "all cell" information. So GSM location is based on only the cell tower you are connected to. With one data point all you will get for your estimated location is OpenCellID's and/or Mozilla Location's estimate of the tower location and coverage radius. In the rural US that can be a radius of 100Km or more, not a very good position estimate so dropping WiFi estimates when you have a mobile tower based estimate could be a very bad thing.
On the WiFi backend that I wrote and on some others that I've read the code for, some of the issues with a moved or moving (I am detecting WiFi on many buses) are attempted to be dealt with. For example, there is logic to require a group of APs that are plausibly close to one another before a location is reported.
I can see that the location reported by WiFi should be within the error radius of the location based on GSM. And that could be a good addition to either a single unified GSM/WiFi backend or built into the UnifiedNLP framework to check that. But a reasonable WiFi backend should account for things like duplicated MAC addresses, moved APs, moving APs, etc. regardless of how the UnifiedNLP framework handles the various backends.
Well put @n76 – full ack on that (I completely forgot about the "neighboring cell" issue). It might still be considered making it an option (with a link to this post of yours for some background) for the user to decide. So if one e.g. knows for sure the device reports neighboring cells and thinks (s)he needs it, it could be configured to "ignore WiFi if it's off more than 1 km" (just an example) – but with your rules in place, that IMHO would be very low prio (if needed at all, as your last paragraph already covers this) :stuck_out_tongue_winking_eye: – for me anyway, as I have Tasker turn WiFi off when leaving AP range, but the mileage of others might differ of course.
(wish there was a way to upvote/highlight/sticky your answer, as it sums up the background nicely!)
Just a thought on the accuracy/confidence issue : why not create accuracy/confidence levels (as @rtreffer suggested) for each backend, weight them (not necessarily linearly), multiply accuracy*confidence, and use the highest value ?
Example weight values :
- low level = 1
- average level = 3
- high level = 6
- perfect level = 10
GPS is perfect accuracy/perfect confidence, so 10 * 10 = 100 GSM current tower is average acc/average con, so 3 * 3 = 9 GSM (country guess) is low acc/perfect con, so 1 * 10 = 10 Wifi current AP is high acc/low con, so 6 * 1 = 6 IP (country guess) is low acc/high con, so 1 * 6 = 6
Of course, those values are very very rough and just come out of my head at the moment, so they'd need lots of refining (GSM country guess being higher rated than GSM current tower is… suboptimal !).
Come to think of it, there could be different numbers of levels for accuracy and confidence, and different weights too… No need for 1/3/6/10 for both !
Just my two €cents…
Hello here,
I still had the old LocalGSMLocationProvider installed, in parallel to Local-GSM-Backend, and a week or two ago, the old one became prioritised overt the more recent Local-GSM-Backend. My location was consistently off by several hundred meters. It took me a while to think of turning off the old backend, but it did the trick instantly, and I could also recreate the problem each time by turning it on again. You might want to look into this. Other than that: thanks for everything! :)
Hello, I frequently have it that déjà vu has a rather precise location <10m accuracty, but unifiednlp still prefers the gsm location services provider, which has often an inaccuracy of >500m or more. This sadly forced me to turn of the gsm location service backend, despite it being very good at giving a coarse location.
