radiocells-scanner-android
radiocells-scanner-android copied to clipboard
openbmap
Wifi scan performance massively decreased after migrating from Android 7.1 to 8.1
What steps will reproduce the problem?
- Go and log some wifis with an device running Android 7.1.
- Upgrade the device to Android 8.1.
- Log again, following the same route as before.
- Compare the number of wifis scanned.
What is the expected output? What do you see instead?
Expected: roughly the same number of wifis in both sessions. Actual: much fewer wifis on 8.1 than on 7.1 (the number on 7.1 is around 5–10 times as high).
What version are you using? On what operating system?
0.8.18 (ef93348-dirty) from F-Droid, on LineageOS 14.1 and 15.1
Please provide any additional information below.
Scan mode was originally set to Background. In an attempt to troubleshoot the issues on 8.1, I changed it to Full Power mode, to no avail. Will try High Performance next.
I do not use “Turn on WiFi automatically”, i.e. WiFi should be on permanently.
Tried again in Full Power Mode. That didn’t seem to lead to any improvement either. However, the app was constantly in the background (as in all previous attempts). At some point I brought the app to the foreground, and all of a sudden the number of wifis started climbing.
I still need to figure out how scan mode affects this, but apparently it makes a huge difference on 8.1 whether the app is in the foreground or not. Bad news if you’re scanning in the background and have something else going on in the foreground (sat-nav, OSM tracker or other stuff)…
Just to be sure: have you disabled battery optimization (see e.g. https://www.verizonwireless.com/support/knowledge-base-202636/)?
Possibly also related: https://developer.android.com/about/versions/oreo/background-location-limits (see the last section on WifiManager)
Both could be causing this. Battery optimization was indeed enabled (appears to happen by default when upgrading to 8.1); I’ve disabled that and will test again.
Tested again: With battery optimization disabled and in Background mode, there is hardly a noticeable difference in the number of wifis capruted while the app is in the background. Similar story in Full Power mode. In High Performance mode things seem better, but there is still a noticeable difference between background and foreground.
I am not quite sure whether the difference is caused by a decreased frequency of location updates, of wifi scans, or both.
As for location updates, a quick fix would be to request updates from the “passive” location provider as well. If something in the foreground (such as a sat-nav, which I often have in the foreground while I’m out logging wifis) is receiving more frequent GPS updates, Radiobeacon would receive those as well. (In that case you might want to filter by source, as the passive location provider would potentially give you locations from any source—and using wifi locations as a reference for wifi logging would lead to errors accumulating.)
On the long run, the logger service (i.e. the component which receives the location updates) would apparently have to be a foreground service, if that is not already the case.
btw, the guys at Mozilla have likely run into this before, maybe there’s something on the mailing list, or in the change log
since I just came across battery optimization behavior: the device was plugged into a charger the whole time (as I was in the car).
Cross-link to possible related issue with Wigle.net (although Android P)
https://github.com/wiglenet/wigle-wifi-wardriving/issues/251
Possibly also related: https://developer.android.com/guide/topics/connectivity/wifi-scan#wifi-scan-throttling
There are restrictions from Android 8 onwards both for wifi scans and location access, which seem to have bitten me here.
Android 9, it seems, will tighten its grip even more. See https://issuetracker.google.com/u/1/issues/79906367?pli=1 (which is linked from the Wigle issue). Comments 19/20 are particularly interesting: if an app holds NETWORK_SETTINGS permission, Android 9 will not count its wifi scans against any quota (which seems to mean they are unlimited as before).
#219 should address the issue on Android 8 (Oreo), still need to test it in practice.
I’ve been trying to find the NETWORK_SETTINGS permission: the ticket, as well as the docs at https://developer.android.com/reference/android/net/wifi/WifiManager#EXTRA_RESULTS_UPDATED, explicitly mention android.Manifest.permission.NETWORK_SETTINGS, but https://developer.android.com/reference/android/Manifest.permission makes no mention of it.
I just posted the part on NETWORK_SETTINGS as a question on StackOverflow: https://stackoverflow.com/questions/51901074/prevent-wifi-scan-throttling-on-android-9-pie-with-network-settings-permission
Got an answer to my question: NETWORK_SETTINGS is a signature permission. If I understand that correctly, that means the app needs to be signed with the same key as the OS itself, so this is out of reach for a third-party application.
As for Android 9, there’s an entirely new wifi RTT API, which uses round-trip time for distance estimates. This seems to be the new preferred API for network-based location. I didn’t find any definite information regarding usage limits/throttling behavior, though.
Downsides are:
- Existing systems will need to be redesigned to work with RTT rather than RSSI.
- It only works on Android 9 and higher.
- Both the device and the access point need to support this method.
The upside is that RTT is by far less vulnerable to obstacles than RSSI, allowing for much better distance measurements. This technology would allow us to collect much more precise data, as well as giving us better position estimates.
Still trying to figure out how to correctly use the RTT API, and what the limits are: The API ironically requires a ScanResult (as returned by the now-vilified wifi scan), and there seem to be usage limits, though no details on what they are. Details at:
https://stackoverflow.com/questions/51938026/correct-usage-of-wifi-rtt-ranging-in-android-9
This also affects Android 9.0 https://github.com/openbmap/radiocells-scanner-android/issues/225 I suggest the title is updated to be "8.1+"
See discussion above. Android 9 is definitely affected, but the mechanism is very different and, as per my research, there is no way around it (other than staying on Android 8 for the time being). Android 9 should probably be a different issue (though there doen’t seem to be any solution on the application side).