Issues when restoring special backups from one device into another

[aelmahmoudy]

Hello,

After restoring special backups from an old phone to a new one, the WiFi of the new phone did not work ! The reason I found out to be is that wpa_supplicant.conf does not only contain APs info, but also some info about the local device, as follows:

ctrl_interface=/data/misc/wifi/sockets
update_config=1
device_name=*******
manufacturer=******
model_name=******
model_number=******
serial_number=*****
device_type=*****
config_methods=physical_display virtual_push_button
p2p_listen_reg_class=**
p2p_listen_channel=*
p2p_oper_reg_class=***
p2p_oper_channel=***

then the APs come later in network={ ... } entries

I think to fix it, oandbackup can just backup/restore starting from the first 'network=' line till the end of file. ie. backup with a command like:

(echo "network={" ; sed -e '1,/network=/ d') /data/misc/wifi/wpa_supplicant.conf > wifi.access.points/wpa_supplicant.conf

then restore as follows:

sed -e '/network=/,$ d -i ' /data/misc/wifi/wpa_supplicant.conf
cat wifi.access.points/wpa_supplicant.conf >> /data/misc/wifi/wpa_supplicant.conf

[aelmahmoudy]

Similarly in the bluetooth/bluedroid.zip, I see the following device specific entry in bt_config.xml:

    <N1 Tag="Local">
        <N1 Tag="Adapter">
            <N1 Tag="BluezMigrationDone" Type="int">*</N1>
            <N2 Tag="Address" Type="string">*****</N2>
            <N3 Tag="LE_LOCAL_KEY_IR" Type="binary">*****</N3>
            <N4 Tag="LE_LOCAL_KEY_IRK" Type="binary">****</N4>
            <N5 Tag="LE_LOCAL_KEY_DHK" Type="binary">*****</N5>
            <N6 Tag="LE_LOCAL_KEY_ER" Type="binary">******</N6>
            <N7 Tag="ScanMode" Type="int">*</N7>
            <N8 Tag="DiscoveryTimeout" Type="int">***</N8>
       </N1>

[ildar]

Ripping device-specific info is required in restore, not backup.

[jensstein]

well, i would say that restoring backup data marked as system across different devices shouldn't really be expected to work without issues. but we could of course try to prevent some issues with these special backups. i agree with ildar that the files should be backed up as they are. you can always take out information afterwards but you can't put it in again if it's not there to begin with.

your sed command seems a little garbled. i assume you mean sed -i '/network=/,$d' /data/misc/wifi/wpa_supplicant.conf. we could use that but what would happen if you actually wanted the whole file restored? this approach is only valid when you restore to a different device so it shouldn't be the default i think. so one option is to check whether the restoring is actually done on a different device and do the substitution then. another option would be to notify the user of a potential problem and then provide the commands to fix it. a dialog showing the sed commands and offering to run them for example. i think i like the less intrusive option better.

[aelmahmoudy]

yup, I agree.

[ildar]

Guys, I can't tell about BT stuff, but as for wpa_supplicant.conf:

1. Device-specific thing are really not important and automatically added by the wpa_supplicant binary.
2. Networks info is important but isn't full. Some APs can have manual settings, e.g. static IPs which stored not in wpa_supplicant.conf

[aelmahmoudy]

@ildar yes, it is auto-added indeed (I tried by deleting /data/misc/wifi/wpa_supplicant.conf, but I didn't try if the file is there, but without the device-specific info existing in it.). But if there is false info in that file, it won't be overwritten, and the WiFi won't work on the device.

[ildar]

surely! That's a good example of how configs are handled in Unix.

[jensstein]

if the device-specific info is added by wpa_supplicant even if there is a wpa_supplicant.conf containing the network parts then i would agree that we could maybe exclude them when restoring in all instances. but another problem is also that the config file doesn't only contain the network info and device info. what if somebody has the country parameter set or load_dynamic_eap? in aelmahmoudy's example there is update_config=1. this is not a device-specific parameter and so it shouldn't just be deleted casually. in fact it controls whether the config file is overwritten automatically or not. i think the best approach would be to find the least amount of lines that need to be stripped from the file.

[ildar]

AFAIK Android always has update_config=1. I'd suggest the maximum amount of lines that need to be stripped from the file.

[jensstein]

well, android will probably have it set like that as a default most of the time. maybe even always. but what if somebody has changed it on their device? i have other configuration files on my devices which i have modified manually to something other than the standard. or what if some specialised distribution / rom has another default? part of the contract of doing backups and restores are to recreate the state of parts that are backed up as faithful as possible. so i think it requires very convincing arguments to strip things like this during a restore. it would be great if one of you (or somebody else) can find out exactly which individual lines need to be replaced to make a restored wpa_supplicant.conf (and other files in the special backups) work. what about the p2p_ lines for example? my devices don't have them.

[aelmahmoudy]

@jensstein I suspect that p2p_ lines are for WiFi direct. Do your devices support WiFi direct ?

[aelmahmoudy]

I found another issue, that restore of 'Data Usage' does not work, ie. after I restore Data Usage (and reboot), I still cannot see the old data usage history.

[jensstein]

yes, p2p are wifi direct / peer to peer: http://wireless.kernel.org/en/developers/p2p/howto but the question is how they should be handled. i would imagine that they actually have a purpose here and that they are not automatically put in by wpa_supplicant. and if they are put into a device which doesn't support p2p, does that then cause problems?

as for data usage, this is what is backed up:

• /data/system/netpolicy.xml
• /data/system/netstats/

if you can find out if there are other files that need backing up then we'll just add them. and if you do find any then it would be great if you could confirm that backing up should work by copying them manually and rebooting. the problem could arise if the system overwrites the files with old values when rebooting.

/data/system/netstats/ are handled by frameworks/base/services/java/com/android/server/net/NetworkStatsService.java in the android source and /data/system/netpolicy.xml are handled by frameworks/base/services/java/com/android/server/net/NetworkPolicyManagerService.java. i'll try looking a little around those files.

[aelmahmoudy]

• Both netpolicy.xml & netstatus have been copied back in their proper places by oandbackup
• The settings in net policy seems to have restored successfully, the problem seems to be net usage history not appearing in the system although the files are there (with proper permissions!)

[jensstein]

hmm, if the restored files in /system are equal to the files in the backup after a reboot (if there have been no data connection in the meantime of course) and the data usage is still not appearing correctly then i would guess there are more files that need to be copied.

[timur-g]

We're still missing this WiFi restore on different device.

[herbsmn]

backing up and restoring wifi.access.points from one device to another using a microSD worked flawlessly for me in the lastest stable version of oandbackup on F-Droid. had to restart the device to see the restored wifi access points, though. was also upgrading my Replicant operating system, which is currently based on Android Marshmellow, to the latest Replicant release candidate at the time. Perhaps this ticket can be closed.