Please add special backups

[aelmahmoudy]

I've seen this feature in TitaniumBackup, it has special backups for:

Wi-Fi access points Bluetooth pairings Accounts Application Widgets stock browser bookmarks call log Country/Language/Timezone Data usage (Policy & Stats) SMS/MMS messages Wallpaper

Some of those items are actually backed up in XML format. The special backup is useful that restoring them to a different ROM does not cause crashes on that new ROM (merely backup up the binary database then restoring to a different ROM causes FCs). Thanks...

[jensstein]

this could be a very nice thing. but since i don't have any dedicated test devices it will probably be a bit hard to achieve (unless i have somebody to do a lot of testing) as i can't easily shift around the roms on devices i use normaly and one of my devices is so old that there's only a single rom under active development. there are the com.android.providers.* apps which contain things like settings and sms messages but is it those you mean when you say that restoring gives crashes (since they are mainly databases)? also, i could maybe be helpful if you could send an example of something backed up in xml and its original database file to my email (if you have something that doesn't contain too personal information).

[ildar]

1st, may I suggest a no-cost test device - an x86 Android in a VM or on a laptop? It is quite functional and may fulfill your needs. URL: http://www.android-x86.org/documents/virtualboxhowto

[jensstein]

yeah, i already have virtualization set up, but unfortunately what is needed here is especially devices running different vendor roms (which have differences like when samsung have 'dbdata' for 'data'). because a virtualized image of android isn't much different from what i already run on my real devices.

[smarquespt]

Hi jens,

I can make some of those tests in android 2.3.7. And I mean some because I don´t have use some of them.

Some of those features are implemented in slight backup - https://github.com/handschuh/Slight-backup and https://f-droid.org/repository/browse/?fdfilter=backup&fdid=de.shandschuh.slightbackup.

[jensstein]

thanks, i'll send you a mail when i get to look at it then. and thanks for the reference to the other app, i had completely forgotten about it but that could be helpful in implementing this.

[aelmahmoudy]

Sorry for late reply. I found that TitaniumBackup only backs up the following items in XML format: Bookmarks WiFi access points Call log Messages

Yet when I inspected the call log & messages XML backups, I found them empty ! Seems it couldn't do them on my system (I am using CyanogenMod 10.1.3)

[aelmahmoudy]

I also inspected the rest of the settings backups, and I found that they are all config files under /data, they are as follows:

• WiFi access points: /data/misc/wifi/wpa_supplicant.conf
• Bluetooth: data/misc/bluetoothd/<BTMAC>/
• Data usage (Policy & Stats):
  /data/system/netpolicy.xml
  /data/system/netstats/
• Accounts: /data/system/users/<USER>/accounts.db
• Wallpaper:
  /data/system/users/<USER>/wallpaper
  /data/system/users/<USER>/wallpaper_info.xml
• Accounts: /data/system/users/<USER>/accounts.db

<BTMAC> is the Bluetooth MAC address of the Android device <USER> is the ID of the user (typically it is 0)

[aelmahmoudy]

A software that properly backs up SMS & call logs is Super Backup: https://play.google.com/store/apps/details?id=com.idea.backup.smscontacts

[jensstein]

thanks for looking into this. the easiest would probably be to keep backups of these files in a separate place so they can be restored easily and without all the other files in the same folder. i need to find out if these directories are the same on most roms, but it seems that most of us use cyanogenmod or something equally close to stock. and i wonder i the /data/misc/ directory is tied to a package somewhere. it doesn't seem like it, so maybe that should have its own entry.

you also mentioned sms messages at first but do you think there is a real need for this when you can backup the com.android.providers.telephony package (phone/messaging storage)? i would imagine that this could be restored on different roms.

and please write if you come across any other items that could be backed up and restored like this. i think we should try to back up widgets too for example [edit: oh, you already mentioned widgets].

[aelmahmoudy]

I think those files are the same between ROMs, because I restored those (non-XML) backups from stock to CyanogenMod, and it worked (I'm not sure about Accounts & Wallpapers though). Also I just tried today restoring backups from CyanogenMod into OMNI, and they also worked (what I tried was just WiFi & Data usage).

According to what I experienced: restoring com.android.providers.* backups from one ROM to another causes the related apps to force close. For example I just tried restoring com.android.providers.contacts backup from CyanogenMod into OMNI, that caused the dialer/contacts apps to crash (force close).

[aelmahmoudy]

Please ignore my comment regarding Super Backup, I didn't know about Slight Backup.

[Cathryne]

[...] if you could tell me which rom you are using and see if the paths to wifi access points and account settings match these: /data/misc/wifi/wpa_supplicant.conf /data/system/users//accounts.db

• FairPhone OS 1.1 "Caju" (Android 4.2.2)
• both files present exactly there :-)

actually, if you are using something other than cyanogenmod, it would help me if you could email me some of those files as they are on your device so i can compare them to my own (i simply don't have enough devices myself to have a good basis for this alone).

Yeah, OK, but I need (a pointer to) advice about how to copy/extract those files. I used DiskUsage to check the above folder paths, but it can't copy. And File Manager, File Explorer and Open Explorer Beta can't seem to access the ~~/system/~~ system folders :-/

[aelmahmoudy]

@Cathryne It is /data not /system Those paths can be accessed if you have root access (which I think is needed anyways for oandbackup to work).

[Cathryne]

Corrected above ;-) With which app? The filemanagers I listed above do not seem to have any option to navigate outside or above /sdcard0. Root access is enabled.

[jensstein]

with cyanogenmod's file manager you need to enable root access in the app settings to handle files in /data/. but the easiest is maybe to do it via a terminal emulator on your device:

su -c 'cp /data/misc/wifi/wpa_supplicant.conf /sdcard/'
su -c 'cp /data/system/users/$user/accounts.db /sdcard/'

($user should probably be 0 unless you have multiple users on your device) or via a connected computer and adb:

adb pull /data/misc/wifi/wpa_supplicant.conf $HOME
adb pull /data/system/users/$user/accounts.db $HOME

($HOME is assuming you are using linux but if you aren't then just substitute with another path that you can access on your computer) just say so if need still need help or if this doesn't make sense.

also, totally nice to have a fairphone user around.

[Cathryne]

OK, thanks, got them :-) https://my.owndrive.com/public.php?service=files&t=326255d4966b51e5fa4511aaa6b4d612 (valid till Feb. 18th). Hope this helps!

In both files, I changed all personal data I could find with Notepad++ & SQLite Database Browser to "TEST".

[jensstein]

thanks a lot. right now it seems pretty manageable. i think i will start looking at implementing it after the next release.

[jensstein]

now i have finally had time to do this so the first support for special backups is now in master with commit 5af50d7d489960f75e120123c4f864225f496bce. not all of the suggested files are added yet though. it still needs some work but it should be useable. some of the paths for the individual files will differ between versions of android so it would be a great help if anybody who is interested in this reports if they have paths which differ from the ones we use now. the paths can be seen in the getSpecialBackups method in OAndBackup.java (here). and if anyone wants to test this out before the next release you can send me an email for an apk if you haven't got the oppurtunity of building it yourself.

and note that most of them probably requires a reboot after being restored.

[timur-g]

I'm glad to see Wi-Fi here. I tried Bluetooth backup and it went fine on my CM 11 snapshot 8, Android 4.4.4 (I didn't try restore). But, when I tried Bluetooth Share (what's the difference?) I got: cp: can't stat '/data/data/com.android.bluetooth/lib': No such file or directory

[jensstein]

actually, "bluetooth share" is not part of the special backups since it's a package provided by android in the normal way. i would guess it's the program that handles sharing things over bluetooth. the package just called "bluetooth" on the other hand is part of the special backups and backs up the /data/misc/bluedroid directory. i should probably specify which apps are which in the readme or something. so since this is an unrelated bug, could you maybe open a new issue for this and tell me whether bluetooth share is the only app you have this error with and whether or not you have the "follow symbolic links" option enabled. could you also please run this command on your device or through adb: ls -l /data/data/com.android.bluetooth/. ultimately this isn't anything to worry about since the lib folder is not backed up anyway (we'll of course fix it nonetheless so there isn't an annoying error all the time).

[smarquespt]

@jensstein

Which are special backups? In my device I have: accounts, widgets, wallpaper and wifi. Is there any more not listed here. I remember having one or two more in test release you sent me.

[smarquespt]

Looking at the commit I realized that i´m missing Bluetooth and Data Usage and Policy

[jensstein]

yes, i left them out on the older platforms because i don't know where the relevant files are. it's not documented and the emulator wasn't really helpful here. so if you can find the paths on your own device i will include them. on android 4+ (at least on my devices) it's /data/system/netstats/, /data/system/netpolicy.xml and /data/misc/bluedroid/.

[smarquespt]

I really can´t tell. But if found this:

/data/system/usagestats/ - would this be it?

/data/misc/bluetooth and /data/misc/bluetoothd - in first i got the file dynamic_auto_pairing.conf and second /folder_with_some_numbers_as_name/config

for netpolicy i can´t find it. I´ll try with terminal emulator to check if file is found somewhere.

[jensstein]

@smarquespt /data/system/usagestats/ doesn't seem to contain network usage. i have it on my devices too and the files there have packages listed which doesn't have access to the internet. i will add the bluetooth directories.

[ghost]

oandbackup seems to backup contacts, all seems to be stored in contacts2.db.

[timur-g]

Related to #84.