[Feature Request] Filter by "changed user data or updated app" and "last backup date"

[dagaz]

It would be great, if it would be possible to filter the app list by two new criteria:

1. by changed user data or updated app
2. by date of last backup

Is your feature request related to a problem? Please describe. "changed user data or updated app": I would like to reduce backup time and write cycles. So it would be helpful if it would be possible to filter all apps where is a change of the app itself (update) OR a change of the user data (more important than the app itself ;)). Right now it can just be filtered by updated apps. But sometimes there is a big change for user data as well so the easiest way now is to backup all apps, regardless of changes.

"by date of last backup": Sometimes I get errors for the backup process. I would like to filter the apps for which no backup was created so that I can manually redo a backup.

Describe the primer solution I would like if there would be added the two filter options for the filter menu. If the user data changed could be taken from the size of the apps user data. Backup dates could be generated as a list from the file dates.

Describe alternative solutions I am not sure if there is an alternative. Regarding errors it might be possible to filter by "apps with backup errors" - but i think that could be the same effort.

System Information(please complete the following information):

• Device: Oneplus 7 Pro
• Android Version: 10
• ROM: LineageOS
• App's Version: 4.0.0 (F-Droid)

[machiav3lli]

1. This would posdibly never make it, as we had long discussions on other issues and in the chat group. The benifits are only marginal considering how much the data of the apps not stable compared to 3-4 years ago. So this doesn't come in question for now.

2. this option is already present in the main filters with the tag old backups. Including it in schedules modes shouldn't be a problem in future releases.

[dagaz]

Thank you for you quick answer.

1. This would posdibly never make it, as we had long discussions on other issues and in the chat group. The benifits are only marginal considering how much the data of the apps not stable compared to 3-4 years ago. So this doesn't come in question for now.

What a pity! But I understand that you have much more important priorities.

2. this option is already present in the main filters with the tag old backups. Including it in schedules modes shouldn't be a problem in future releases.

That would be nice.

[leso-kn]

Hello! Revisiting this issue in 2021 I'd like to add a comment on the first criteria described: 'only backup when app data was modified'

A scenario, where this could be beneficial is the case of backing up directly over a network (for example by mounting the target location via rclone).

@machiav3lli I didn't quite understand your comment 'considering how much the data of the apps not stable compared to 3-4 years'. Could you ellaborate this point?

[leso-kn]

Thinking forward on this issue, there are two possible implementations that come to my mind, maybe they're worth considering:

1. Determine modifications using fs (ext4) file modification time
2. Determine modifications using the Android Usage Statistics API

---

Ellaborating the two approaches:

1. File modification time:

• Check modification date of all files belonging to an app (recursively)
• Determine the latest (newest) modification time
• If no backup has been taken yet, perform a backup and remember the determined modification time (e.g. save to the OAndBackupX internal database)

Next time the backup is considered:

• Recalculate the modification time
• Compare if the new modification time is newer than the saved (remembered) one

2. Android Usage Statistics API:

The API provides methods to determine when an app was last used. This would be a great criteria to determine whether an app needs to be backed up or not, since (at least for launchable apps) it is very unlikely, that data changed if the app was not in use since the last backup.

Following the format of approach 1, the rough steps would be:

• When a backup is considered for an app:
• Use the UsageStats API to determine when the app was last used
• If no backup has been taken yet, perform a backup and remember the determined last-used time (e.g. save to the OAndBackupX internal database)

Next time the backup is considered:

• Use UsageStats API again, to determine the last-used time
• Compare if the new last-used time is later than the saved (remembered) time

[machiav3lli]

@leso-kn thanks for your documented research.

First more to what I meant by data of apps being not stable: background services to do some checks or syncing are being used more than before which is resulting some data files to change (mostly cache which is generally not needed). This dynamic behaviour makes the first solution obsolete.

Now to the suggested solutions:

1. Ignoring the forementioned point, such scanning using su cli will be a huge resources eater, which will reflect to a bad UX. The app should do the most possible with the least resources. An API is a far better - and maybe the only acceptable - solution.

2. This one seems like a great suggestion. Bit I would jump off comparing the two opened dates (before backup and last opened) and would just compared last opened with the last backup date. This should have the same usage.

The 2. with the change I mentioned can be a first iteration in an upcoming major release (unfortunately not 7.0.0, as it's already in rc-phase and should come next week).

[leso-kn]

Good idea with comparing it to the last backup date, right, that simplyfies the solution even more.

Looking forward to v7 next week! By the next major release you mean 8.0.0? It will probably take some time until that's being released, or does scheduling it for the major release mean it can be introduced in a 7.x.x version?

[machiav3lli]

Maybe I wasn't thinking that clear at the time. It could be possible to include such (after the change I suggested) in a minor release (7.1.0 should benext) as no change to the backup object is needed.

[leso-kn]

Sounds good. Is there a board/roadmap where this should be tracked as a potential feature for the next release?

[hg42]

some thoughts (despite I had no look at the UsageStats API):

what exactly is "last used"? I guess opening an app does not necessarily mean it will change it's data? does it include changed data that is collected by services in the background?

[hg42]

Once I reported that data changes very often for all my over 600 apps (say 400-500 had changed data every day). I think this was one big argument not to scan for changed data.

However, this observation has dramatically changed since I am using SuperFreeze and TrackerControl. Every app that I don't actively use is sleeping until I open it. Currently I find that about 120 of 650 apps (incl. ~260 system apps) are changing their data every day (only looking at /data/data, excluding cache directories doesn't matter much) and this doesn't change much if I look for 7 or 14 days. This seems to be the number of apps that I excluded from freeze and partly from tracking control.

I use commands like:

find /data/data -mtime -1d | awk -v FS=/ '{print $4}' | sort -u
find /data/data -mtime -1d | awk -v FS=/ '{print $4}' | sort -u | wc

to list or count packages with changed internal data (use -7d for 7 days).

[leso-kn]

@hg42 Sorry for the delay! Been quite busy lately.

Having a look at Android's UsageStats API (here and here), it seems to be possible to measure the last time a foreground service was used.

Since 'used' refers to whether the service is running in the background (regardless of changes to its data), this method might however not be very accurate for our case, so measuring changes in service-only apps might in fact be connected to more effort.

I believe having a changed-apps-only option, that only considers Activity usage for the moment, would already be a big advantage and could later be improved in order to support service-only apps as well.

What do you think?

[leso-kn]

@machiav3lli I see that a lot of feature requests have been submitted lately (> thus I was suggesting a step-by-step implementation for this one, to minimize the initial effort).

Is there a document (release schedule or the like), where potential features for upcoming versions are tracked? Maybe picked by both priority and estimated effort.

[Victor239]

Came across this issue after finding out that one of my apps despite being used daily for important data hadn't been backed up for months. I was only using a "New and updated apps" filtered backup and didn't realise that since the app itself hadn't gotten any F-Droid updates it would never backup until then.

Any sort of "changed data" measurement would be good for me since otherwise I have to select all apps through an "old backups" filtered periodic backup anyway, plus I also use SuperFreezZ to stop background usage. Selecting only those which have used a foreground service however sounds like a more efficient direction to go in.

[cipitaua]

I also warmly second this feature request, it's the same as TitaniumBackup's "redo backups for modified data", which is very useful when the backup folder is synced to the cloud (I use FolderSync for that and it's awesome).