bit-team
Best practices to exclude cache folders to speed up backup
Hey folks!
Going through the log, I've noticed that a lot of small files within cache folders significantly slow down the backup process and in addition to that it actually makes no sense to back them up anyway.
So I wonder what's the best way to exclude such folders elegantly by using excluding patterns?
Adding them manually would be too cumbersome, since only some apps have only one dedicated cache folder, some have several, but in Firefox each site has its own cache folder. See several examples:
~/.config/BraveSoftware/Brave-Browser/ShaderCache
~/.config/BraveSoftware/Brave-Browser/GrShaderCache
~/.config/BraveSoftware/Brave-Browser/GraphiteDawnCache
~/.config/BraveSoftware/Brave-Browser/component_crx_cache
~/.config/BraveSoftware/Brave-Browser/extensions_crx_cache
~/.config/BraveSoftware/Brave-Browser/Guest Profile/GPUCache
~/.config/BraveSoftware/Brave-Browser/Default/GPUCache
~/.config/BraveSoftware/Brave-Browser/Default/DawnCache
~/.config/BraveSoftware/Brave-Browser/Default/optimization_guide_hint_cache_store
~/.config/BraveSoftware/Brave-Browser/Default/Shared Dictionary/cache
~/.config/BraveSoftware/Brave-Browser/Default/Service Worker/CacheStorage
~/.config/BraveSoftware/Brave-Browser/Default/Service Worker/ScriptCache
~/.mozilla/firefox/oyoajxci.default/storage/default/https+++www.reddit.com/cache
~/.var/app/com.discordapp.Discord/cache
~/.var/app/com.discordapp.Discord/config/discord/Cache
~/.var/app/com.discordapp.Discord/config/discord/Code Cache/
~/.var/app/com.discordapp.Discord/config/discord/DawnCache/
~/.var/app/com.discordapp.Discord/config/discord/GPUCache/
~/.var/app/org.signal.Signal/cache
~/.var/app/org.signal.Signal/config/Signal/DawnCache/index
~/.var/app/org.signal.Signal/config/Signal/GPUCache/index
~/.var/app/org.telegram.desktop/cache
~/.var/app/org.telegram.desktop/data/TelegramDesktop/tdata/user_data/media_cache
The solution could be added to the FAQ section, since every BiT user would profit from it.
Hi, thank you for bringing this up! I noticed the same when using BackInTime myself.
I'm afraid there is no easy way that works for all users. Excluding all folders named cache or Cache is a start, but you already discovered: It's different for most applications.
So maybr the FAQ could just include a hint to check the logs and find folders to exclude, like you did.
Have you seen this "Add" button and the "Exclude patttern" dialog?
Do you have an idea or suggestion how we could improve the situation for our users?
@buhtz
Sure, my question was more about proper exclude patterns for cache folders in general.
At the moment I decided to play it save and use the following
.var/app/**/[Cc]ache/*
.var/app/**/media_cache/*
.mozilla/firefox/**/cache/*
.config/BraveSoftware/Brave-Browser/Default/Service Worker/CacheStorage/*
for the above mentioned folders.
So maybe the FAQ could just include a hint to check the logs and find folders to exclude, like you did.
Definitely!
And we could also add two buttons to link to the FAQ for
- "Best practices": How to check the logs to find "exclude candidates"
Exclude pattern syntax: Here with "pattern" I mean the regular language syntax- Optional "Application exclude patterns" with hints to candidate folders to exclude for common apps (possibly maintained by our community in the future)
So maybe the FAQ could just include a hint to check the logs and find folders to exclude, like you did.
Definitely!
And we could also add two buttons to link to the FAQ for
* "Best practices": How to check the logs to find "exclude candidates" * `Exclude pattern syntax`: Here with "pattern" I mean the regular language syntax * Optional "Application exclude patterns" with hints to candidate folders to exclude for common apps (possibly maintained by our community in the future)
Those are excellent points!
One thing that we might have to decide is whether to recommend using * after / or not, f.i.
.var/app/**/[Cc]ache/*
will back up the folder itself, but not its content, while
.var/app/**/[Cc]ache/
won't even back up the folder.
So I wonder which approach should we recommend in the FAQ?
For rsync pattern matching rules see here or here.