Backups accumulate indefinitely

[Spangenhelm]

Hi, it seems like the backup copies are limitless, how can we specify a number to keep? i can't imagine having thousand copies of my 2Mo wiki (first because it's useless and second because it's incompatible with a cloud backup)

thanks

[pmario]

The most successful backup plugin for TWc was LessBackupsPlugin

It uses the following mode:

• It always keeps the latest version ... sure
• one TW per second ... I personally disabled this one.
• 1 per minute, 1 per hour,
• 1 per day, 1 per week, 1 per month
• 1 per year.

Maximum number of backups eg. for 10 years

y(10) + m(12) + d(31) + h(24) + min(60) + sec(60) = 197 backups. ... That's about 400+ MByte of text based content.

This number increases very slowly, since only the "per year" files are never overwritten.

[pmario]

Hmm, I don't understand those arguments:

• first because it's useless
• and second because it's incompatible with a cloud backup

I'm ok with the first one.

But I don't understand, why 1000+ files should be incompatible with a cloud backup mechanism. They would completely fail, if that where true.

Harddisk space is cheap. eg: Have a look here 25GB for free and 100GB for 1€/month. ... So if you add up all the free space you get from different companies, I'm sure you are up to 500GB free space in no time. The big pulus here is. Different companies = better protection against failure.

[Spangenhelm]

Hi @pmario thank you for responding to me,

about this

it's incompatible with a cloud backup

i should of explain it better: what i meant is that for every new tw backed up it needs to send it to the cloud (in my case 2Mo each time) it is not really about the space, i have plenty of it, but more of the bandwith that it takes each time i modify my wiki, even a very small change will create a full backup of the file and therefore it will upload all of it, for example in less that ten minutes i had modified it 8 or 10 times and if i don't pay attention i will soon have a hundred of tws that need back up, this is why i said it is not compatible with a cloud sync solution (but this is true too with a local one to me) what i would like is something that saves every x modifications or every x hours (like the plugin you gave to me, is it available for tw5? apparently no) and only x number of copies, best of all would be to only keep the difference between the versions (is it what they call "deltas"?) for now my cloud provider (Symform) does not do such thing but even though it did since tw does a full copy then it would not be efficient. Last solution, make my own saves manually but there is no way so far to disable the backup feature in nw.

• Spangenhelm

[pmario]

.. it is not really about the space, i have plenty of it, but more of the bandwith that it takes each time i modify my wiki, ..

I see. imo a valid point.

LessBackupPlugin available?

no .. but it's a mechanism Jeremy is aware of. It has been discussed several times in hangouts.

deltas

hmmm. IMO not good. This will open an other can of worms. .. It's easy to fall out of sync, with deltas. ... eg: what if you loose a "middle one" of them. .. all later diffs will be invalid. ..

So from time to time you'll need to save a full copy, just to be safe. .... and and and ... so it immediately adds a lot of complexity to the core. ..

[Spangenhelm]

It's easy to fall out of sync

Right. Thank you

[Jermolene]

Hi @Spangenhelm I'd certainly like to add the ability to prune the backup directory, but it's something that I'm fairly cautious about. We're talking about code that nukes files; it's pretty important that it nukes the right ones. In my personal use, I'd rather manage the backup files myself than trust the code.

[Spangenhelm]

Then at least we should be able to disable this feature totally, don't get me wrong i love the idea and really think that backup is obvioulsy essential but if this stays the way it is now it will soon be really too messy for me.

About nuking the right files we simply delete the older ones first no ? Thank you

[pmario]

@Jermolene I think LessBackupPlugin mechanism has proven to work for many users. It ends up with a maximum number of files. The max grows very slowly and if you disable seconds and minutes, the max is about 130 files. ...

Here the code doesn't delete files, you just overwrite them. .. So if you use a backup mechanism that supports versioning, you will have the possibility to resurrect the history. ... So imo its very safe.

[Spangenhelm]

Hi @pmario +1 Btw i am looking for a versionning system for my tws (on linux) but except things like git and stuff i did not find any that is simple and efficient, do you have one in stock ? Thank you

[pmario]

------ OT -------------- My git repos go to the cloud and lokal backups. GitHub for open source, GitLab and BitBucket for private git repos.

For Ubuntu I use Back In Time SW. The problem I have there is, that my external HD is very loud, so I only start it manually but regularly :) .. Windows automatically backs up to the ubuntu laptop. History is done with back in time.

IMO Back In Time GUI and settings are very error prone. eg: It doesn't back up hidden folders by default.

Depending on your file system. eg: ZFS, BTRFS it may be possible to easily use system snapshots and or sub-volumes. ... I didn't explore these possibilities in detail, because there are no GUIs atm. :) But imo these would be very promising and fast possibilities, since they work very low level.

---- Other OSes

For Mac I'd use Time Machine or/and Time Capsule. It just works and goes out of your way. (Second hand info from a friend of mine, since I don't own a Mac. :)

With Win8/8.1 the built in File History function may be an option, if and only if you use an external or network drive. IMO also the default options are crap. Default it only creates a history, if you include directories into the default windows libraries. eg: Image, Desktop, Download, ... You can't directly select eg: 'd:/' for data only. You have to include it into one of the above system libraries. :/

Kind of a versioning, below.

For my Windows System I also use built in Recovery Points before I install new SW. For testing new SW I start a "throw away" Windows VM, which can easily be reset to a certain saved point. VirtualBox may do the same job, but since Hyper-V is already there. ...

[Spangenhelm]

Hi @pmario sorry i thought i had already responded to you.. thank you for your informations, i have heard of back in time before i think i will try it, i am on linux and i'm looking for something with a gui so it may be a good bet, a last question, just by curiosity: what software do you use when you code ? (Notepad++, Atom, LightTable, Gedit...??), same question to @Jermolene.

Thank you guys

[pmario]

With Ubuntu I did use GEdit first and then Geany. .. but now I use Brackets.io (without Extract) from Adobe.

IMO the ctrl - e quick edit feature is unbeatable. Global search works very well with TW repo. It works for unix, win and mac.

Plugins I use:

• Brackets Git V 0.14.27
• JSHint V 2.2.18
• Show Whitespace V 2.0.1

Brackets uses codemirror as the code editor. ... So it would be possible to create a TW syntax highlighter. ... If I would have more time and motivation ;)

[Spangenhelm]

Looks really nice i'm going to ckeck it out thanks ! On my side i have tried Atom (from Github's team) which is fine but a little slow to me and also LightTable which is good too (with an embbed browser!) but i felt like it is hard to configure.. so i'm still looking :-) +

[pmario]

yup. sry. I deleted it already. Hoped nobody sees it :)

[Jermolene]

@Spangenhelm the two tools that I use are the text editor Sublime Text and GitHub's Mac GUI client. Other than that, I work with all the available OS X browsers and the terminal command line.

[Spangenhelm]

Thanks @Jermolene !

[arcandio]

3 years later and I'm still having this issue. I use dropbox, where my space is limited, and besides that I don't really like the idea that I have to manually set something up to clean out my backups, regardless of whether they're stored on Dropbox or on my harddrive alone, because if I don't pay attention and I work on my wikis a good bit, I could fairly easily fill up a small solid state drive. I don't think the solution of "get more storage" is the right answer to the question of "why does this app create an uncontrolled, endless amount of data."

Was there ever any work on the idea of limiting the download folder? I've got TD 0.0.13, but I don't see any options for that aside from retargeting where to put the downloads.

[TBat]

Are there no plans to implement a Limited Backup feature for TiddlyDesktop? Either by number of backups, or by cache size? As others have said, allowing a program to consume infinite storage for backups isn't a good design, especially when using Dropbox where storage can't be written off as cheap/unlimited.

I've lost wikis before due to corruption while saving, so obviously I don't want to turn backups off completely. But manually pruning excess backups for each different wiki isn't a very desirable alternative.

[dvanbrug]

I had a similar issue to @TangoBat and then found that I had to click the "Advanced" button on the Wiki Picker screen when I first start TiddlyDesktop, which then gave me a checkbox to disable backups.

[mreishus]

I found a python module called rotate backups. You can run it and tell it to keep a certain number of daily, weekly, monthly backups and delete the rest. After installing it, a command like:

rotate-backups --dry-run -H 4 -d 7 -w 4 -m 4 -y 2 ~/Nextcloud/wikis/mywiki.html_backup/

Will do something like what you want. (Remove the --dry-run to make it actually delete and not just report what it would delete.) Then you just need to run it automatically on a schedule (I put it in my crontab.)

[SonephetR2]

Another solution was posted on Reddit, relying on tweaking the saving.js file. I don't know Javascript but this seems to be advanced enough to be included in the main release and tweakable from the GUI.