Misfeature so bad it's a bug

[robertII]

I've been a happy persistent-scratch user for many months now, but I just got bitten quite badly.

A couple of nights ago, I as usual left my computer to put itself to sleep. Emacs was running, and I had a bundle of notes in my (persistent) scratch buffer.

The following morning, I realised the machine had rebooted itself (possibly in order to install Windows updates). Emacs was not running, so I started it. And it came up with scratch saying just ;; This buffer is for notes you don't want to save, and for Lisp evaluation. ;; If you want to create a file, visit that file with C-x C-f, ;; then enter the text in that file's own buffer.

What had happened to my persistent notes? I looked at persistent-scratch.el to find where the scratch buffer would be stored, but when I looked, I found just a single newly modified file containing just the standard "This buffer is for notes..." message shown above. The notes I previously had in scratch had disappeared without trace.

Forcibly killing emacs (by shutting down the machine) and then re-starting it had lost all my "persistent" notes. Somehow, "persistent scratch" had either failed to stash the buffer contents on disk, or failed to recognise the stashed contents as valid when Emacs re-started. This seriously undermines the value of the tool: "scratch is persistent unless it vanishes entirely"

[Fanael]

That sounds quite bad, I agree. Can you think of a way to reliably reproduce this issue? It would greatly help me with understanding what the problem is so I could fix it.

[TinaRussell]

Maybe a good work-around for now might be to have, by default, persistent-scratch make a backup of the old file when overwriting, and keep the last few backups around (maybe one backup per session, from the past few sessions). I know persistent-scratch has an existing backup facility, would it be amenable to this?

[Fanael]

Maybe a good work-around for now might be to have, by default, persistent-scratch make a backup of the old file when overwriting, and keep the last few backups around

I agree, but that's exactly how persistent-scratch backups (are supposed to) work: when persistent-scratch-save is called with nil path, which is the usual case, the data is saved to the persistent scratch file proper and to the backup directory when one is set. Backups are never deleted by default, you have to set your own backup filter function (which may be time based, cf. persistent-scratch-keep-backups-not-older-than) to get persistent-scratch to delete anything at all.

This issue really leaves me mystified, because Emacs is normally very good about data durability, fsync when saving is the default, so by the time persistent-scratch-save returns, the data is physically on the disk, assuming the OS properly honors fsync like Linux does.

[TinaRussell]

Huh. Well, persistent-scratch-backup-directory is nil by default, right? So, maybe it just needs a good default value, like (expand-file-name "scratch-backups" user-emacs-directory). Then we’d need a good default value for persistent-scratch-backup-filter… except then we’d have to handle the case of users who prefer to keep their scratch backups forever and so they’ve set persistent-scratch-backup-directory but left persistent-scratch-backup-filter to its nil default. Maybe detect those cases upon upgrade and set persistent-scratch-backup-filter to nil explicitly via customize or something? (…I wouldn’t know how to do that, I’m just spitballing…)