koreader
FR: simplify backups procedures
Does your feature request involve difficulty completing a task? Please describe. I have to make some fixes to my Kobo Glo, and after that I'm afraid I'll have to reinstall KOReader, or have it wiped clean at least. I found no way on the UI to create a backup for a clean reinstall.
The second Google result points to a forum post from 2021, and only the fourth points to the KOReader user guide - but the wrong part of it. Not that the fact the guide is a single-page, forever-scrolling document helps, given there's not even a section for Google to link to. The actual backup procedure is listed under an anchor-less answer on the troubleshooting section, impossible to find unless you use text search on the page.
Not to mention, files to back up may change between versions, and there's no way to link the online user guide to the version I'm running. For instance, for some reason I have no way of knowing why, my device only has defaults.custom.lua and defaults.lua, but no defaults.persistent.lua. Now I'm afraid my backup won't work properly. And I don't even know what that file is about!!! And I shouldn't need to know, of course, because as a user, I use KOReader to read books, not to develop or fiddle with its filesystem.
Describe the solution you'd like The backup/restore procedure should be a one-click action. You should not expect users to plug their device and copy weird folders and lua files, listed in a dark corner of the user guide (or worse, on an old forum post). Ideally, there should be a UI option in some menu that would create a zip folder with all the relevant files. Then, on a new install, that same option would allow the user to use that zip file to restore its settings. Optionally, that backup command would also back up all e-books and their sidecar files/folders.
Bonus to help users find their way around in the meantime:
- the guide's ToC should list the troubleshooting questions?
- troubleshooting questions should be anchored, so they can be linked to
- [ideal world but off-topic] guide should be split into multiple pages, helping with indexing and navigation on computers
Describe alternatives you've considered There's no alternative. The backup option is completely missing and having to dig for it just terrible for user experience, makes it feel like an incomplete piece of software, instead of a mature and safe-to-use one.
The backup/restore procedure should be a one-click action. You should not expect users to plug their device and copy weird folders and lua files, listed in a dark corner of the user guide (or worse, on an old forum post).
You still have to plug in your device and copy a weird backup file in this scenario, so I'm not convinced it changes much. The logical thing to do is simply to copy the entire folder; the other stuff is merely about what's the least you can get away with.
There's no alternative. The backup option is completely missing and having to dig for it just terrible for user experience, makes it feel like an incomplete piece of software, instead of a mature and safe-to-use one.
Good!
Because is NOT safe to use. You're running it as root on a embedded device. There're seven million ways to shoot yourself on the foot, albeit they're not documented for a very good reason.
Implementing a backup/restore button on the UI it isn't going to work. Because a backup/restore button assumes you can backup from whatever device runs KOReader and restore in whatever other device you want. That's not possible unless you do it yourself, do trial and error and discover all corner cases that makes impossible to make a backup on A and restore it on B. Then submit a workaround for each corner case.
And for a backup from device A to restore on the same device the current implementation is simpler and makes users aware of what they're dealing with: a community supported, provided as is, program that runs on embedded devices.
And what you need to backup:
- on embedded devices the whole installation dir.
- On regular OSes there's no need to backup anything as the installation dir is decoupled from the data dir. So you're able to wipe the entire program without affecting user settings. Reinstalling the program will bring you back your own settings.
For regular OSes we following each OS best practices (except on Android, which sucks badly), so you can commit your settings as part as your dotfiles, like in any other regular application. In android data dir is /sdcard/koreader.
This is for backuping app user data. Document data go, by default, in sidecar directories near to the documents themselves, so you'll need to backup each document and its sidecar directory.
Once we're all on the same page I would agree with you. Instructions above can be simplified even more and make a nice wiki page or tweak the user guide. Those contributors are very welcome :)
Responding to the user guide parts as the author.
This is what I did Igor:
- I opened our guide
- Searched for "backup"
- Pressed next
Second result is "Backing up your KOReader settings".
And you call this "listed in a dark corner of the user guide".
I measured with a chronometer and this very simple process took less than 15 seconds (including page load time). And I did this on a 10 year old, slow-as-a-turtle Android tablet to prove a point.
And you call this "terrible for user experience".
impossible to find unless you use text search on the page
I am having trouble parsing this sentence. Are you really complaining about using search to find things on a web page? How else you can find things on a web page?
The second Google result points to...
That's Google's problem, we are not responsible for their crappy results and can't do anything about it. See, now this is a terrible user experience you can complain about.
I just checked and our guide is the first result on Yandex and Bing when I asked "How can I backup my koreader settings?". Also our guide is indexed by all AI search engines like ChatGPT, Perplexity, Phind. If you ask them, they can give you step-by-step instructions.
In short, your over-dramatic complaints has no basis in reality. Our user guide is objectively fast to open, search and download on even the slowest computers and this was praised by other novice and expert users.
As a closing note, I advise you to learn how to use Ctrl+F. It is not that hard and as a programmer it will improve your life tremendously. (Also use a real search engine)
to see how real reports about user experience are turned down as "learn to be a real programmer" :clown_face:
You still have to plug in your device and copy a weird backup file in this scenario, so I'm not convinced it changes much. The logical thing to do is simply to copy the entire folder; the other stuff is merely about what's the least you can get away with.
I can't disagree more. A file named "backup.zip" is way different than having to sift through folders to find .adds/data/settings.persistent.lua. And the said guide doesn't say about copying the whole folder; in my experience, that's not a good idea when you might be overwriting stuff that shouldn't be. Again, that's why the backup procedure should be dictated by the system, not a textual guide for the user to follow.
a backup/restore button assumes you can backup from whatever device runs KOReader and restore in whatever other device you want. That's not possible unless you ....
...Unless you implement a backup system that takes note of the version and model used, and allows you to restore on the same model or similar models or whatever are the constraints, for instance. Shouldn't be that hard to say "you should not restore this backup into a different model/system" and have the user acknowledge it before getting the file. If there's a guide explaining which files to copy, a backup is doable and automatable. Not to mention that guide doesn't even talk about transferring to other devices, so it's either missing explanations or you're being excessively careful (but I'm not blaming you, I totally get the risk there). Then, again, the user is not supposed to know this, but you devs should, and hence it can be coded into KOReader.
makes users aware of what they're dealing with: a community supported, provided as is, program that runs on embedded devices
Instead of complaining back, I will skip and agree with your last statement: there is room for improvement, and the first step for it is when maintainers agree with it. After all, the community is not only based on developers and bug-fixers, but also on problem reports and improvement suggestions. Not everyone has the same point of view, and that's why community-driven software can be powerful. Or destructive, when you're greeted with "learn to be a proper programmer" by someone that doesn't even have a name or avatar.
For regular OSes we following each OS best practices (except on Android, which sucks badly)
Can't help but also respectfully laugh at Android :joy:
And now, the troll on the room:
@offset-torque glad you know where's the KOReader guide. I guess that's because your profile is solely for working on it. I don't. It's just a tool I use to read my ebooks, hence I went to Google. Glad you're so amazingly fast like that; it took me a lot more to sift through what could be trustworthy and what could not, given the first results were not official information and when it was, it that didn't even mention the backup procedure directly. Not to mention that, if you have to find-in-page for what you need, the page navigation has failed.
I'm mesmerized at your behavior: you're basically telling the user how to behave, instead of offering a nice user experience so they don't have to look around. The problem is exactly needing to Google and Ctrl+F to find a simple procedure such as a safety backup - and then getting a list of weird files to copy. Users are not developers with the guide at hand and ready to go digging for information on a simple procedure as creating a backup. Users want to use the tool.
That's Google's problem, we are not responsible for their crappy results and can't do anything about it
You're wrong again. You ARE responsible for crappy Google results, when you pretend to know nothing about search engines and maintain a single-page guide which doesn't even has proper anchors and titles are styled div tags.
It's easy to find it as the first result if you engines are used with your KOReader usage. Not my case. Not to mention you just searched for the exact sentence from the guide :clown_face: you're making a silly of yourself. My Bing results are even worse than Google's, if you skip the small boxes of related questions. I get an Android related result, a video about KOReader settings, a video about backing it up... KOReader guide is the 6th(!!!) normal-sized result.
As a closing note: isn't it completely bonkers I have to tell you how to write plain HTML and do basic SEO, since you're a great programmer that know how to use Ctrl+F?
PS: I'm not being overly-dramatic, I'm exposing a frustration from a user point of view. If that hurts you so bad, as a professional you'd better help improve it, or at least accept it as a missing feature, instead of harass back the user and invalidate their point of view because you know better. I'm surprised how offended you felt with such tangential suggestions I made. This guide must indeed be your most prized contribution to humankind.
isn't it completely bonkers I have to tell you how to write plain HTML and do basic SEO
I searched for just two words as you wished and first result. Looks very optimized to me:
Our page insight report. Our FCP is 300ms. LCP is 500ms. Our guide opens faster than the Google's empty page with a 93 score.
What exactly do you want me to improve and how?
...Unless you implement a backup system that takes note of the version and model used, and allows you to restore on the same model or similar models or whatever are the constraints, for instance. Shouldn't be that hard to say "you should not restore this backup into a different model/system" and have the user acknowledge it before getting the file. If there's a guide explaining which files to copy, a backup is doable and automatable. Not to mention that guide doesn't even talk about transferring to other devices, so it's either missing explanations or you're being excessively careful (but I'm not blaming you, I totally get the risk there).
None of the maintainers in here is interested in fixing that for you :).
So either provide feedback about the instructions themselves on my previous post or receive the software as-is and do some google foo.
My suggestions were made before your complaints and somewhere in between my responses. Summarizing what comes to my mind:
- split into multiple pages, so you don't have to scroll to look for stuff, and instead need to use ToC to navigate exactly to what you want. This is specially helpful in small devices.
- This also helps with SEO, and that's why the only result in the searches is named "User Guide" instead of "Backing up your KOReader settings".
- it looks like the guide is built with Jekyll, so not sure how achievable this would be - but titles should be title tags (
h1,h2...), not only styleddivtags. This is true even in the largest titles of the page. This helps with page structure and SEO. - Last but not least, it's not exactly welcoming to see on the first fold a guide on how to read a guide. As I said before, the user wants to use stuff, wants to take the problem out of the way, not being taught how to read to then find what they want. You should be presented with a list of topics, a ToC, a search field... And I'm telling that from a vertical screen; on a normal one, the first fold stops at the color coding description....... I would even go ahead and suggest replacing all those colors with milder versions + icons; it would start to make it more interesting to read on, you guessed right, an e-ink device.
- additionally, Lighthouse gives other SEO suggestions besides mine, such as
img[alt], meta descriptions (which would be quite beneficial to individual pages), among other minor ideas.
Why would the user bother with FCP in a text guide? They want to find what they're looking for, not having the fastest result ever.
But yeah, fights aside, I'm somewhat puzzled with your search results. That's exactly what I used in my first search. Although I live in Brazil, my computer and everything is set to English, so it's not biased by Portuguese results. Bing even found a related issue from two years ago here at GitHub (and I even did a search for backup here, but got tired after the third result page and decided to write up) - #8885.
@pazos I don't want anything fixed for me. I made a feature request to improve the software for everyone that uses it. I'm not the first one asking how to backup it - given there's posts from years ago (forum and GitHub) about the same thing, and one ends up as the first search results on the web; and among that, a bunch of video tutorials on how to do it properly. No questions would be asked and no tutorials would be needed if this was an option on the device. And the guide wouldn't even need to have a file list.
If the standard practice here is to yell back at users and throw suggestions on the trash, it would be helpful if this was described in the issue template :).
I've already provided feedback about your instructions, about what's found on the guide and on web searches: users should not be supposed to dig for files to backup in an SD Card, when there's, surprise surprise, actual software that could do this in a much more reliable fashion. I'll just quote myself, given you didn't find my suggestions and you all preferred to complain back at the user:
Ideally, there should be a UI option in some menu that would create a zip folder with all the relevant files. Then, on a new install, that same option would allow the user to use that zip file to restore its settings. Optionally, that backup command would also back up all e-books and their sidecar files/folders. implement a backup system that takes note of the version and model used, and allows you to restore on the same model or similar models or whatever are the constraints, for instance. Shouldn't be that hard to say "you should not restore this backup into a different model/system" and have the user acknowledge it before getting the file.
Main reasoning why the backup should be done by the system instead of relying on the user to manually copy its stuff:
Files to back up may change between versions, and there's no way to link the online user guide to the version I'm running. For instance, my device only has
defaults.custom.luaanddefaults.lua, but nodefaults.persistent.lua. Now I'm afraid my backup won't work properly. And I don't even know what that file is about!!! And I shouldn't need to know, of course, because as a user, I use KOReader to read books, not to develop or fiddle with its filesystem.
If the standard practice here is to yell back at users and throw suggestions on the trash, it would be helpful if this was described in the issue template :).
Yep, trashy suggestions go to the trash :).
I have supported KOReader for many years and yet encountered that question just a few times. People were mostly happy you only need to backup an entire folder and nobody felt entitled to request a "backup/restore" UI button as the only way to simplify backup procedures and demand that we should work on that to make things less terrible.
We're fine with being terrible as far as the feature you're requesting. We don't need one and you don't seem to care about other ways of simplifying, and I mean:
Instructions above can be simplified even more and make a nice wiki page or tweak the user guide. Those contributors are very welcome :)
And, to be clear, A backup/restore option is a nice addition: please do it elsewhere. You can write a plugin, publish somewhere and help those users who feel afraid their backups are not complete.
We don't want one in here unless it comes from a trusted source that has shown a record of supporting the features they've implemented. Otherwise this is a maintainer's nightmare.
Let me summarize: We don't need SEO because as I showed you we are number one result (on a cookieless private window search) using a:
- Giant search engine
- Tiny search engine
- Russian search engine
So it is not a SEO problem, it is a Google problem. This person can help you I think:
By the way our accessibility and SEO scores are low because we don't have ALT tags on images. Who needs ALT tags? Blind people. I assure you no blind person is using our guide. How do I know? Because KOReader doesn't have any blind users.
I don't have the time and patience to continue this non-productive discussion. So I am just reminding you this: "KOReader is free software. So you are free not to use it." And if you have a problem with the guide, you are also free in the same way. I don't owe you anything. This is the offer, take it or leave it.
There are tons and tons of reader apps on every platform including native and 3rd party apps. If a simple Ctrl+F is too hard for a user, surely they will be happier with a less complex reader than KOReader.
I can't disagree more. A file named "backup.zip" is way different than having to sift through folders to find
.adds/data/settings.persistent.lua. And the said guide doesn't say about copying the whole folder; in my experience, that's not a good idea when you might be overwriting stuff that shouldn't be. Again, that's why the backup procedure should be dictated by the system, not a textual guide for the user to follow.
Right, in that case I'd suggest the problem is that the guide doesn't mention the simplest thing to do is just to copy the entire folder. (Or to do nothing at all, since I doubt there's much you can't set up quickly again.)
Right, in that case I'd suggest the problem is that the guide doesn't mention the simplest thing to do is just to copy the entire folder. (Or to do nothing at all, since I doubt there's much you can't set up quickly again.)
Do nothing at all might screw you since it wipes user patches, user style tweaks, dictionaries, centralized metadata, history, lookup history and a few other things.
OTOH backup the whole directory has no problems and it is easy to restore (you don't need to reinstall the app, just the launcher for the platform in case you did a factory reset)
I am writing this reply to show that I am not dismissing a valid user feedback (as you want to make it seem like) but to prove that you have no idea about the concept of user experience that you are repeating continuously. And also to use this reply as a reference if any other user repeats this kind of nonsense in the future.
OK, I listened to you and splitted our guide into 12 html files according to the chapters. A user is curious about ligatures. We don't have a chapter or heading dedicated to ligatures. They are mentioned in the text in two places. Current guide steps: Ctrl+F > liga - Takes 3 seconds. In your version, user has to open all 12 files to do this. USER EXPERIENCE: WORSE
You will say "Well add a search feature". Our budget is zero and Google search is crappy so we added a client-side search system. Just dependencies are bigger than our whole guide (which is 450 KB) so now guide is slower to load. Also Ctrl+F is instant, search boxes are laggy and annoying. Just check any website. USER EXPERIENCE: WORSE
User searched for a more common term on our shiny new search. 36 matches in 8 different files. Now they have to navigate to each file to see if it is what they need. USER EXPERIENCE: WORSE
User just installed KOReader and they thought "Probably I will use this document very much until I learn this program" and they decide to download the guide to their computer because their internet access is slow/metered/intermittent. Current procedure: File > Save as... - Takes 2 seconds. Split files: Click on each section individually to save them (and repeat this after an update). USER EXPERIENCE: WORSE
You said guide should be suitable for e-ink devices. We have plans yes. Oopsie, our search is not working on the devices. So users now have to open those 12 files on a sluggish device with a sluggish screen? USER EXPERIENCE: WORSE
You say "Well duh just combine all those files into an EPUB so they can search using the app's integrated search". So now we created a single file, which is searchable using the integrated search feature of the platform.
Hey wait a minute! This is the exact definition of our current guide. We came full circle after spending so much time and effort to make everyone's experience worse as I listed above.
And this is how I know you have no idea what you are talking about, but you are too full of yourself to notice this.
You are the customer storming into a shop and shouting "I asked your shop's location to someone and they gave me wrong address and this is all your fault". And when the shop owner says "That person knows our address very well but he intentionally gave you the wrong address, if you asked anyone else they would tell you the correct address" your answer is "Well I want to ask THAT person!".
Please stop wasting our and your time.
On the topic of settings backup/restore, I would like to suggest one possible solution that I’ve had in the back of my mind for a while, and I would like to take this opportunity to share.
I agree that fonts, dictionaries, user patches, books, and book sidecar folders are best backed up manually by the user themselves, due to the involved file sizes and quantities. Also, services that use SQLite databases including reading statistics and vocab builder can already be synchronized to the cloud and restored from there to any device.
However, the following settings are device agnostic (as far as I can tell), and the user could be offered a selection of which settings they would like to back up for easy restore by copying just one file that could also be retrieved from a cloud service.
E.g., the backup dialog could look something like this:
Back up settings:
□ Gestures □ Profiles □ Status bar [1] □ Network settings □ Default document layout settings □ Advanced settings [2]
Save to local file Upload to cloud service
[1] including progress bar [2] particularly for things like tap zones, margins, and page overlap. If there are any device or platform-specific settings there, they should be left out.
Things that are device and/or folder structure specific (e.g., dpi-setting, frontlight, History list of opened documents, Folder shortcuts, etc.) can be left out in my opinion. Those settings can be quickly set up again on the new device and chances are that the folder structure will not be compatible. Similarly, I don’t think there’s a need to automatically back up other histories like dictionary look-ups. (Also, neither document nor look-up histories are settings, so they shouldn’t be expected to be backed up anyway.) Users wanting to rebuild the exact same set-up on a new device will always be better off with copying the complete folder structure (to the extent compatible with the other device).
Besides users switching devices, particularly gestures and profile settings backup/restore would also be very helpful for users with more than one device, who occasionally set up new gestures or profiles and then would like to transfer those to the other device(s) without reprogramming them or copying any files (or partial contents of files) manually. The cloud backup/restore function would be perfect for this purpose.
I’d imagine it to work such that KOReader will copy the relevant sections from the settings files of the items selected in the backup-dialog into a new file, e.g., settings_backup.lua, with separate sections for each back-up category. This file will be saved locally in the koreader directory and/or uploaded to one of the supported cloud services.
Analogously, the restore feature would allow selecting a local settings_backup.lua file or on a cloud server, and just replace the relevant local settings by the ones in the restored file.
I think many users could benefit from a feature like this and I hope there's interest in it.
Thanks for reading.
Just sharing how I manage that:
I unzip koreader.zip and apply my own patch, and when manually updating every other week, I rm -rf koreader/ and start again.
My solution to not lose my settings (and to easily backup everything every other month) is a small bit of my patch:
--- koreader_orig/datastorage.lua 2024-05-26 08:00:52.000000000 +0200
+++ koreader/datastorage.lua 2024-05-26 09:21:34.000000000 +0200
@@ -31,7 +31,8 @@
data_dir = string.format("%s/%s", user_rw, "koreader")
end
else
- data_dir = "."
+ -- datadir outside koreader/ for easier upgrade
+ data_dir = "/mnt/onboard/.adds/mykoreader"
end
if lfs.attributes(data_dir, "mode") ~= "directory" then
local ok, err = lfs.mkdir(data_dir)
May be closed by https://github.com/koreader/koreader/pull/13762. I doubt about too much granulated backup settings, proposing only "main settings", "history", "plugins", "user style tweaks" separation.
Thanks @hius07. Will this work cross-platform? (Probably not for History if the paths are not compatible.) Any plans to enable upload/retrieve to/from cloud storage?
Tha paths are not absolute but like ".\history.lua", so should be cross-platform I hope. No cloud so far.
Yay! Someone indeed had the same idea and thought a backup managed by the system could be useful, instead of all the rage and rant from the guide author - that even argued against himself, as I left the thread after all that hatred and he didn't realize.
Repeating my comment from the PR thread:
I see the following settings related to absolute paths:
download_dir
clipping_dir (within Exporter plugin settings)
folder_shortcuts
home_dir
inbox_dir (for Calibre plugin)
screensaver many paths
screenshot_dir
We can preserve (do not restore) all of them but it's not good for users who back up / restore on the same device.
The alternative is to check the device id and show a warning that the backup is from the other device and some settings may be incompatible.
Or maybe give platform/device-specific settings its own checkbox? Then users who plan to use the backup on a different device can keep it unchecked.