koreader icon indicating copy to clipboard operation
koreader copied to clipboard

KOReader 2024.04 Sourdough - EPUB with custom fonts unusably laggy

Open Darthagnon opened this issue 1 year ago • 47 comments

I have an EPUB I built myself using ABBYY, Sigil and a scan of the book from Archive.org. It is 18MB (due to illustrations) and KOReader reports it as 538 pages on my K4.

The older version of the EPUB, from before I figured out the CSS that only uses fallback fonts, works fine.

I recently added Bembo as the custom font, and added some CSS to apply the font and add drop-caps. I also simplified the CSS (it was rather messy from ABBYY). It works fine in Edge Spartan, SumatraPDF, Calibre and Sigil (though dropcaps and tables seem to render inconsistently). However, in KOReader, the new version takes around 30s to turn 1 page.

Is this just my bad EPUB editing, or is it a bug with KOReader? If anyone could help, I'd sure appreciate it. I'm happy to share or upload the book wherever (NB. it is likely still in copyright).

Darthagnon avatar Aug 11 '24 16:08 Darthagnon

You can share a version scrambled with https://www.mobileread.com/forums/showthread.php?t=267998.

benoit-pierre avatar Aug 11 '24 16:08 benoit-pierre

And please provide the requested details (KOReader version, device).

benoit-pierre avatar Aug 11 '24 16:08 benoit-pierre

KOReader 2024.4 Kindle 4

Scrambled epub: https://drive.google.com/drive/folders/1Yzdeu9-nzvVhhjMS4tbnilsfHWDm2wfT?usp=drive_link

Please note, the scrambling plugin removed all the fonts and images and decreased the size by a lot, which more or less invalidates it for the test.

Darthagnon avatar Aug 11 '24 16:08 Darthagnon

After exiting the EPUB, KOReader as a whole slows down a lot.


I've also notice the same issue with another (very long) EPUB (800p), even though it's only 600kb and each chapter is separated. From its metadata, someone generated it with an old version of Calibre. As that one doesn't have images, it might scramble better. I'll upload it to the Google drive folder I linked.

Darthagnon avatar Aug 11 '24 16:08 Darthagnon

It is useful to enable the "Memory used" indicator in the status bar. Kindles slow down when it shows about 70 Mb and upper.

hius07 avatar Aug 11 '24 16:08 hius07

Is the memory cleared on exiting from a book? I'll see if I can find that option to enable it. Notably, I did split the epub into 1 xhtml per chapter, so it shouldn't use that much RAM

Darthagnon avatar Aug 11 '24 16:08 Darthagnon

No, only restarting helps.

hius07 avatar Aug 11 '24 16:08 hius07

That's not exactly true, there are some things left cached, but memory use should still come down after closing.

You also might want to try the latest release.

benoit-pierre avatar Aug 11 '24 16:08 benoit-pierre

Tested right now. Opened big pdf, turned pages, got 68 Mb. Closed pdf, opened small epub. M 71 Mb. Kindle Voyage.

hius07 avatar Aug 11 '24 16:08 hius07

Actually no slowdown with 70 Mb memory used. Seems the threshold is closer to 100 Mb.

hius07 avatar Aug 11 '24 16:08 hius07

MuPDF is loaded unconditionally at startup (used by some widget, e.g. for the dictionary lookups). The koreader-cre library on first opening a CREngine document.

Additionally, fonts, loaded when using menus, also increase memory use.

benoit-pierre avatar Aug 11 '24 16:08 benoit-pierre

But really things have improved, compared to a couple years ago.

benoit-pierre avatar Aug 11 '24 17:08 benoit-pierre

memory use should still come down after closing.

Lua stuff may take some more time, because that doesn't guarantee a GC sweep.

(Opening a new document will, on the other hand).

NiLuJe avatar Aug 11 '24 17:08 NiLuJe

I did split the epub into 1 xhtml per chapter, so it shouldn't use that much RAM

That's irrelevant for CRe, it's not a chunked renderer.

NiLuJe avatar Aug 11 '24 17:08 NiLuJe

I recently added Bembo as the custom font, and added some CSS to apply the font and add drop-caps.

What if you don't embed the font, and instead set-it up as your default font in KOReader?

(Do try with a current release before all else, though).

NiLuJe avatar Aug 11 '24 17:08 NiLuJe

I'm testing adding Bembo as a font to KOReader, though I have also just found that after rebooting and relaunching without framework, I am able to comfortably read the problematic ebook with the custom embedded font (though it still doesn't look quite right haha).

Darthagnon avatar Aug 11 '24 18:08 Darthagnon

You're going to make @offset-torque sad by trusting third-party doc over the fancy user guide ;).

NiLuJe avatar Aug 11 '24 18:08 NiLuJe

Nice guide though. ;-)

Frenzie avatar Aug 11 '24 18:08 Frenzie

You're going to make @offset-torque sad by trusting third-party doc over the fancy user guide ;).

Thank you for the link! tbh the one I posted was just the first thing DuckDuckGo showed me. Wondering, is the official guide available as an EPUB or HTML with the KOReader installation these days? I remember there was always a quickstart guide on first install that I could never find again, but it didn't seem as comprehensive as the manual you linked.

Darthagnon avatar Aug 11 '24 19:08 Darthagnon

You're going to make offset-torque sad by trusting third-party doc over the fancy user guide ;).

:) Now I am curious, what percentage of our users are aware that we have a user guide.

Wondering, is the official guide available as an EPUB or HTML with the KOReader installation these days?

We are thinking of adding it yes. Needs some work for e-ink screens.

offset-torque avatar Aug 11 '24 20:08 offset-torque

Now I am curious, what percentage of our users are aware that we have a user guide.

I was thinking one day, how about we add a popup notification (with a required tap to dismiss) that appears after each OTA update?

For example, KOReader says, click OK to restart in the end of the OTA download, restarts, then after opening says “As a reminder, there is a User Guide at http://koreader.rocks/user_guide/“

This will fix awareness issue :D

mergen3107 avatar Aug 11 '24 21:08 mergen3107

This will fix awareness issue :D

At the cost of making me want to murder it with fire ;p.

NiLuJe avatar Aug 11 '24 22:08 NiLuJe

hey @NiLuJe I was going to test loading a large 70+ MB ebook to see how the K4 handles it but was greeted with a "Full storage" warning but --I only have like 33MB worth of books saved on device, wtf?-- so I started trying to figure out what's going on and there appears to be a hidden folder called .Spotlight-V100 that is taking up 870MB (out of 1.4GB). Any ideas what that is and what to do about it?

Commodore64user avatar Aug 11 '24 22:08 Commodore64user

Some macOS thing? https://superuser.com/questions/429091/how-can-i-permanently-delete-hidden-files-folders-on-mac-usb-drive

benoit-pierre avatar Aug 11 '24 22:08 benoit-pierre

yes, it was in fact a macOS thing, just reclaimed back 870MB. thanks @benoit-pierre

Commodore64user avatar Aug 11 '24 23:08 Commodore64user

a large 70+ MB ebook

FWIW, size is relative.

69MB of JPGs? Nice!

70MB of text? You're in for a bad time.

68MB of CSS or random HTML crap? You're probably also in for a bad time, but differently than the previous case ;p.

NiLuJe avatar Aug 12 '24 01:08 NiLuJe

Is there still some issue to be looked at here?

I downloaded Vanilla/..1.7..., and it loads and render fast (< 1s) on the emulator. (The kerning or its absence with that font is rather ugly: image)

poire-z avatar Aug 17 '24 21:08 poire-z

I also have problem with large EPUB getting worse recently. I have a large (~75MB) EPUB with lots of text and images (and probably having some bad formatting.) Previously it might took some time for KOReader to open it, but with recent builds it just gets stuck. (I tried to wait for more than half an hour, and it was still stuck.)

ilyats avatar Aug 24 '24 11:08 ilyats

How "recently" ? With v2024.07 or older - or since a nightly AFTER v2024.07 ? Looks like we still have some bug with an optimisation that went into a post-v2024.07 nightly: https://github.com/koreader/crengine/pull/584#issuecomment-2307889188 - and you may also get stuck in an infinite loop :/ Can you share that EPUB (or if not free, a scrambled version of it if you can reproduce the issue with that scrambled version of it) ?

poire-z avatar Aug 24 '24 11:08 poire-z

Pinging @ilyats about previous comment.

poire-z avatar Aug 29 '24 07:08 poire-z