[Bug] Recent update breaks plugin

[rutgervandenberg]

KOreader Sync plugin version

0.7.1

KOreader version

2025.04

Operating System

Linux

Connection type

Wireless (over wifi)

Describe the bug

After the recent updates the plugin seems to be broken on my machine.

How to reproduce

Start calibre, connect KOreader to calibre through the wireless interface.

Expected behavior

I expect the sync button to appear in the menu at the top - it does not.

Provide details output from plugin pop-up window

Screenshots

No response

Any additional info

• I've reinstalled the plugin to no effect. I'm not sure if some state is saved after uninstall or if a fresh install would also fail.
• When I click 'customize plugin' through the Calibre preferences I get a popup saying

calibre, version 8.2.100 ERROR: Must restart: You must restart calibre before you can configure the KOReader Sync plugin

I have of course already restarted calibre (before and after reinstalling the plugin).

[kyxap]

Please run calibre in debug and provide output with error before the crash calibre-debug -g. At this point I do not know how to help you since we are using latest version with no issue

[sokolgeo]

Also on Linux, USB mounted Android reader device, with new version of the plugin trying to sync crashed Calibre with the following log trace. I had to SKIP some filenames, do let me know if the log is sufficient:

DEBUG:   85.1 koreader:action:KoreaderAction:sync_to_calibre:Trying to get sidecar from <calibre.devices.folder_device.driver.FOLDER_DEVICE object at 0x7f8c1e5ce380>, with sidecar_path: /run/user/1000/gvfs/mtp:host=rockchip_Supernote_A5_X_SN100B20012692/Supernote/INBOX/books/SKIP.sdr/metadata.epub.lua
DeviceJob: metadata_downloaded: Starting set_books_in_library
DeviceJob: set_books_in_library: books to process= 128
Job: 2 Get list of books on device finished
No details available.
DEBUG:   85.1 koreader:action:KoreaderAction:get_sidecar:Parsing: /run/user/1000/gvfs/mtp:host=rockchip_Supernote_A5_X_SN100B20012692/Supernote/INBOX/books/SKIP.sdr/metadata.epub.lua
DEBUG:   85.2 koreader:action:KoreaderAction:sync_to_calibre:sidecar_contents
DeviceJob: set_books_in_library finished: time= 0.1802978515625
DeviceJob: metadata_downloaded: updating views
DeviceJob: metadata_downloaded: syncing
DeviceJob: metadata_downloaded: refreshing ondevice
DeviceJob: metadata_downloaded: sending metadata_available signal
DeviceJob: 2 Get list of books on device callback returned
DEBUG:   85.6 koreader:action:KoreaderAction:sync_to_calibre:sidecar_contents is found!
DEBUG:   85.6 koreader:action:KoreaderAction:sync_to_calibre:transforming value for #current_percent_read
DEBUG:   85.6 koreader:action:KoreaderAction:sync_to_calibre:transforming value for #current_percent_read_integer
DEBUG:   85.6 koreader:action:KoreaderAction:sync_to_calibre:subproperty "rating" not found in value
DEBUG:   85.6 koreader:action:KoreaderAction:sync_to_calibre:transforming value for #bookmarks_and_highlights
QObject::setParent: Cannot set parent, new parent is in a different thread
Segmentation fault

[sokolgeo]

In my case unchecking the "Bookmarks column" in sync settings allowed plugin to run without error. The custom database column created for this field sync was Long text with Interpret this column as set to Plain text formatted using markdown - just exactly as in setup instructions on the plugin GitHub page.

PS: I guess I should move this bug report into separate ticket? The only relevance with earlier bug report in the current ticket is the new version of the plugin?

[rutgervandenberg]

Relevant section of logs:

Using calibre Qt style: True
Traceback (most recent call last):
  File "/nix/store/agm60a7gb6s26ny6c1lvzkwwzqyhv3pg-calibre-8.2.100/lib/calibre/calibre/gui2/ui.py", line 148, in __init__
    ac = self.init_iaction(action)
         ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/agm60a7gb6s26ny6c1lvzkwwzqyhv3pg-calibre-8.2.100/lib/calibre/calibre/gui2/ui.py", line 166, in init_iaction
    ac = action.load_actual_plugin(self)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/agm60a7gb6s26ny6c1lvzkwwzqyhv3pg-calibre-8.2.100/lib/calibre/calibre/customize/__init__.py", line 658, in load_actual_plugin
    ac = getattr(importlib.import_module(mod), cls)(gui,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/fqm9bqqlmaqqr02qbalm1bazp810qfiw-python3-3.12.9/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "/nix/store/agm60a7gb6s26ny6c1lvzkwwzqyhv3pg-calibre-8.2.100/lib/calibre/calibre/customize/zipplugin.py", line 210, in exec_module
    exec(compiled, module.__dict__)
  File "calibre_plugins.koreader.action", line 17, in <module>
    import brotli
ModuleNotFoundError: No module named 'brotli'
calibre 8.2.100  embedded-python: False


***snip***

Turns out the problem is that the recent update added a dependency on brotli. The NixOS package for calibre which I've installed does not include brotli

From a quick search on github calibre itself doesn't actually seem to use brotli even though it packages it. It's only included as a dependency of py7zr, which in turn is only needed if you try to open 7zip archived files, which is probably why it has never come up. Manually adding brotli is a good (temporary) workaround.

[kyxap]

hey @jbhul there some other bugs reports, any chance you can take a look ?

[kyxap]

It seems like we are talking about 3 different issue here...

[jbhul]

The original issue is related to brotli which is a required dependency for ProgressSync (the response is compressed with brotli). The solution here would be to move the import to the progress sync method in a try/except and instruct the user to install via pip or similar. Or package it alongside? @kyxap Any advice or preferences here? For some reason I thought this was a default package, my apologies there.

The other issue mentioned in this thread is something related to threading and bookmarks? Will have to look into this when I'm back at my PC.

[kyxap]

@jbhul I'm not using ProgressSync so not sure if we actually need that compression. Better to look for any alternatives if it still needed and avoid any extra steps or dependency for end user

[jbhul]

@jbhul I'm not using ProgressSync so not sure if we actually need that compression. Better to look for any alternatives if it still needed and avoid any extra steps or dependency for end user

The response from the PS Server is encoded/compressed and this needs to be de-compressed.

[kyxap]

@jbhul got it, it seems like I do have brotli installed locally (not sure why hehe) so I was not getting any errors. So yeah I think we should ask user to install it only if they are planning to use ProgressSync, if my understanding is correct and this can be done in python

[sokolgeo]

Two points, please:

1. I get a crash when enabling the "Bookmarks column" for synchronisation - the "MD5 Hash column" is always not mapped for synchronisation. So, in theory, the plugin should not even attempt ProgressSync?

2. Installing brotli does not seem to be easy either. My system seems to have user in Python 3.10 and Calibre Python 3.12. Therefore simply installing brotli into my Python would not help?

ChatGPT says:

Your version of Calibre is installed via Nix and uses Python 3.12 from the Nix store:

/nix/store/...-python3-3.12.9/lib/python3.12/...

That environment doesn't see your system Python or packages installed via apt or pip. If you want to fix the Calibre plugin crash (caused by missing brotli), do this:

Create a shell.nix file:

with import <nixpkgs> {};


mkShell {

  packages = [

    calibre

    (python312.withPackages (ps: with ps; [ brotli ]))

  ];

}

Run this in the terminal:

nix-shell

calibre

This should launch Calibre with Python 3.12 + brotli available. But, for the record, I seem not to have nix-shell available as terminal command/installed application so no idea if that command fixes the problem.

[wfdewith]

mkShell {
  packages = [
    calibre
    (python312.withPackages (ps: with ps; [ brotli ]))
  ];
}

This does not work unfortunately, as the Python environments for Calibre and your Nix shell will be separate, so brotli will still not be available in the Calibre plugins.

---

@kyxap I think the only real solution to the problem of dependencies in a Calibre plugin in general is to vendor (i.e. copy-paste the code) them in the plugin zip file. This is what FanFicFare does, for example.

Vendoring dependencies is can only really be done with pure Python dependencies, which brotli is not because it's just a Python wrapper over a C library. FanFicFare solved this by using a pure Python implementation of Brotli: https://github.com/sidney/brotlidecpy.

I'm willing to make a PR for migrating to a vendored brotlidecpy if you want.

[kyxap]

@jbhul any objections? or maybe other ideas on how to resolve it?

@wfdewith I'm fine we can add some code to the zip, but I'm not using ProgressSync and not sure how needed this feature is

[wfdewith]

Actually, I realize there is a much easier fix: we can just remove the Accept-Encoding: gzip, deflate, br header from the ProgressSync requests, so that the ProgressSync client no longer advertises any compression support.

[kyxap]

@wfdewith awesome!