calibre-utf8-path icon indicating copy to clipboard operation
calibre-utf8-path copied to clipboard

改成bin/python-lib.bypy.frozen了

Open pfnsnp opened this issue 5 years ago • 21 comments
trafficstars

修改后重新编译https://github.com/kovidgoyal/calibre/blob/master/bypy/README.rst吧!

pfnsnp avatar Oct 27 '20 11:10 pfnsnp

请问是否有计划支持5.2版本自动化?

YR1044 avatar Jul 28 '21 08:07 YR1044

请问是否有计划支持5.2版本自动化?

暂无……目前 release 里面的 5.1.0 安装包+修改包 套件对我还够用,5.2.0之后需要编译calibre,然后编译方法之前试了一晚上没能把环境配出来于是暂时放弃了(不排除啥时候发现calibre有新功能我很想要了再折腾一遍……)……要是哪位dalao给5.2.0以上搞成功的话,欢迎把链接贴到这个issue里面,贴过来!

snomiao avatar Aug 01 '21 05:08 snomiao

放弃了, 超级难搞。 回 5.1

zhshuixian avatar Dec 06 '21 08:12 zhshuixian

说起来有没有什么能在Docker上把那套东西编译出来的办法……有人知道相关项目吗 查到了以下几个项目,可惜都不是编译用的。只是些web server

snomiao avatar Dec 07 '21 12:12 snomiao

使用开发者模式 修改backend.py这个文件后 都不需要编译他就可以实现这个功能了

wyyqyl avatar Nov 08 '22 14:11 wyyqyl

使用开发者模式 修改backend.py这个文件后 都不需要编译他就可以实现这个功能了

哇确实🍗🍗,这个方法应该对未来的新版本都有效。实测Windows 10 Calibre v6.10.0有效。 正常安装后下载源码然后添加环境变量以及修改源码就可以了!

但是毕竟是开发者模式下运行的,与正常使用相比还是存在一些问题的

c0ur1er avatar Dec 16 '22 19:12 c0ur1er

https://github.com/kurikomoe/calibre-utf8-path 使用开发者模式,把东西打包了一下。同时改了一下,现在可以正常的通过批量重命名的方式刷新旧 calibre 保存的文件的文件名了。

同时编译了一个 6.10 的 Linux 版本。

kurikomoe avatar Dec 30 '22 10:12 kurikomoe

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


https://github.com/kurikomoe/calibre-utf8-path Using the developer mode, I packaged things up. At the same time, it was changed, and now the file names of the files saved by the old caliber can be refreshed normally through batch renaming.

At the same time, a 6.10 Linux version was compiled.

Issues-translate-bot avatar Dec 30 '22 10:12 Issues-translate-bot

说起来有没有什么能在Docker上把那套东西编译出来的办法……有人知道相关项目吗 查到了以下几个项目,可惜都不是编译用的。只是些web server

现在的编译方法是利用 kvm 虚拟机跑编译的,Linux 上编译改改编译脚本还是比较方便的编译一份的。Windows 目前咱自己没啥需求就没有尝试编译。

kurikomoe avatar Dec 30 '22 10:12 kurikomoe

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Speaking of which, is there any way to compile that set of things on Docker... Does anyone know the relevant projects? I found the following projects, but unfortunately they are not used for compilation. just some web servers

The current compilation method is to use the kvm virtual machine to run and compile. It is more convenient to compile a copy of the compiled script on Linux. Windows At present, we have no need to try to compile.

Issues-translate-bot avatar Dec 30 '22 10:12 Issues-translate-bot

现在的编译方法是利用 kvm 虚拟机跑编译的,Linux 上编译改改编译脚本还是比较方便的编译一份的。Windows 目前咱自己没啥需求就没有尝试编译。

@kurikomoe 请教一下如何编译?我正在尝试改造成 github action,但是他这个文档讲的太不清楚了。

rogeryoungh avatar Jan 28 '23 05:01 rogeryoungh

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


The current compilation method is to use the kvm virtual machine to run the compilation. It is more convenient to compile a copy of the compilation script on Linux. Windows At present, we have no need to try to compile.

@kurikomoe Please tell me how to compile? I'm trying to transform it into a github action, but his documentation is too unclear.

Issues-translate-bot avatar Jan 28 '23 05:01 Issues-translate-bot

现在的编译方法是利用 kvm 虚拟机跑编译的,Linux 上编译改改编译脚本还是比较方便的编译一份的。Windows 目前咱自己没啥需求就没有尝试编译。

@kurikomoe 请教一下如何编译?我正在尝试改造成 github action,但是他这个文档讲的太不清楚了。

如果用 github action 的话,直接用官方的编译方法应该是不行的,github action(非 hosted)不支持嵌套虚拟化,因此无法在免费机子上启动 kvm。只能自己看一下 calibre 的 bypy repo [https://github.com/kovidgoyal/bypy.git],把 python 发往 kvm 的编译命令拉出来重写一个编译脚本。(感觉修改bypy 里面的命令发送函数,直接提取那堆 shell 指令还是可行的)

另一个想法是改 libcalibre-launcher.so 这个文件,stirngs 看了一下,感觉最后的 python scripts 似乎是编译到这里了,理论上把这玩意的编译抓出来直接替换 so 就够了

kurikomoe avatar Jan 30 '23 07:01 kurikomoe

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


The current compilation method is to use the kvm virtual machine to run and compile. It is more convenient to compile a copy of the compiled script on Linux. Windows At present, we have no need to try to compile.

@kurikomoe Please tell me how to compile? I'm trying to transform it into a github action, but his documentation is too unclear.

If you use github action, it should not be possible to directly use the official compilation method. github action (non-hosted) does not support nested virtualization, so you cannot start kvm on a free machine. I can only look at caliber's bypy repo [https://github.com/kovidgoyal/bypy.git], pull out the compilation command sent by python to kvm, and rewrite a compilation script. (I feel that it is still feasible to modify the command sending function in bypy and directly extract the pile of shell instructions)

Issues-translate-bot avatar Jan 30 '23 07:01 Issues-translate-bot

昨天有空的时候跟了一下编译脚本。 TL;DR 执行 CALIBRE-ROOT/bypy/linux/main.py 的话可以绕过虚拟机编译出 frozen 文件。但是需要自己处理 python packages 依赖。

BYPY_ROOT=$PWD/.. python3 bypy/linux/__main__.py  
sudo ln -sf /tmp/t/frozen-*/lib/libcalibre-launcher.so /opt/calibre/lib/libcalibre-launcher.so 
sudo ln -sf /tmp/t/frozen-*/lib/calibre-extensions/python-lib.bypy.frozen  /opt/calibre/lib/calibre-extensions/

(bypy.git 和 calibre.git 在同一个目录下,在 calibre.git 下执行,需要手动修改 bypy.git/**/constants.py 的一些常量定义)

分析:

  1. frozen 文件实际上是把各种依赖的 pyc 文件打包起来,之后 bypy-frozen-importer 负载 hook python import 机制从 frozen 文件解包 pyc 执行
  2. frozen 在打包的时候会生成一个一次性的 bypy-byte-index.h 的文件用于记录 pyc 文件在 frozen 里面的偏移,但是编译完之后由于 strip 的问题, header 文件里面的常量定义拿不到了,因此想解包 frozen 替换文件后再重新打包回去会比较难(可能分析 pyc 文件头能做到之类的)。
  3. 打包同时也会打包对应的 python dep 文件。在 manjaro 上用 python3.10.9 的话,手动按报错一个一个装 dep 能拿到一个勉强能跑的版本(一堆 import 没处理,但是 calibre 启动,转换 epub->mobi,设备识别正常) 需要替换 .frozen 文件和 .so 文件,同时提供/复用原本的 lib

相关 repo(注意,需要手动 bypy/linux/__main__.py 里面的一堆路径,同时 bypy.git 需要根据 0001 patch 改动一下) https://github.com/kurikomoe/calibre-utf8-path/tree/poc

编译好的文件: https://github.com/kurikomoe/calibre-utf8-path/blob/poc/calibre-patch-poc.txz

kurikomoe avatar Feb 02 '23 03:02 kurikomoe

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


When I was free yesterday, I followed the compilation script. TL;DR If you execute CALIBRE-ROOT/bypy/linux/main.py, you can bypass the virtual machine and compile frozen files. But you need to handle python packages dependencies yourself.

BYPY_ROOT=$PWD/.. python3 bypy/linux/__main__.py
sudo ln -sf /tmp/t/frozen-*/lib/libcalibre-launcher.so /opt/calibre/lib/libcalibre-launcher.so
sudo ln -sf /tmp/t/frozen-*/lib/calibre-extensions/python-lib.bypy.frozen /opt/calibre/lib/calibre-extensions/

(bypy.git and calibre.git are in the same directory and executed under calibre.git, some constant definitions of bypy.git/**/constants.py need to be manually modified)

analyze:

  1. The frozen file actually packs various dependent pyc files, and then bypy-frozen-importer loads the hook python import mechanism to unpack the pyc from the frozen file for execution
  2. When frozen is packaged, it will generate a one-time bypy-byte-index.h file to record the offset of the pyc file in frozen, but after compiling, due to the problem of strip, the constant definition in the header file takes Not anymore, so it will be more difficult to unpack the frozen replacement file and then repack it back (maybe it can be done by analyzing the pyc file header).
  3. Packaging will also package the corresponding python dep file. If you use python3.10.9 on manjaro, you can manually install dep one by one according to the error report to get a version that can barely run (a bunch of imports are not processed, but caliber starts, converts epub->mobi, and the device recognition is normal) Need to replace .frozen files and .so files while providing/reusing the original lib

Relevant repo (note that a bunch of paths in bypy/linux/__main__.py need to be manually edited, and bypy.git needs to be changed according to 0001 patch) https://github.com/kurikomoe/calibre-utf8-path/tree/poc

Issues-translate-bot avatar Feb 02 '23 03:02 Issues-translate-bot

我们不用很麻烦很累就可以用上中文路径: https://github.com/snomiao/calibre-utf8-path/issues/6#issuecomment-1479487070

Cirn09 avatar Mar 22 '23 15:03 Cirn09

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


We can use the Chinese path without much trouble and fatigue: https://github.com/snomiao/calibre-utf8-path/issues/6#issuecomment-1479487070

Issues-translate-bot avatar Mar 22 '23 15:03 Issues-translate-bot

新版的 Patch 方案: https://github.com/Cirn09/calibre-do-not-translate-my-path

Cirn09 avatar Mar 28 '23 11:03 Cirn09

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


The new version of the Patch scheme: https://github.com/Cirn09/calibre-do-not-translate-my-path

Issues-translate-bot avatar Mar 28 '23 11:03 Issues-translate-bot

great job!

2023年3月28日(火) 20:53 Cirn09 @.***>:

新版的 Patch 方案: https://github.com/Cirn09/calibre-do-not-translate-my-path

— Reply to this email directly, view it on GitHub https://github.com/snomiao/calibre-utf8-path/issues/2#issuecomment-1486726180, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABX33FTJQ7DEAZR46BWWHIDW6LGKNANCNFSM4TAVM3KA . You are receiving this because you modified the open/close state.Message ID: @.***>

snomiao avatar Mar 28 '23 12:03 snomiao