python-libzim icon indicating copy to clipboard operation
python-libzim copied to clipboard

Move to libzim 9.4.0

Open benoit74 opened this issue 2 months ago • 6 comments

This is a minimal PR to move to libzim 9.4.0

It does not add new stuff like the new illustrations API (I propose to do this in a distinct PR https://github.com/openzim/python-libzim/pull/233)

It adapts to libzim changes:

  • remove Archive.dirent_lookup_cache_max_size, does not exists anymore in libzim
  • move Archive.cluster_cache_max_size and Archive.cluster_cache_current_size to methods outside of Archive object: get_cluster_cache_max_size, set_cluster_cache_max_size, get_cluster_cache_current_size (size is now in bytes)
  • deprecate usage of get_illustration_sizes()

benoit74 avatar Nov 04 '25 08:11 benoit74

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 93.77%. Comparing base (f2b3a59) to head (75c4e87).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #234      +/-   ##
==========================================
+ Coverage   93.69%   93.77%   +0.08%     
==========================================
  Files           1        1              
  Lines         539      546       +7     
==========================================
+ Hits          505      512       +7     
  Misses         34       34              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Nov 04 '25 08:11 codecov[bot]

@rgaudin do you have any idea about how to fix the wheel build error on Ubuntu cp39 aarch64?

Is this something which is induced by libzim 9.4.0 or did I broke something?

auditwheel: error: cannot repair "/tmp/cibuildwheel/built_wheel/libzim-3.7.1.dev0-cp39-cp39-linux_aarch64.whl" to "manylinux_2_28_aarch64" ABI because of the presence of too-recent versioned symbols.

I tried update CI actions (kinda desperate move tbh) but it did not helped except having much nicer logs for the wheel building action.

benoit74 avatar Nov 04 '25 09:11 benoit74

I use this link and this one to check which distro version has which glibc version easily. It's not up to date unfortunately but Ubuntu jammy 22.04 has glibc 2.35.

Regular libizim is now built on jammy so if we were to use it, we could only link on images that supports 2.35+. We would thus need to update our manylinux-*-image instructions to use manylinux_2_39 which is very recent. That would imply:

  • Only supporting Ubuntu 24.04+ (dropping 18.10, 20.04, 20.10, 22.04, 22.10)
  • Only supporting Debian 13+, released a month ago (dropping 10, 11, 12)

That's why we don't use regular libzim in pylibzim but a special build named manylinux. It's built on manylinux_2_28 for the sole purpose of pylibzim.

I see here though that this is an x86_64 image so obviously there will be issues on aarch64. You can open a ticket in kiwix-build because this is a problem

rgaudin avatar Nov 04 '25 11:11 rgaudin

Thank you! In addition, manylinux_2_39 is only supported on aarch64 and riscv64 according to https://github.com/pypa/manylinux

image

benoit74 avatar Nov 04 '25 12:11 benoit74

Blocked by https://github.com/kiwix/kiwix-build/issues/881 for the time being

benoit74 avatar Nov 04 '25 12:11 benoit74

This is now ready for review, only waiting for https://github.com/kiwix/kiwix-build/pull/884 to be merged and released

benoit74 avatar Nov 06 '25 14:11 benoit74

It's available and should be working

❯ auditwheel-symbols -m 2_28 libzim_linux-aarch64-manylinux-9.4.0-1/lib64/libzim.so.9.4.0
libzim.so.9.4.0 is manylinux_2_28 compliant.

rgaudin avatar Nov 07 '25 16:11 rgaudin

@rgaudin can you please review this PR?

benoit74 avatar Nov 09 '25 19:11 benoit74