kiwix-js icon indicating copy to clipboard operation
kiwix-js copied to clipboard

Compile the libzim js bindings in a separate module

Open mossroy opened this issue 4 years ago • 2 comments

Like it is done for python : https://github.com/openzim/python-libzim/ It should be run by kiwix-build, too : see https://github.com/kiwix/kiwix-build/issues/503

It might allow this wasm version of libzim to be used in other javascript projects, like mwoffliner

mossroy avatar Oct 29 '21 16:10 mossroy

The bindings I used for the prototype are in https://github.com/mossroy/libzim_wasm/blob/master/demo_file_api.cpp (done with the help of @mgautierfr ) But we only did a few bindings for what was necessary for the prototype. The goal would be to expose all the libzim APIs with these bindings (or the same ones exposed by the Python bindings)

Note that kiwx-js only needs the read APIs. But the write APIs might be of some interest for some other projects, like the node.js scrapers

mossroy avatar Jun 06 '22 14:06 mossroy

Note that I suspect there is a memory leak in the current version of these bindings: #872

mossroy avatar Jun 06 '22 15:06 mossroy

@mgautierfr is working on the wasm compilation of the libzim with Emscripten. This might be ready next week in libzim nightly and release (from libzim 8.1.0). But this won't make the libzim directly usable from kiwix JS, we will need the wrapper. So far I understand that https://github.com/mossroy/libzim_wasm is the best we have current regarding this. Do you confirm? If "yes", I would recommend to migrate it to openzim/javascript-libzim. Would that make sense to you?

kelson42 avatar Oct 26 '22 15:10 kelson42

Yes, it's the git repo where we've compiled libzim with emscripten. I can transfer it under the openzim umbrella, or wherever you want. However, there is not a lot of code in it either. Feel free to copy it for kiwix. Maybe it would be worth having a visio meeting to discuss this and its usage in kiwix-js. So that you have all the info

mossroy avatar Oct 27 '22 15:10 mossroy

If this is ok to both of you, yes would probably be beneficial to move it to openzim so further dev can take place

kelson42 avatar Oct 27 '22 15:10 kelson42

I currently don't have the right to create a git repo under openzim, so can't move it.

It would be needed to publish on gh-pages the same pages as currently: at least https://mossroy.github.io/libzim_wasm/ that is useful for testing. And maybe https://mossroy.github.io/libzim_wasm/index-bigfile.html (which is now less useful as the issue has been fixed in emscripten) It should be possible to enable github issues on it, even if it's a fork from a very old git repo of dattaz.

mossroy avatar Oct 27 '22 19:10 mossroy

@mossroy I have sent you an invitation to be owner, shoukd be possible then.

kelson42 avatar Oct 30 '22 14:10 kelson42

Done: repo transferred to https://github.com/openzim/javascript-libzim I've enabled issues on it. gh-pages are enabled, so the demo pages are available again at different URLs: https://openzim.github.io/javascript-libzim/ and https://openzim.github.io/javascript-libzim/index-bigfile.html You should have the necessary rights on the repo. If not, feel free to ask me

mossroy avatar Nov 01 '22 09:11 mossroy

@mossroy Thank you, I will check.

kelson42 avatar Nov 01 '22 13:11 kelson42

Looks good to me. I guess this ticket should be closed now as everything will be handled in https://openzim.github.io/javascript-libzim/

kelson42 avatar Nov 07 '22 13:11 kelson42

I've transferred what remains to be done in https://github.com/openzim/javascript-libzim/issues/11

mossroy avatar Nov 19 '22 11:11 mossroy