kiwix-js
kiwix-js copied to clipboard
Compile the libzim js bindings in a separate module
Like it is done for python : https://github.com/openzim/python-libzim/ It should be run by kiwix-build, too : see https://github.com/kiwix/kiwix-build/issues/503
It might allow this wasm version of libzim to be used in other javascript projects, like mwoffliner
The bindings I used for the prototype are in https://github.com/mossroy/libzim_wasm/blob/master/demo_file_api.cpp (done with the help of @mgautierfr ) But we only did a few bindings for what was necessary for the prototype. The goal would be to expose all the libzim APIs with these bindings (or the same ones exposed by the Python bindings)
Note that kiwx-js only needs the read APIs. But the write APIs might be of some interest for some other projects, like the node.js scrapers
Note that I suspect there is a memory leak in the current version of these bindings: #872
@mgautierfr is working on the wasm compilation of the libzim with Emscripten. This might be ready next week in libzim nightly and release (from libzim 8.1.0). But this won't make the libzim directly usable from kiwix JS, we will need the wrapper. So far I understand that https://github.com/mossroy/libzim_wasm is the best we have current regarding this. Do you confirm? If "yes", I would recommend to migrate it to openzim/javascript-libzim. Would that make sense to you?
Yes, it's the git repo where we've compiled libzim with emscripten. I can transfer it under the openzim umbrella, or wherever you want. However, there is not a lot of code in it either. Feel free to copy it for kiwix. Maybe it would be worth having a visio meeting to discuss this and its usage in kiwix-js. So that you have all the info
If this is ok to both of you, yes would probably be beneficial to move it to openzim so further dev can take place
I currently don't have the right to create a git repo under openzim, so can't move it.
It would be needed to publish on gh-pages the same pages as currently: at least https://mossroy.github.io/libzim_wasm/ that is useful for testing. And maybe https://mossroy.github.io/libzim_wasm/index-bigfile.html (which is now less useful as the issue has been fixed in emscripten) It should be possible to enable github issues on it, even if it's a fork from a very old git repo of dattaz.
@mossroy I have sent you an invitation to be owner, shoukd be possible then.
Done: repo transferred to https://github.com/openzim/javascript-libzim I've enabled issues on it. gh-pages are enabled, so the demo pages are available again at different URLs: https://openzim.github.io/javascript-libzim/ and https://openzim.github.io/javascript-libzim/index-bigfile.html You should have the necessary rights on the repo. If not, feel free to ask me
@mossroy Thank you, I will check.
Looks good to me. I guess this ticket should be closed now as everything will be handled in https://openzim.github.io/javascript-libzim/
I've transferred what remains to be done in https://github.com/openzim/javascript-libzim/issues/11