pyxel
pyxel copied to clipboard
kitao
Any tips on compiling Pyxel's Rust codebase into web assembly?
I don't have much experience with compiling Rust projects but I have learnt about Pyodide it allows running Python code in your browser & powers sites like Iodide.
Pyodide is Python compiled into Web Assembly (a fast, low-level language that runs in the browser). With Pyodide a user can edit & run Python (with pip installed modules) in their browser instantly.
I tried this with Pyxel but Emscripten (the platform for web assembly) isn't supported.
By compiling Pyxel's Rust library info web assembly, Pyxel games would be runnable & editable all in the browser!
I think that could create an awesome site for people to make & share games, (like PICO-8 but with the much more powerful Python language & live editing the game's code in the browser).
My question is... does anyone have some tips for adding Emscripten support?
I've done a bit of work on converting pyxel to a Pyodide package (tutorial) but I'm still learning the basics of Rust builds.
Compiling pyxel to webassembly is a very tricky task. The pyxel core uses SDL and is written in Rust. We also need to write python code to make a game.
There are some prior works that:
- Compile Rust SDL binding to wasm target Relative issue Example project (I recommend you to check projects on credits too.)
- Compile PyO3 to wasm target. PyO3 can be compiled to wasm pyodide PR However, there are issues on wasm-unknown-emscripten target: rust-lang/rust#93015
A while ago, I tried to compile pygame wasm target. Pygame uses SDL directly, so I thought that it compiles easily. But it wasn't. I faced confusing errors. The emscripten does not allow submodules to link to the system libraries. I think compiling pyxel will be tough too. However, this is a valuable issue. I hope you succeed.
I tried several attempts to compile pyxel to the wasm32 target. However, there are many obstacles.
First, we should use the wasm32-unknown-emscripten target. wasm32-unknown-unknown lacks many std libs and OS functionalities, so we can not compile SDL2. In addition, wasm32-unknown-wasi does not support i/o properly.
Second, Rust's ecosystem has poor support to the wasm32-unknown-emscripten target. cargo-web is not maintained. Also, there are several compile issues on the emscripten target.
I tried to compile PyO3 first, but it does not compile even with the pyodide. So I tried to compile the pyxel engine instead, but I faced a weird error:
wasm-ld: error: /home/km19809/hobby/pyxel/lib/target/wasm32-unknown-emscripten/debug/deps/libinstant-6163d6d51301788e.rlib(instant-6163d6d51301788e.instant.d94e7e5e-cgu.3.rcgu.o): undefined symbol: _emscripten_get_now
emcc: error: '/home/km19809/emsdk/upstream/bin/wasm-ld @/tmp/emscripten_tltpl9nc.rsp' failed (returned 1)
I think the instant means std::time::instant but I cannot find any helpful documents. Even weirder is the symbol, emscripten_get_now, is one of the emscripten's default API.
So, I think, I faced a dead end.
I cannot rewrite all the SDL2 dependencies, nor find a way to compile wasm32-unknown-emscripten target.
But if somebody finds ways to compile correctly, I'll be glad.
After several months, I managed to compile the only engine of pyxel.
the Python binding part is not compiled yet.
This is the compiled version of test_pyxel.rs
It works. However, there are several limitations now.
- The
chronowill not compile until0.4.20is released. So, I have to patch this. - There are a bunch of configures that need to be fixed.
Cargo.toml,Config.toml, ASYNCIFY loops... - The emscripten uses deprecated web audio API.
emscripten-core/emscripten#16470.
It can't play any sound. I tried to fix it, but my tweaks made terrible sounds. - Rust's emscripten support is quite outdated, so it needs some additional objects.
- I tried to build PyO3 for the web. I finally found a working example, but its development setup is quite difficult.
pyodide/pyodide#2378
Anyway, it was not a "dead-end" as I thought. It is super challenging, though.
It's really exciting to just see this in a browser!
Shame about these setup issues. Hopefully as all the backend packages update this will get easier.
@km19809 I've started checking pyodide and its example using pyo3. I'd like to support web in the next version. So any update would be appreciated.
I've successfully compiled pyxel pyo3 wrapper to wasm32 wheel using a development version of maturin, some tweaks needed
- Add a
pyproject.tomlfile with proper configuration - Add
sdl2emscripten target dependency inlib/engine/Cargo.toml - Disable
sysinfocrate and it's usage in pyo3 wrapper
$ RUSTUP_TOOLCHAIN=nightly maturin build -o dist --target wasm32-unknown-emscripten --release -- -Clink-arg=-sUSE_SDL=2
πΉ Building a mixed python/rust project
π Found pyo3 bindings with abi3 support for Python β₯ 3.7
π Not using a specific python interpreter
π‘ Using build options manifest-path from pyproject.toml
Compiling autocfg v1.1.0
Compiling libc v0.2.127
Compiling cfg-if v1.0.0
Compiling scopeguard v1.1.0
Compiling crossbeam-utils v0.8.11
Compiling once_cell v1.13.0
Compiling proc-macro2 v1.0.43
Compiling quote v1.0.21
Compiling rayon-core v1.9.3
Compiling unicode-ident v1.0.3
Compiling adler v1.0.2
Compiling syn v1.0.99
Compiling crc32fast v1.3.2
Compiling adler32 v1.2.0
Compiling target-lexicon v0.12.4
Compiling either v1.7.0
Compiling getrandom v0.1.16
Compiling weezl v0.1.7
Compiling bitflags v1.3.2
Compiling byteorder v1.4.3
Compiling futures-core v0.3.21
Compiling version-compare v0.1.0
Compiling smallvec v1.9.0
Compiling cc v1.0.73
Compiling parking_lot_core v0.9.3
Compiling color_quant v1.1.0
Compiling futures-sink v0.3.21
Compiling bytemuck v1.11.0
Compiling scoped_threadpool v0.1.9
Compiling ppv-lite86 v0.2.16
Compiling half v1.8.2
Compiling sdl2 v0.35.2
Compiling bit_field v0.10.1
Compiling lebe v0.5.1
Compiling hashbrown v0.12.3
Compiling lazy_static v1.4.0
Compiling indoc v1.0.7
Compiling array-macro v2.1.5
Compiling unindent v0.1.10
Compiling miniz_oxide v0.5.3
Compiling deflate v1.0.0
Compiling miniz_oxide v0.3.7
Compiling inflate v0.4.5
Compiling memoffset v0.6.5
Compiling crossbeam-epoch v0.9.10
Compiling num-traits v0.2.15
Compiling rayon v1.5.3
Compiling num-integer v0.1.45
Compiling lock_api v0.4.7
Compiling miniz_oxide v0.4.4
Compiling num-rational v0.3.2
Compiling num-iter v0.1.43
Compiling num-rational v0.4.1
Compiling indexmap v1.9.1
Compiling deflate v0.8.6
Compiling sdl2-sys v0.35.2
Compiling gif v0.11.4
Compiling blip_buf-sys v0.1.4
Compiling crossbeam-channel v0.5.6
Compiling num_cpus v1.13.1
Compiling getrandom v0.2.7
Compiling dirs-sys-next v0.1.2
Compiling time v0.1.44
Compiling flate2 v1.0.24
Compiling png v0.16.8
Compiling png v0.17.5
Compiling threadpool v1.8.1
Compiling rand_core v0.6.3
Compiling nanorand v0.7.0
Compiling pyo3-build-config v0.16.5
Compiling rand_core v0.5.1
Compiling dirs-next v1.0.2
Compiling spin v0.9.4
Compiling parking_lot v0.12.1
Compiling zip v0.6.2
Compiling crossbeam-deque v0.8.2
Compiling rand_chacha v0.3.1
Compiling rand_xoshiro v0.6.0
Compiling rand_hc v0.2.0
Compiling rand_xorshift v0.2.0
Compiling platform-dirs v0.3.0
Compiling blip_buf v0.1.4
Compiling chrono v0.4.20
Compiling rand v0.8.5
Compiling rand v0.7.3
Compiling pyo3-macros-backend v0.16.5
Compiling pyo3-ffi v0.16.5
Compiling pyo3 v0.16.5
Compiling pin-project-internal v1.0.11
Compiling pin-project v1.0.11
Compiling pyo3-macros v0.16.5
Compiling flume v0.10.14
Compiling jpeg-decoder v0.1.22
Compiling jpeg-decoder v0.2.6
Compiling exr v1.4.2
Compiling tiff v0.6.1
Compiling tiff v0.7.3
Compiling image v0.23.14
Compiling image v0.24.3
Compiling noise v0.7.0
Compiling pyxel-engine v1.7.2 (/Users/messense/Projects/pyxel/lib/engine)
Compiling pyxel-wrapper v1.7.2 (/Users/messense/Projects/pyxel/lib/wrapper)
Finished release [optimized] target(s) in 32.56s
π¦ Built wheel for abi3 Python β₯ 3.7 to dist/pyxel-1.7.2-cp37-abi3-emscripten_3_1_17_wasm32.whl
@messense Thank you for your efforts and the really interesting information! Actually I don't expect maturin can handle the structure of the current Pyxel project. Is it possible to share the detail of your modifications includes pyproject.toml? If maturin is promising for supporting many platforms, I'd like consider to use it for Pyxel on not only WASM but also other platforms.
@kitao Here you go: https://github.com/messense/pyxel/tree/maturin
maturin main branch code needed to build it.
Edit: Now you can use maturin v0.13.2, can be installed via pip install "maturin>=0.13.2".
@messense Thank you very much for your quick reply! Does this maturin setting can build Pyxel package for other platforms?
Does this maturin setting can build Pyxel package for other platforms?
Yes, I think so, needs to test it out.
It's awesome! Did you tested compiled code?
I haven't got the time to tested it, will try it later.
Edit: quick update, it's importable on macOS, need to test in pyodide.
In [1]: import pyxel
In [2]: pyxel
Out[2]: <module 'pyxel' from '/Users/messense/Projects/pyxel/pyxel/__init__.py'>
In [3]: pyxel.pyxel_wrapper
Out[3]: <module 'pyxel.pyxel_wrapper' from '/Users/messense/Projects/pyxel/pyxel/pyxel_wrapper.abi3.so'>
I created "maturin" branch for experimentation.
I'll continue to modify this branch for supporting multiple platforms includes emscripten with maturin.
I'm testing this branch with maturin develop -m lib/wrapper/Cargo.toml command on virtualenv for now.
Changed the folder structure slightly.
The current build command is maturin develop -m src/pyxel-wrapper/Cargo.toml for your reference.
I'll continue trial and error.
Changed the folder structure slightly. The current build command is
maturin develop -m src/pyxel-wrapper/Cargo.tomlfor your reference. I'll continue trial and error.
maturin develop should also work, no need to supply the extra -m src/pyxel-wrapper/Cargo.toml since it will be read from pyproject.toml.
Hi @messense ,
maturin develop should also work, no need to supply the extra -m src/pyxel-wrapper/Cargo.toml since it will be read from pyproject.toml.
I tried but it doesn't work. The reason is I'm not using the latest main branch version? I'm wondering why manifest-path doesn't work. The manifest-path is used for two times and it seems that the second timing doesn't work well.
@messense Probably I imported your modification for Maturin and Emscripten in maturin branch. (Emscripten SDL2 configuration is in .cargo/config) Could you check it works for Emscripten on your develop environment? Because I couldn't build Emscripten version on my M2 Mac.
@kitao I can build it. pyxel-1.8.0-cp37-abi3-emscripten_3_1_14_wasm32.whl.zip
You can install Emscripten on macOS with brew: brew install emscripten
@messense Thank you for the information. I updated to use the latest Maturin and its new parameter.
Regarding Emscripten build, did you compile it with maturin build --release --target wasm32-unknown-emscripten?
Yes, it requires Rust nightly so also I set RUSTUP_TOOLCHAIN=nightly env var.
Pyodide requires emscripten 3.1.14 (newer version may work but it's not tested), but I don't have that version so I also set MATURIN_EMSCRIPTEN_VERSION=3.1.14 to ask it to output the right platform tag for pyodidie.
Thank you! I forgot to add the wasm32-unknowm-emscripten target for nightly rust.
Now the maturin brunch can build the emscripten-version wheel with make WASM=1 command!.
The next step is to link to pyodide (or other emscripten-version of Python) and render Pyxel's output on web canvas.
The next step is to link to pyodide (or other emscripten-version of Python) and render Pyxel's output on web canvas.
I haven't looked into this part yet, but there is a canvas example for matplotlib in pyodide: https://github.com/pyodide/pyodide/blob/b497ce26ed94774a8d695a3e61016a7a26e45e3b/packages/matplotlib/src/html5_canvas_backend.py, not sure if it's helpful though since pyxel uses SDL2.
Thank you! The current status is like this: https://twitter.com/kitao/status/1557923653773578240?s=21&t=TptebjKXMLBkLg0I42hPeg
2022εΉ΄8ζ12ζ₯(ι) 14:19 messense @.***>:
The next step is to link to pyodide (or other emscripten-version of Python) and render Pyxel's output on web canvas.
I haven't looked into this part yet, but there is a canvas example for matplotlib in pyodide: https://github.com/pyodide/pyodide/blob/b497ce26ed94774a8d695a3e61016a7a26e45e3b/packages/matplotlib/src/html5_canvas_backend.py, not sure if it's helpful though since pyxel uses SDL2.
β Reply to this email directly, view it on GitHub https://github.com/kitao/pyxel/issues/354#issuecomment-1212738982, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFFXEVXMRUJIK3NM32GFGDVYXNFPANCNFSM5LCX2AOA . You are receiving this because you were mentioned.Message ID: @.***>
BTW, here is an emscripten wheel building example on GitHub Actions: https://github.com/pydantic/pydantic-core/blob/0e2446d70ff68bf05c809b209bee923d6795012d/.github/workflows/ci.yml#L176-L228
I tried the latest code from maturin branch locally and encountered TypeError: getWasmTableEntry(...) is not a function, reported to pyodide: https://github.com/pyodide/pyodide/issues/2964
@messense Thank you for your cooperation. Could you try the latest maturin branch? Actually I am struggling with other errors. The strange thing is that the code that should have worked yesterday no longer works...
You can build the package and start web server by: `make clean-wasm build-wasm' './scripts/start_server'
On macOS with Emscripten 3.1.17 I get the following error which I think is expected since you need to use 3.1.14 for pyodide 0.21.0.
RuntimeError: null function or function signature mismatch
On Ubuntu with Emscripten 3.1.14 I get the following error which I already reported to upstream.
TypeError: getWasmTableEntry(...) is not a function
Great! Yes, I changed to use the Homebrew-version of Emscripten and specify the version with its version option yesterday. Probably I misunderstood that it worked because of the last built cache. Now I restored my Emscripten environment and it worked (it means I got the same error as yours).
As for the error you mentioned, it was the error I tackled on yesterday.
It happens at let sdl_event_pump = sdl_context.event_pump().unwrap(); in pyxel-engine/platform_sdl2.rs.
And I'm checking for an oversight somewhere in the compile options now.
I have some analysis in https://github.com/pyodide/pyodide/issues/2964#issuecomment-1213693026, I think you might needed to adjust the code for Emscripten support.
Building a pyxel wheel with maturin is incredible.
I'm just curious how maturin detects appropriate environment variables.
I thought that PyO3 needs to be configured when cross-compiling.
Does it cross-compile to webassembly, with system python or does it somehow download and compiles pyodide-compatible python?
@km19809 On Unix systems you don't really need the target Python just for cross compiling, all you need is to set the right filename for .so files and set the right platform tag for .whl files. These information came from Python sysconfig, we collect it on CI and bundle them in maturin, see https://github.com/PyO3/maturin/tree/main/src/python_interpreter
And we pass these information to PyO3 via PYO3_CONFIG_FILE.
If the current missing symbol issue resolves, then we may change the loops.
Pyxel uses loops in engine/system.rs. run, show and wait.
However, these loops may make the app hang.
There are possible solutions, but they are not easy.
- emscripten_set_main_loop: This example uses it. We need to write a function for it.
- ASYNCIFY: RuSDLem and ruggrogue use it. However, it may need additional configurations with dynamic linking.
When I compiled the engine only, I used ASYNCIFY and built a rust code as the main module. But as a library, a pyxel should be a side module.
OK. I'l modify the Pyxel's main loop so that it can be called from outside in web environments.
Current status: https://github.com/pyodide/pyodide/issues/2964 was resolved. (Thank you, @messense and hoodname!)
To run the wheel properly, we must use pyodide with some patches.
In Makefile.envs,
we add -s USE_SDL=2 and -s GL_WORKAROUND_SAFARI_GETCONTEXT_BUG=0 :
export MAIN_MODULE_LDFLAGS= $(LDFLAGS_BASE) \
-s MAIN_MODULE=1 \
-s EXPORT_NAME="'_createPyodideModule'" \
-s EXCEPTION_CATCHING_ALLOWED=['we only want to allow exception handling in side modules'] \
-s DEMANGLE_SUPPORT=1 \
-s USE_FREETYPE=1 \
-s USE_LIBPNG=1 \
-s USE_SDL=2 \
-s GL_WORKAROUND_SAFARI_GETCONTEXT_BUG=0 \
-s FORCE_FILESYSTEM=1 \
-s TOTAL_MEMORY=20971520 \
-s ALLOW_MEMORY_GROWTH=1 \
-s EXPORT_ALL=1 \
-s POLYFILL \
However, it cannot detect the canvas element.
It should detect the canvas and assign it as Module.canvas.
I could not figure out why, but it does not set Module.canvas at all.
So I monkey-patched pyodide.js:
// inside the function loadPyodide
Module.canvas = document.getElementById("canvas") || document.createElement("canvas");
Then I faced a familiar error:
The emscripten uses deprecated web audio API. https://github.com/emscripten-core/emscripten/issues/16470. It can't play any sound. I tried to fix it, but my tweaks made terrible sounds.
To take a further step, I must handle the buffer size problem. It may be related to SDL emscripten implementation.
Also, I did not prepare the patched version of pyodide.
I'll share the code if pyodide detects canvas properly.
For now, I have to patch it every time I rebuild pyodide.
(I'll be glad if someone fixes them!)
To test the current state:
- Download pyodide.js
- Store them in the same directory.
- Use
pyodide.jsas a source, instead of CDN-delivered pyodide 2.1.0.
Thank you, @km19809
This is not directly related to your new information. I've updated the Maturin branch for Emscripten.
Specifically I changed the main loop of Pyxel not to keep the loop inside on Emscripten environment.
Emscripten is supposed to call the pyxel.run_one_frame method periodically.
But I can't compile it due to Rust's strict lifetime control, so commented out it temporary and insert a message instead for now.
For now, a few examples work. The problems that I mentioned were not solved yet, but I built some examples. These examples are:
- Using
filp()only. Because of browser limitations, I cannot use loops likerun(). It will be resolved whenrun_one_frame()works. - Not using audio. I hope SDL 2.24.0 will solve this issue.
- Some pyxel functions are broken. For example,
load()can read from virtual FS, but it panics while unzipping. I don't know why. - Sources are modified to replace loops. It uses the browser's event loop now.
from pyodide.webloop import WebLoop # Browser event loop
import pyxel
pyxel.init(120, 80)
def frame(loop): # the frame function must receive the loop object for recursive call.
if pyxel.btnp(pyxel.KEY_Q):
return # if the function does not call recursively, it halts
pyxel.cls(3)
pyxel.rectb(pyxel.frame_count % 160 - 40, 20, 40, 40, 7)
pyxel.flip()
loop.call_soon(frame, loop) # the function calls recursively to run at next frame.
loop = WebLoop() # initialize loop
loop.call_soon(frame, loop) # start loop
Examples:
Broken:
-
offscreen : Unable to unzip
.pyxres. It panics.
Finally I managed to call emscripten_set_main_loop via pyxel.run method. And I tried the special version of pyodide.js by @km19809 but error occurs while loading it. Is it enough to copy your pyodide.js and load it in index.html?
Thank you, @messense ! But when I try it with the latest index.html, some errors occurs. Am I missing something?
I copied the whole directory of pyodide.
Works for me with a canvas element in index.html
diff --git a/server/index.html b/server/index.html
index 244d881d..ab76d8d6 100644
--- a/server/index.html
+++ b/server/index.html
@@ -2,10 +2,13 @@
<html>
<head>
- <script src="https://cdn.jsdelivr.net/pyodide/v0.21.0/full/pyodide.js"></script>
+ <script src="pyodide/pyodide.js"></script>
</head>
<body>
+ <!-- Create the canvas that the Rust code will draw into -->
+ <canvas id="canvas" oncontextmenu="event.preventDefault()"></canvas>
+
<script type="text/javascript">
async function main() {
let pyodide = await loadPyodide();
Thanks. I added a canvas and disabled the audio according to your sample. I commit the code in which the audio module is disabled when on Emscripten.
@km19809 Example links are broken.
Not using audio
How to disable audio?
I updated the links. I used the relative paths by mistake.
To prevent crashing audio, I modified pyodide.asm.js.
//from
SDL2.audio.scriptProcessorNode = SDL2.audioContext['createScriptProcessor']($1, 0, $0);
//to
SDL2.audio.scriptProcessorNode = SDL2.audioContext['createScriptProcessor'](0, 0, $0);
But I assume it will be solved when SDL2 releases the newer version.
And I tried the special version of pyodide.js by @km19809 but error occurs while loading it. Is it enough to copy your pyodide.js and load it in index.html?
About crashing, as @messense mentioned, it needs a canvas element now. Actually, as I know, it should create a canvas element when it cannot find one. However, it is somehow malfunctioning, so I used the method of this article. I'm going to ask emscripten about this.
With the pre-released version of SDL2, the sound issue is resolved.
Also, the file system issue was my mistake, I corrected them.
Check this out.
However, there are mismatches between the mouse cursor and its actual position.
These mismatches make some examples impossible to play. (I faced a similar issue when creating a wasm game; It required some JS tweaks π’)
Also, examples still use the web loop, instead of run().
I'm working on a fork of pyodide, to build a pyodide with SDL2 support. This fork resolves the canvas detecting issue, but still uses an older version of SDL2. I'll share if I succeed in automatically patching SDL2.
And I tried the special version of pyodide.js by @km19809 but error occurs while loading it.
The package @km19809 provided before misses some required files, you need to pack the whole dist/ directory after building pyodide.
For the time being, if we cannot get Emscripten main loop to work properly, we could just use pyodide's WebLoop.
https://github.com/kitao/pyxel/blob/05e0e23870b6406e2d8c5d15accd1460d761f4d8/python/pyxel/init.py
Change it to
import sys
from .pyxel_wrapper import * # type: ignore # noqa F403
if "pyodide" in sys.modules:
# running in Pyodide
from pyodide.webloop import WebLoop
def run(update, draw):
def frame(loop):
update()
draw()
pyxel.flip()
loop.call_soon(frame, loop)
loop = WebLoop()
loop.call_soon(frame, loop)
to override the run function when running in Pyodide.
See https://pyodide.org/en/stable/usage/faq.html#how-to-detect-that-code-is-run-with-pyodide
@messense @km19809 I completely redesigned the base of Pyxel in maturin branch and managed to run Emscripten loop. At least, it works on my environtment. There is a lot of provisional code, so it would be helpful if you could confirm the operation.
Remaining tasks are file system, sound playing, and input peripherals includes mouse and keyboard? I'm waiting for everyone's further information.
@kitao It works, so I think you can remove this line in audio.rs
Here is an example of an editor.
Without any modification, it works.
However, due to the cursor problem, it is almost impossible to use with a mouse.
Also, as you can see, some fatal error occurs, which makes it impossible to save work.
The error does not crash the app but blocks any further access via python.
It does not occur when I use flip() based approach.
I assume that some python exception is not properly handled when run? I'm not sure.
@km19809 Ah, sorry. I missed excellent work. Thank you for your great effort!
My goal is that Pyxel without any patches can build a wasm-wheel which can be installed and run on pyodide from CDN. Let me confirm what kind of things I should do and wait.
Things I should do are:
- Fix the mouse handling code for web browsers
- Restore commented-out sound feature for web browsers
- Perhaps adding code to Pyxel's server/index.html to specify the proper canvas size? (No need to do anything related to file handling?)
Things I should wait for are:
- Release of new version of SDL2
- Release of bug-fixed version of Pyodide And the packaged you built includes the above changes in advance. (Actually I don't understand the relation between Pyodide and Emscripten version of SDL2)
Thank you.
Actually I don't understand the relation between Pyodide and Emscripten version of SDL2
It's related to how Emscripten implementing dynamic linking, see https://github.com/pyodide/pyodide/issues/2964#issuecomment-1214651891, and some parts of SDL2 is implemented in JavaScript in its Emscripten port that prevents us from static link SDL2 in pyxel side module. (And last time I tried rust-sdl2 doesn't build properly on wasm32-unknown-emscripten when static link SDL2 either)
- Perhaps adding code to Pyxel's server/index.html to specify the proper canvas size? (No need to do anything related to file handling?)
Regarding file handling, emscripten uses an in-memory virtual file system.
It's virtual, so we need to do extra work on uploading and downloading a file.
My examples use fetch API to load assets and scripts, but I think we eventually need some local file access.
Drag-n-drop File accessing But don't need to hurry.
In addition, the fatal error (which crashes a pyodide session) must be solved. I wanted to make a simple web editor, but its session crashed as soon as initialized. I have no idea now but should examine it to edit files online.
Finally, for pyodide, I wonder if the authors accept SDL2 support. Without modification, it cannot run an SDL2-based application. However, it makes the binary size larger and useless to the non-SDL user. I will ask them about it soon.
@km19809 @messense I'd like to have the latest patched pyodide to check audio and mouse features. Can you tell me how to get it?
@kitao @km19809 uploaded them to here https://github.com/km19809/km19809.github.io/tree/main/pyxelodide
@messense Thank you, but which files are pure pyodide? It seems that Pyxel's files, experimental code, and pyodide-related files are mixed.
@kitao
If you want to build by yourself, here is a pure pyodide port.
https://github.com/km19809/pyodide/tree/SDL2
The pyodide authors recommend you to build with docker.
To use docker, install docker and run ./run_docker then make.
Thank you for the information. I replaced @messense 's pyodide.zip placed in pyxel/server/pyodide with @km19809 's pyxelodide folder for now, but it doesn't work regardless of audio module is enabled or not. Pyxel is also modified for that?
@km19809
Though I think it includes useful information, the output from browser is like this.
What I did was put the pyodide folder (renamed from pyxelodide) under pyxel/server.
I also tried your some html which don't use asset instead of pyxel/server/index.html, but the result was the same.
pyodide_server.zip Will you try that above? In my case, it works. I assume that this issue is related to panic behavior.
Your error shows abort, but mine shows unwind.
I am using f4f51d6c80a63e25bcfa53176c73e4623f46c416 and my emscripten, pyodide versions are same as https://github.com/km19809/pyodide/tree/SDL2
@km19809 I tried on my Mac like this:
unzip pyodide_server.zip
cd server
python3 -m http.server 8000
and opened localhost:8000. But the result was the same as my previous post. I'm wondering what is the difference and why the previous @messense 's zip worked,
@km19809 I tried @messense 's version from zip file again and it seems that the same errors occurred. Security-related issue on Mac? I'l continue to check.
@kitao If you use chrome, there is a disable cache option in Dev Tools > network tab.
https://www.technipages.com/google-chrome-how-to-completely-disable-cache
I recommend you to check if browser uses cached version of resources (wheel, html, etc.)
With disable cache, browser loads the new resource everytime.
You are correct! After disabling cache, it works. Thank you very much. I commited the audio-enabled code for Emscripten.
@km19809 @messense
Do you think it is possible to fetch file and call pyodide.FS.writeFile automatically when Pyxel's load function is called?
I would be really useful if possible.
@kitao I think there are 3 possible ways.
- In Rust
- In Python
- In Javascript.
In Rust, you can use Emscripten's fetch API. But it would be a terrible experience. You already suffered it. An outdated example.
In Python, you can use pyodide.http.pyfetch before calling a load function. However, it modifies the python wrapper only. Not applicable to bare rust.
Finally, in Javascript, open is a JS function, so theoretically we can modify it to fetch files. However, JS cannot receive the return value of the async function (such as fetch) in the synchronous function (such as pyodide.FS.open).
So, practically, option 2 is the most promising way.
@km19809 Thank for your clear advice. I agree that wrapping file-related APIs in python is the most reasonable way. But I also would like to try Rust-version once though I don't know how hard it is...
BTW, I finally merged the maturin branch into the develop branch. I'll continue to develop for web browsers on it instead of the maturin branch.
@kitao
My goal is that Pyxel without any patches can build a wasm-wheel which can be installed and run on pyodide from CDN. Let me confirm what kind of things I should do and wait.
That may be not the best way for performance and simplicity. Dynamic linking adds a lot of overhead and makes it difficult to access browser functions in a straightforward way.
It is much simpler to embed libpython statically and link to the wasm module directly ( like pygame-web / Panda3D webgl port or Harfang3D do).
Using vanilla CPython via embedding is more durable since 3.11 has partial official support for emscripten/wasi and allows for building/using simpler toolchains like pygame CI already does ( see https://github.com/pygame/pygame/blob/main/.github/workflows/build-emsdk.yml ).
@pmp-p Thank you very much for the very good information. I have checked this page, and now I'm curious for how much overhead occurs by dynamic linking. And it seems that some binary packages such as NumPy are embedded in Pyodide project. I also interested in those are linked as static library.
Everyone, thank you for your cooperation. Now Pyxel works on web browsers though it has several limitations yet. So let me close this issue once and make a new issue to improve web support because this thread is too long to discuss new topics.
You could enable discussions on this repository, which could be more convenient than a ticket (replies can be threaded).
