XPython startup time suffers from frequent calls to wasm grow

[martenrichter]

I have analyzed the xeus python startup times a bit more and the majority of the time is consumed by wasm grow.

This issue to crosslink to:

https://github.com/emscripten-core/emscripten/issues/17891

[martenrichter]

Ok, I made a PR against emscripten: https://github.com/emscripten-core/emscripten/pull/23633 I don't know if it's worth backporting it as a patch to the emscripten build used by emscripten-forge. The gain is a factor of 50 in function initialization, which sounds impressive, but as it only lasts one second, we gain only a second. (I thought it would be more).

But I have other ideas, and I hope they will add up. (Current ideas: using webstream from fetch to unpack and rearranging some of the initial loading will probably also gain some seconds.) . Just for the record, I will use Jupyter Lite for creating applets in lectures, and startup time is crucial for user acceptance, that is why I care about it.

[martinRenou]

I don't know if it's worth backporting it as a patch to the emscripten build used by emscripten-forge.

cc. @DerThorsten. I'd personally be up for it 👍🏽

Current ideas: using webstream from fetch to unpack and rearranging some of the initial loading will probably also gain some seconds

I can't wait to see what you come up with :D

Thanks a lot for your valuable contributions!

[martenrichter]

Thanks to the Jupyter community for providing this extensible software. So that I have a way to extend my software with applets that average PhD students and researcher! So I am happy to give something back on the way.

[DerThorsten]

I don't know if it's worth backporting it as a patch to the emscripten build used by emscripten-forge.

cc. @DerThorsten. I'd personally be up for it 👍🏽

we could try to make this a patch for the 3.1.73 branch.

[martenrichter]

Ok, but we have to wait a bit. During review process my initial changes were changed and are failing currently in CI/CD (the initial should work fine), so we have to wait until next weekend.

[westurner]

Would RustPython be faster at init and why?

• https://github.com/jupyterlite/xeus/issues/189

[martenrichter]

I do not know; it depends on the number of functions that are imported. If it is a huge number, yes, it would be faster. But we are speaking, for example, about a timespan of almost a second, which can be made after (see my measurements in the emscripten path), so it is not much.

Also, my patch hangs currently at review.

[martinRenou]

Thanks again for your PR in emscripten. Once/if this gets merged, the remaining pain points for startup times may be:

• Import of IPython/traitlets, which could potentially be made faster if we're able to speedup traitlets itself
• Have a faster way to handle pre-installed packages (we're already working on something at QuantStack concerning this, stay tuned!)

Concerning the IPython issue, we also provide a XPython-raw kernel which doesn't use IPython under the hood. We currently don't provide it as a WASM kernel but that wouldn't be too much to add. Although without IPython you'd be missing many features it provides.