frankenphp icon indicating copy to clipboard operation
frankenphp copied to clipboard

feat: use mimalloc instead of musl's mallocng for static builds

Open dunglas opened this issue 1 year ago • 11 comments

The default musl allocator is slow for multi-threaded apps such as FrankenPHP and triggers weird bugs such as https://github.com/php/php-src/issues/13648.

The trickery in this patch (massively inspired by this article by Tweag) replaces the musl allocator by https://github.com/microsoft/mimalloc, a fast allocator created by Microsoft.

Hopefully, this will also help for https://github.com/dunglas/frankenphp/issues/481#issuecomment-1987287765.

dunglas avatar Mar 15 '24 16:03 dunglas

Oho, gonna test this one real quick.

withinboredom avatar Mar 15 '24 17:03 withinboredom

I also get random segfaults with the OpenSSL extension. This isn't ready for prime time 😅

dunglas avatar Mar 15 '24 18:03 dunglas

Could it be the libraries are linked without mimalloc?

withinboredom avatar Mar 15 '24 18:03 withinboredom

@withinboredom could you give more details on the modules sefgfaulting in your tests? It looks like the OpenSSL issue may not be related to malloc after all.

dunglas avatar Apr 03 '24 07:04 dunglas

@dunglas why can't glibc be used there even if it means shipping something else than Alpine Linux? Or alternatively wouldn't it work with gcompat?

theofidry avatar Apr 03 '24 16:04 theofidry

@theofidry I'm not sure to follow. We provide glibc builds too (in the Docker images), but glibc doesn't support creating static builds. We have to use musl for that. See also: https://github.com/crazywhalecc/static-php-cli/issues/376

dunglas avatar Apr 03 '24 16:04 dunglas

Just FYI lots of openssl tests fails in Alpinelinux with openssl (even years ago with libressl)

Current set of disabled openssl tests can be found at repo At the end of the file new failures for # openssl 3.2.1

ext/openssl/tests/openssl_error_string_basic_openssl3.phpt
ext/openssl/tests/openssl_private_decrypt_basic.phpt
ext/openssl/tests/openssl_x509_parse_basic.phpt

andypost avatar Apr 04 '24 00:04 andypost

Checked where alternate *malloc is used in Alpinelinux

  • https://pkgs.alpinelinux.org/package/edge/main/x86_64/scudo-malloc (rust,firefox,lld)
  • https://pkgs.alpinelinux.org/package/edge/main/x86_64/jemalloc (only blender)

andypost avatar Apr 04 '24 17:04 andypost

I'll give this a test again this weekend.

withinboredom avatar Apr 11 '24 09:04 withinboredom

@withinboredom what benchmark were you running in https://github.com/dunglas/frankenphp/pull/666#pullrequestreview-1939793858 ? Is there a way I could reproduce this ? In my benchmarks the difference between glibc and musl are not as dramatic (and alternative allocators also don't do as much difference)

arnaud-lb avatar Apr 11 '24 20:04 arnaud-lb

It's actually a benchmark of an app I'm building. It results in a lot of syscalls deep under the hood (lots of file access, shm, proxy generation, etc).

withinboredom avatar Apr 12 '24 11:04 withinboredom

I added a new experimental flag to build using mimalloc and added a CI pipeline to build a mimalloc variant automatically.

dunglas avatar Apr 23 '24 10:04 dunglas