apcu 5.1.17 crash on windows with php 7.3.3 in apc_cache_wlocked_real_expunge

[samisallinen]

I consistently get an access violation crash on windows with the above APCU version when running unit tests.

I run the x86 thread safe version of PHP. I also use apc-bc 1.0.5.

Attached is a screenshot of the crash location in visual studio.

I will keep the crash dump here: https://www.dropbox.com/s/vl80nq2gv73yblu/apcu_crash_dump.zip?dl=0 at least for a month in case someone is interested.

[nikic]

I'm assuming you're running unit tests with a CLI binary, so there's no web server or threading involved here, right?

[samisallinen]

Actually, the unit test scripts accesses some mock objects on apache running on localhost so there is some multithreading.

to 21. maalisk. 2019 klo 16.43 Nikita Popov [email protected] kirjoitti:

I'm assuming you're running unit tests with a CLI binary, so there's no web server or threading involved here, right?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/krakjoe/apcu/issues/375#issuecomment-475258170, or mute the thread https://github.com/notifications/unsubscribe-auth/ARJ_VzrXx7e5oOz2oJPmHckNiDALbURYks5vY5qogaJpZM4cBANd .

[samisallinen]

Enabling apc only for cli seems to avoid the crash, so it indeed might have something to do with multithreading.

The kind of multithreading in my case should be quite benign, as whenever the web server does something, the unit test script just waits for the web request to complete.

[samisallinen]

Hi, I started looking at whether I can fix this myself to enable using apcu in my employer's product. So far I have found 3 things I think are issues and fixed 2 of them. I am wondering if I fix issue #3, whether there are chances of getting my changes merged to apcu. Also I am curious whether my analysis of the issues seems faulty to someone who has more history with apcu than I do. Here are the issues I have found so far:

1 - According to MS, using the slim read write locks on windows between processes does not work: https://docs.microsoft.com/en-us/windows/win32/sync/slim-reader-writer--srw--locks says "Slim reader-writer locks cannot be shared across processes.". 2 - Both apc_sma_init and apc_cache_create always overwrite the allocated shared memory at startup, so starting a child process always erases the cache (or may do other nasty things too if the main process is just handing the cache). 3 - The cache shared memory items apc_cache_entry_t contains pointers "key" and "next" that should be stored as offsets to the shared memory origin instead to be usable across processes. The same goes for apc_cache_t->slots.

While issue 1 only occurs on Windows, I think that the other 2 should also happen on linux.

This is what I have found so far, I am curious for comments on this.

[krakjoe]

Hi ...

I have a feeling you are trying to use apcu in an unsupported way ... 1 shouldn't matter, because only multithreaded environments are supported on windows. There's no problem with 2, it's correct to use MINIT as it's used for the models that apcu supports (prefork multiprocess or multithreaded). As for 3, refer to 1 and 2 (I think this is a result of your confusion about process models) :)

[nikic]

FWIW I think it would be nice if apcu support cross-process on Windows, but based on Anatol's previous attempts to make this work, this seems to be more complicated than just swapping the lock implementation. I believe his attempt is in https://github.com/krakjoe/apcu/tree/win_global_mutex.

3 - The cache shared memory items apc_cache_entry_t contains pointers "key" and "next" that should be stored as offsets to the shared memory origin instead to be usable across processes. The same goes for apc_cache_t->slots.

There's quite a few more pointers in the persistence implementation. If SHM can be mapped at different addresses we'd have to make the entirety of the SHM representation position-independent.

[samisallinen]

@krakjoe:

to use MINIT as it's used for the models that apcu supports (prefork multiprocess or >multithreaded). As for 3, refer to 1 and 2 (I think this is a result of your confusion about process > models) :)

I was not aware of the process model limitations you mentioned.

My use case arises from having unit tests run from cli that also access code within the web server running on the same machine. Also in our production system, some php command line scripts are run in addition to the web server, which probably also breaks the prefork part of the multi-process model.

@nikic:

There's quite a few more pointers in the persistence implementation. If SHM can be mapped at >different addresses we'd have to make the entirety of the SHM representation position-independent.

I started modifying the pointers I mentioned to be position independent, but if there is lots more, then I may have to drop this project and do some actual work instead :-(. As an very hacky solution, it is possible to suggest an address for the Windows API call MapViewOfFileEx, but there is no guarantee the OS is able to allocate the same address in different processes....

[TysonAndre]

2 - Both apc_sma_init and apc_cache_create always overwrite the allocated shared memory at startup, so starting a child process always erases the cache (in php 7.3) (or may do other nasty things too if the main process is just handing the cache).

php 7.4.6 implemented support for IPC_PRIVATE in the shmget polyfill, though I discovered a bug that would cause warnings when using multiple segments. https://github.com/php/php-src/issues/9944

• now new processes have their own distinct shared memory caches

The approach of using random numbers as keys and using shmctl(key, IPC_STAT, &out_buf) != -1 to check if the key was already allocated (before calling shmget) in the windows polyfill can be used to avoid this bug in older php versions during module initialization(MINIT) (and to avoid other conflicts in current/older versions if userland shmop_open is also called with 0 (see 9944))

• An implementation of that approach can be seen in https://github.com/TysonAndre/immutable_cache-pecl/commit/d217546aadb13aea17e46e3804252b7e7cd698ca#diff-d764ed4b44130f01561898a738b92f6fb3294b229e2047fe86c58e25159dee75 along with unrelated changes specific to that PECL

As an very hacky solution, it is possible to suggest an address for the Windows API call MapViewOfFileEx, but there is no guarantee the OS is able to allocate the same address in different processes....

Yeah, that's the workaround/hacky solution opcache uses in https://www.php.net/manual/en/opcache.configuration.php#ini.opcache.mmap-base for Windows only (mentioned in other issues)