ut-parla
Avoid VEC Cache Clearing Hacks By Fixing Upstream
Our hacks to clear internal CPython caches to get VECs to work both seem like they may be addressing issues that could arguably be fixed upstream.
One is working around an ancient cache of shared object handles at https://github.com/python/cpython/blob/2d2af320d94afc6561e8f8adf174c9d3fd9065bc/Python/dynload_shlib.c#L51-L55. That looks like an old optimization that may not even be applicable with more modern libc implementations.
Another is working around the fact that the cache of module-spec objects for compiled extensions isn't cleared properly when a user runs importlib.invalidate_caches. See https://github.com/python/cpython/blob/dff1ad509051f7e07e77d1e3ec83314d53fb1118/Python/import.c#L440-L454 for the cache this refers to and https://github.com/ut-parla/Parla.py/blob/master/parla/multiload.py#L74-L79 for our hack to get at it so we can swap out entries when working in different VECs. While I'm not 100% sure how to fix this upstream, this does go against the documented behavior of the import system. invalidate_caches is supposed to invalidate all internal caches, and IIUC, that applies to this cace. A similar issue https://bugs.python.org/issue33169 was fixed semi-recently.
In both cases we'll need to make the case for fixes upstream, but in both cases I think we could eventually divest ourselves of some of the more evil hacks involved in getting VECs working. For now we just need to figure out how to get the ball rolling on fixes upstream. Even an acknowledgment that a fix belongs in CPython would help us with our story since these would become temporary workarounds for bugs upstream rather than insane implementation details.
Someone should create an upstream issue for this against cpython. I agree that either fix would be reasonable. I suspect the second option would be easier to get merged. Also I think having VECs require a really new or even pre-release cpython is very reasonable.
Adding this to the milestone for the next VEC paper. My main goal with the time period that we have is to get these things acknowledged as stuff that needs to be fixed upstream so we can position our hacks as temporary workarounds.
Okay, bug reported the module spec cache issue upstream at https://bugs.python.org/issue43819. No idea what they'll say, but at least now we've asked.
I'll try bringing up the shared object handle issue on python-dev since it's more of a meta-level "what do we do about this?"
Sigh. I tried getting rid of the shared object handle cache in cpython and initial experiments indicate that may actually still be providing a slight performance improvement. I'll collect some more careful/thorough numbers.
Yah, we'll see if that was a fluke from me testing on a busy machine or if there's actually any difference. Even if it actually matters, I'm actually more okay living with the "saturate the cache" hack anyway, so it's not as big of a deal.
Alright, after collecting more data, I'm seeing some slight performance improvement without the cache. It's not statistically significant though since there's a lot of noise in the data. I'll collect more samples and see if there's a more direct way to test this other than just timing the cpython test suite.
Clearing from the milestone since I don't expect upstream to move quickly on any of these issues. We'll just have to de-emphasize the hacks as central since we're still trying to get some stuff fixed upstream to make the hacks unnecessary.
Also submitted https://bugs.python.org/issue43895 and https://github.com/python/cpython/pull/25487 upstream. No idea if it'll garner any comment there since it's a minor cleanup item from their perspective. Either way we have a workaround here, so it's no longer a big deal.
Shared object handles patch is accepted upstream, so that's one down.