Antoine Pitrou
Antoine Pitrou
cc @felipecrv
I disagree and think we should try to add a test for this, if only to validate that we are actually fixing something.
Let's cc some of the maintainers of [parquet-rs](https://github.com/apache/arrow-rs/tree/master/parquet): @adamgs @tustvold @alamb
> @mapleFU @gszadovszky @pitrou @wgtmac What is your opinion on this proposal? It's difficult to say without understanding the implications. Say a data page contains some NaNs, what happens?
@JFinis Thanks a lot! I agree that makes sense. The main problem IMHO is that old readers wouldn't support page filtering on such files. That said, we have to move...
Here is a quick REPL example: ```python >>> import pyarrow as pa # mimalloc >>> pool = pa.mimalloc_memory_pool() >>> a = pa.array([0]*1_000_000, memory_pool=pool) >>> pool.bytes_allocated() 8000000 # jemalloc >>> pool...
> The other option is that it should have a symbol called `mi_malloc@plt` or similar (in the normal symbol table). Hmm. How would you do that using gcc or clang?...
Hmm, actually, a function attribute wouldn't work, because we would have to patch the mimalloc source code for that... (also, we use `-fno-semantic-interposition` and I'm unsure how it influences `__attribute__((visibility("default")))`)
> I am afraid this is the classic compromise between performance and observability. I agree. We could definitely make an exception for mimalloc and jemalloc calls, however, it's just that...
> I think trying to use a `__attribute__((visibility("default")))` or marking the symbol as weak (`__attribute__((weak))`) may be worth a try. I thought so, but I realized it required patching the...