edgedb-python icon indicating copy to clipboard operation
edgedb-python copied to clipboard

Audit for thread safety

Open msullivan opened this issue 8 months ago • 3 comments

I'm not sure if we have thought sufficiently about thread safety and the blocking client. We document that we are thread safe, but there are some shared caches, at least, and so we should double check and internally document that their use is OK.

msullivan avatar Apr 24 '25 23:04 msullivan

https://github.com/geldata/gel-python/blob/bb93ed75d420a4b7029c819fc8b9e5b467cb267f/gel/protocol/protocol.pyx#L1181-L1184

CodecsRegistry uses LRUMappings. If type_id gets evicted from the LRU between has_codec and get_codec, then we will raise a LookupError.

msullivan avatar Apr 24 '25 23:04 msullivan

https://github.com/geldata/gel-python/blob/bb93ed75d420a4b7029c819fc8b9e5b467cb267f/gel/protocol/lru.pyx#L51

The thread-safety of OrderedDict is not 100% clear to me. It's certainly not documented as thread-safe, and the reference python implementation is not. The C implementation probably is, though I should double check, and I wonder what it will be like with nogil.

msullivan avatar Apr 25 '25 00:04 msullivan

gel-python/gel/protocol/protocol.pyx

Lines 1181 to 1184 in bb93ed7 if reg.has_codec(type_id): out_dc = reg.get_codec(type_id) else: out_dc = reg.build_codec(type_data, self.protocol_version)

CodecsRegistry uses LRUMappings. If type_id gets evicted from the LRU between has_codec and get_codec, then we will raise a LookupError.

Oh, though, since this is all cython, maybe this can't happen unless we call to something that runs actual python that might drop the GIL? I'm a little nervous to rely on this, though.

msullivan avatar Apr 25 '25 00:04 msullivan