lagom
lagom copied to clipboard
Context container might not be thread safe
Lastly, I encountered an error while trying to use a same context container with a context manager in parallel threads. One of my dependency could not be resolved in one thread as if the definition of that dependency was no longer existing in the container.
I've made a code snippet to reproduce the issue
import random
from collections.abc import Iterable
from concurrent.futures import ThreadPoolExecutor
from lagom import (
Container,
ContextContainer,
ExplicitContainer,
context_dependency_definition,
dependency_definition,
)
class Object:
def __init__(self) -> None:
self.id = random.randint(0, 9999)
class MySingleton(Object):
pass
class MyContextSingleton(Object):
def __init__(self, foo: str) -> None:
super().__init__()
self.foo = foo
def close(self) -> None:
print(f"closing {self.id}")
class MyDependency(Object):
def __init__(self, ctx: MyContextSingleton) -> None:
super().__init__()
self.ctx = ctx
class MyService(Object):
def __init__(self, dep: MyDependency) -> None:
super().__init__()
self.dep = dep
container = ExplicitContainer(log_undefined_deps=True)
@dependency_definition(container, singleton=True)
def _get_my_singleton() -> MySingleton:
return MySingleton()
@dependency_definition(container)
def _get_my_service(c: Container) -> MyService:
return MyService(c[MyDependency])
@dependency_definition(container)
def _get_my_dependency(c: Container) -> MyDependency:
return MyDependency(c[MyContextSingleton])
@context_dependency_definition(container)
def _get_my_context_singleton() -> Iterable[MyContextSingleton]:
cli = MyContextSingleton("bar")
try:
yield cli
finally:
cli.close()
context_container = ContextContainer(
container, context_types=[], context_singletons=[MyContextSingleton]
)
def test(num: int) -> None:
with context_container as c:
service1 = c[MyService]
service2 = c[MyService]
print(
f"Thread {num} - MySingleton {c[MySingleton].id} - MyService {service1.id} - MyDependency {service1.dep.id} - MyContextSingleton {service1.dep.ctx.id}"
)
print(
f"Thread {num} - MySingleton {c[MySingleton].id} - MyService {service2.id} - MyDependency {service2.dep.id} - MyContextSingleton {service1.dep.ctx.id}"
)
with ThreadPoolExecutor() as pool:
f1 = pool.submit(test, 1)
f2 = pool.submit(test, 2)
f1.result()
f2.result()
This snippet tries to run a same function in two parallel threads. That function is creating a context manager for our context container and pull our singleton and service. I expected to have the same singleton instance for both threads as well as an unique MyContextSingleton
instance per thread (because it is a context singleton).
However It ends up with the following error
Thread 1 - MySingleton 7874 - MyService 3904 - MyDependency 922 - MyContextSingleton 989
Thread 1 - MySingleton 7874 - MyService 1379 - MyDependency 9802 - MyContextSingleton 989
closing 989
Traceback (most recent call last):
File "lagom\container.py", line 391, in _reflection_build_with_err_handling
lagom.exceptions.UnresolvableType: Unable to construct dependency of type str The constructor probably has some unresolvable dependencies: str
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "lagom\container.py", line 392, in _reflection_build_with_err_handling
File "lagom\container.py", line 406, in _reflection_build
File "lagom\container.py", line 425, in _infer_dependencies
File "lagom\container.py", line 265, in resolve
File "lagom\container.py", line 395, in _reflection_build_with_err_handling
lagom.exceptions.UnresolvableType: Unable to construct dependency of type str The constructor probably has some unresolvable dependencies: str
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\Users\TBEBGOD\Developement\My-project\snippet.py", line 93, in <module>
f2.result()
File "C:\Users\TBEBGOD\.pyenv\pyenv-win\versions\3.10.11\lib\concurrent\futures\_base.py", line 458, in result
return self.__get_result()
File "C:\Users\TBEBGOD\.pyenv\pyenv-win\versions\3.10.11\lib\concurrent\futures\_base.py", line 403, in __get_result
raise self._exception
File "C:\Users\TBEBGOD\.pyenv\pyenv-win\versions\3.10.11\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "c:\Users\TBEBGOD\Developement\My-project\snippet.py", line 78, in test
service1 = c[MyService]
File "lagom\container.py", line 381, in __getitem__
File "lagom\container.py", line 259, in resolve
File "lagom\definitions.py", line 37, in get_instance
File "c:\Users\TBEBGOD\Developement\My-project\snippet.py", line 54, in _get_my_service
return MyService(c[MyDependency])
File "lagom\container.py", line 381, in __getitem__
File "lagom\container.py", line 259, in resolve
File "lagom\definitions.py", line 37, in get_instance
File "c:\Users\TBEBGOD\Developement\My-project\snippet.py", line 59, in _get_my_dependency
return MyDependency(c[MyContextSingleton])
File "lagom\container.py", line 381, in __getitem__
File "lagom\container.py", line 265, in resolve
File "lagom\container.py", line 395, in _reflection_build_with_err_handling
lagom.exceptions.UnresolvableType: Unable to construct dependency of type MyContextSingleton The constructor probably has some unresolvable dependencies: MyContextSingleton
The first thread shows the container is working as expected, in particular the two services resolved ends up with the same MyContextSingleton
. The second shows that the container level singleton is working as expected because we got the same instance in both threads. However, the second thread crash when trying to resolve a MyContextSingleton
.
It feels like on the second thread the definition for MyContextSingleton
no longer exist causing the container to use reflection to resolve it (while it should not) which obviously fails because it can not resolve the foo: str
dependency.
Note that, if I clone the container before opening the context, it works as expected (with context_container.clone() as c:
.
If I run the two threads serially it works as well.