loguru
loguru copied to clipboard
Pickle error on using loguru with ray
Hi. I'm facing this error on running loguru with ray remote. I think I'm missing something here. I went through the documentation and I couldn't find anything helpful to solve my specific issue. I noticed from issue #386 that @Delgan has mentioned "The logger is picklable as long as the added handlers are picklable too". Is it because of this? Appreicate the help. Thanks for the wonderful library. Loguru has made logging very easy.
Code
from loguru import logger
import ray
logger.add(f'logs/{__name__}', filter=__name__, rotation='1 week')
@ray.remote
class StartUp:
def start(self):
for i in range(10000):
logger.info(i)
def run():
logger.info("started!")
agents = [StartUp.remote() for i in range(4)]
result = []
for count , agent in enumerate(agents):
logger.info(count)
result.append(agent.start.remote())
done_id, _ = ray.wait(result)
logger.info("complete!")
if __name__ == "__main__":
run()
Dependencies
[tool.poetry.dependencies]
python = "3.8.13"
ray = {extras = ["default"], version = "1.13.0"}
loguru = "0.6.0"
Error
Traceback (most recent call last):
File "test.py", line 23, in <module>
run()
File "test.py", line 13, in run
agents = [StartUp.remote() for i in range(4)]
File "test.py", line 13, in <listcomp>
agents = [StartUp.remote() for i in range(4)]
File "/home/blue/.cache/pypoetry/virtualenvs/loguru-ray-test-l4hNMfu6-py3.8/lib/python3.8/site-packages/ray/actor.py", line 529, in remote
return self._remote(args=args, kwargs=kwargs, **self._default_options)
File "/home/blue/.cache/pypoetry/virtualenvs/loguru-ray-test-l4hNMfu6-py3.8/lib/python3.8/site-packages/ray/util/tracing/tracing_helper.py", line 387, in _invocation_actor_class_remote_span
return method(self, args, kwargs, *_args, **_kwargs)
File "/home/blue/.cache/pypoetry/virtualenvs/loguru-ray-test-l4hNMfu6-py3.8/lib/python3.8/site-packages/ray/actor.py", line 844, in _remote
worker.function_actor_manager.export_actor_class(
File "/home/blue/.cache/pypoetry/virtualenvs/loguru-ray-test-l4hNMfu6-py3.8/lib/python3.8/site-packages/ray/_private/function_manager.py", line 461, in export_actor_class
serialized_actor_class = pickle.dumps(Class)
File "/home/blue/.cache/pypoetry/virtualenvs/loguru-ray-test-l4hNMfu6-py3.8/lib/python3.8/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/home/blue/.cache/pypoetry/virtualenvs/loguru-ray-test-l4hNMfu6-py3.8/lib/python3.8/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 620, in dump
return Pickler.dump(self, obj)
File "/home/blue/.cache/pypoetry/virtualenvs/loguru-ray-test-l4hNMfu6-py3.8/lib/python3.8/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 320, in _file_reduce
raise pickle.PicklingError(
_pickle.PicklingError: Cannot pickle files that are not opened for reading: a