tutorial icon indicating copy to clipboard operation
tutorial copied to clipboard

Inconsistency between "Starting Ray with <x> GiB" and docker shm_size: <y>

Open free-soellingeraj opened this issue 5 years ago • 0 comments

My docker-compose.yml defines the shm_size: '32gb' but when I run ray.init(num_cpus=4, ignore_reinit_error=True, include_dashboard=False) I get the log line:

2020-08-26 20:31:09,651 INFO resource_spec.py:231 -- Starting Ray with 37.5 GiB memory available for workers and up to 18.76 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).

So I am naturally curious:

How is ray getting more memory than is allotted to the container in which it's running?

Thanks in advance

free-soellingeraj avatar Aug 26 '20 20:08 free-soellingeraj