exo Feature request: specify GPU # if several installed

Found for now it will pick first shown by nvidia-smi

Oct 06 '24 01:10 FFAMax

This also related partially to https://github.com/exo-explore/exo/issues/123

Oct 06 '24 03:10 FFAMax

I believe you can already do this with tinygrad by specifying VISIBLE_DEVICES see https://docs.tinygrad.org/env_vars/

Oct 06 '24 13:10 AlexCheema

I believe you can already do this with tinygrad by specifying VISIBLE_DEVICES see https://docs.tinygrad.org/env_vars/

Got chance to verify it. VISIBLE_DEVICES=0,1,2 and VISIBLE_DEVICES=1,2 will lead to the same result. Also in manual mentioned it may work only for AMD|NV Also opened a case to chase for it https://github.com/tinygrad/tinygrad/issues/7318

Need or chance make tinygrad work with multiply devices natively, or may specify exact device therefore can run several instances on the same host per each device.

Oct 27 '24 11:10 FFAMax

This one is correct for CUDA: to use both GPU need run something like CUDA_VISIBLE_DEVICES=0 exo --node-id=1 --node-port=8001 --listen-port=65001 CUDA_VISIBLE_DEVICES=1 exo --node-id=2 --node-port=8002 --listen-port=65002

Oct 30 '24 06:10 FFAMax