Haithem Satoutah

EXO on Jetson AGX Orin: full GPU load yet only 5 tokens/s—any ideas?

6

 Hi everyone, I’m running EXO on a Jetson AGX Orin with 32 GB of VRAM, but the performance is nowhere near what I expected. jtop shows the GPU pegged...

Installation of Exo (clustering software) fails inside Jetson Docker container

4

### Search before asking - [x] I have searched the jetson-containers [issues](https://github.com/dusty-nv/jetson-containers/issues) and found no similar feature requests. ### jetson-containers Component Docker ### Bug Problem I first tried installing Exo...

Support an MLX CUDA backend to improve Exo inference on NVIDIA GPUs

9

MLX just landed a native CUDA backend - see PR #1983 [Daily.dev](https://app.daily.dev/posts/wip-cuda-backend-by-zcbenz-pull-request-1983-ml-explore-mlx-ergfm6crk , https://github.com/ml-explore/mlx/pull/1983) ; installing pip install mlx-cuda gives matmul and other kernels running directly on NVIDIA GPUs. My...