Alex Cheema
Alex Cheema
- We support several `InferenceEngine` implementations including MLX, tinygrad and PyTorch - Sometimes you want to test everything **except** the inference itself - It would be helpful to have a...
- As expected since we don't have tests for vision models, they broke unexpectedly: https://github.com/exo-explore/exo/issues/327 - Create a test similar to the existing one (`chatgpt_api_integration_test_mlx`) we have for text in...
# Background I experimented with a rust-based exo implementation that used [UniFFI](https://mozilla.github.io/uniffi-rs/latest/) for foreign language bindings so I could run it from a Swift iOS app. I didn't like this...
Like this: https://x.com/reach_vb/status/1846545312548360319 ```sh exo run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 ``` This should work out of the box with #139
Right now, inference engines are not interoperable e.g. you can't run one machine on `MLX` and another on `tinygrad` - they need to both be using the same inference engine....
- Currently we support Llama 3.2 1B on MLX but not tinygrad - Add support for Llama 3.2 1B - Might just work out of the box, if not I...
#368 + #381 added support for manual configuration but if you update the configuration file while exo is running it won't update. This should be a simple fix, hence $100...
- Tried adding this recently - Commented out because it was causing issues - No reason this shouldn't work, should be a simple fix