Oliver Schiessl comments

Repositories
Issues
Comments

Results 2 comments of


                                            Oliver Schiessl

error "Item size 2 for PEP 3118 buffer format string B does not match the dtype B item size 1"

> I think the problem is that numpy doesn't support bf16 and in [sharded_inference_engine.py](https://github.com/exo-explore/exo/blob/main/exo/inference/mlx/sharded_inference_engine.py), the program tries to convert mlx tensors to numpy array hence causing the error. > >...

error "Item size 2 for PEP 3118 buffer format string B does not match the dtype B item size 1"

> [@lordoliver](https://github.com/lordoliver) can you check if it slows down non-bf16 models too? or if it breaks the code in worst case scenario? I am not much into it, maybe its...