torchchat
torchchat copied to clipboard
unimplemented operators - workarounds and long term perspective
Today users have to set PYTORCH_ENABLE_MPS_FALLBACK=1 when they call torchchat if they want to use _weight_int4pack_mm. Can we set that automatically, from inside the program. This is a crude workaround, maybe we can get an implementation of _weight_int4pack_mm for MPS? (This would also be goodness for mobile.)