torchchat icon indicating copy to clipboard operation
torchchat copied to clipboard

unimplemented operators - workarounds and long term perspective

Open mikekgfb opened this issue 9 months ago • 0 comments

Today users have to set PYTORCH_ENABLE_MPS_FALLBACK=1 when they call torchchat if they want to use _weight_int4pack_mm. Can we set that automatically, from inside the program. This is a crude workaround, maybe we can get an implementation of _weight_int4pack_mm for MPS? (This would also be goodness for mobile.)

mikekgfb avatar Apr 30 '24 05:04 mikekgfb