💡 [REQUEST] - Run model v2.6 on Apple Silicon with mps-Pytorch Backend

[RobKnop]

起始日期 | Start Date

No response

实现PR | Implementation PR

No response

相关Issues | Reference Issues

No response

摘要 | Summary

I would like to run the model with Pytorch on my Macbook (in this case M2 Pro) with mps to have greater control about the prompting. Inspiration is the v2.5 model.

基本示例 | Basic Example

Current issue, when trying is: NotImplementedError: The operator 'aten::isin.Tensor_Tensor_out' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

缺陷 | Drawbacks

No, I can think of

未解决问题 | Unresolved questions

No response