yadamonk

Results 2 comments of yadamonk

device_map='auto' doesn't use MPS backend on Apple M2

Manually moving a model to MPS does not seem to work. Below is a minimal example: ``` Python 3.10.9 | packaged by conda-forge | (main, Feb 2 2023, 20:26:08) [Clang...

NTK-Aware Scaled RoPE allows LLaMA models to have extended (8k+) context size without any fine-tuning and minimal perplexity degradation.

@Narsil So we can now use models like the llama2 orca 8k mentioned by @andreaskoepf?