radhacr comments

Repositories
Issues
Comments

Results 1 comments of


                                            radhacr

Gemma inference CUDA illegal memory access error

Same thing happens even without accelerate. ``` $ python -m axolotl.cli.inference examples/gemma/qlora.yml --lora_dir="axolotl-gemma-aplaca-qlora/" [2024-04-10 13:45:56,207] [INFO] [datasets.:58] [PID:2747] PyTorch version 2.2.2 available. [2024-04-10 13:45:58,395] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to cuda...