Not running on GPU

[inar-vision]

Prerequisites

• [X] I have read the documentation.
• [X] I have checked other issues for similar problems.

Backend

Local

Interface Used

CLI

CLI Command

autotrain llm --train --project-name legalized-gpt --model TurkuNLP/gpt3-finnish-xl --data-path . --use-peft --quantization int4 --lr 2e-4 --train-batch-size 6 --epochs 3 --trainer sft

UI Screenshots & Parameters

No response

Error Logs

(base) C:\Users\korho>autotrain llm --train --project-name legalized-gpt --model TurkuNLP/gpt3-finnish-xl --data-path . --use-peft --quantization int4 --lr 2e-4 --train-batch-size 6 --epochs 3 --trainer sft

INFO Running LLM INFO Params: Namespace(version=False, text_column='text', rejected_text_column='rejected', prompt_text_column='prompt', model_ref=None, warmup_ratio=0.1, optimizer='adamw_torch', scheduler='linear', weight_decay=0.0, max_grad_norm=1.0, add_eos_token=False, block_size=-1, peft=True, lora_r=16, lora_alpha=32, lora_dropout=0.05, logging_steps=-1, evaluation_strategy='epoch', save_total_limit=1, save_strategy='epoch', auto_find_batch_size=False, mixed_precision=None, quantization='int4', model_max_length=1024, trainer='sft', target_modules=None, merge_adapter=False, use_flash_attention_2=False, dpo_beta=0.1, apply_chat_template=False, padding=None, train=True, deploy=False, inference=False, username=None, backend='local-cli', token=None, repo_id=None, push_to_hub=False, model='TurkuNLP/gpt3-finnish-xl', project_name='legalized-gpt', seed=42, epochs=3, gradient_accumulation=1, disable_gradient_checkpointing=False, lr=0.0002, log='none', data_path='.', train_split='train', valid_split=None, batch_size=6, func=<function run_llm_command_factory at 0x0000020D32F04A40>) INFO Starting local training... INFO {"model":"TurkuNLP/gpt3-finnish-xl","project_name":"legalized-gpt","data_path":".","train_split":"train","valid_split":null,"add_eos_token":false,"block_size":-1,"model_max_length":1024,"padding":null,"trainer":"sft","use_flash_attention_2":false,"log":"none","disable_gradient_checkpointing":false,"logging_steps":-1,"evaluation_strategy":"epoch","save_total_limit":1,"save_strategy":"epoch","auto_find_batch_size":false,"mixed_precision":null,"lr":0.0002,"epochs":3,"batch_size":6,"warmup_ratio":0.1,"gradient_accumulation":1,"optimizer":"adamw_torch","scheduler":"linear","weight_decay":0.0,"max_grad_norm":1.0,"seed":42,"apply_chat_template":false,"quantization":"int4","target_modules":null,"merge_adapter":false,"peft":true,"lora_r":16,"lora_alpha":32,"lora_dropout":0.05,"model_ref":null,"dpo_beta":0.1,"prompt_text_column":"prompt","text_column":"text","rejected_text_column":"rejected","push_to_hub":false,"repo_id":null,"username":null,"token":null} WARNING No GPU found. Forcing training on CPU. This will be super slow!

Additional Information

So trying to run this locally, but it seems to start on CPU instead of my GPU. I have RTX306012gb card. I might be doing something very wrong, as I'm not too familiar with this yet.

NVIDIA-SMI 546.33 Driver Version: 546.33 CUDA Version: 12.3