[BUG] AttributeError: module 'autotrain.trainers.clm.utils' has no attribute 'DEFAULT_CHAT_TEMPLATE'

[altunyurt]

Prerequisites

• [X] I have read the documentation.
• [X] I have checked other issues for similar problems.

Backend

Colab

Interface Used

CLI

CLI Command

!autotrain llm --train --project-name "llama2-openassistant"
--model TinyPixel/Llama-2-7B-bf16-sharded
--data-path timdettmers/openassistant-guanaco
--text_column text
--use-peft
--quantization int4
--lr 2e-4
--batch-size 2
--epochs 3
--trainer sft
--model_max_length 2048
--push-to-hub
--repo-id $USERNAME/llama2-openassistant
--token $TOKEN
--block-size 2048 > training.log &

UI Screenshots & Parameters

No response

Error Logs

INFO Running LLM INFO Params: Namespace(version=False, text_column='text', rejected_text_column='rejected', prompt_text_column='prompt', model_ref=None, warmup_ratio=0.1, optimizer='adamw_torch', scheduler='linear', weight_decay=0.0, max_grad_norm=1.0, add_eos_token=False, block_size=2048, peft=True, lora_r=16, lora_alpha=32, lora_dropout=0.05, logging_steps=-1, evaluation_strategy='epoch', save_total_limit=1, save_strategy='epoch', auto_find_batch_size=False, mixed_precision=None, quantization='int4', model_max_length=2048, trainer='sft', target_modules=None, merge_adapter=False, use_flash_attention_2=False, dpo_beta=0.1, chat_template=None, padding=None, train=True, deploy=False, inference=False, username=None, backend='local-cli', token='...', repo_id='.../llama2-openassistant', push_to_hub=True, model='TinyPixel/Llama-2-7B-bf16-sharded', project_name='llama2-openassistant', seed=42, epochs=3, gradient_accumulation=1, disable_gradient_checkpointing=False, lr=0.0002, log='none', data_path='timdettmers/openassistant-guanaco', train_split='train', valid_split=None, batch_size=2, func=<function run_llm_command_factory at 0x7a0ba3377010>) INFO Starting local training... INFO {"model":"TinyPixel/Llama-2-7B-bf16-sharded","project_name":"llama2-openassistant","data_path":"timdettmers/openassistant-guanaco","train_split":"train","valid_split":null,"add_eos_token":false,"block_size":2048,"model_max_length":2048,"padding":null,"trainer":"sft","use_flash_attention_2":false,"log":"none","disable_gradient_checkpointing":false,"logging_steps":-1,"evaluation_strategy":"epoch","save_total_limit":1,"save_strategy":"epoch","auto_find_batch_size":false,"mixed_precision":null,"lr":0.0002,"epochs":3,"batch_size":2,"warmup_ratio":0.1,"gradient_accumulation":1,"optimizer":"adamw_torch","scheduler":"linear","weight_decay":0.0,"max_grad_norm":1.0,"seed":42,"chat_template":null,"quantization":"int4","target_modules":null,"merge_adapter":false,"peft":true,"lora_r":16,"lora_alpha":32,"lora_dropout":0.05,"model_ref":null,"dpo_beta":0.1,"prompt_text_column":"prompt","text_column":"text","rejected_text_column":"rejected","push_to_hub":true,"repo_id":".../llama2-openassistant","username":null,"token":"..."} INFO ['accelerate', 'launch', '--num_machines', '1', '--num_processes', '1', '--mixed_precision', 'no', '-m', 'autotrain.trainers.clm', '--training_config', 'llama2-openassistant/training_params.json'] The following values were not passed to accelerate launch and had defaults used instead: --dynamo_backend was set to a value of 'no' To avoid this warning pass in values for each of the problematic parameters or run accelerate config. Downloading readme: 100% 395/395 [00:00<00:00, 2.56MB/s] /usr/local/lib/python3.10/dist-packages/huggingface_hub/repocard.py:105: UserWarning: Repo card metadata block was not found. Setting CardData to empty. warnings.warn("Repo card metadata block was not found. Setting CardData to empty.") Downloading data files: 0% 0/2 [00:00<?, ?it/s] Downloading data: 0% 0.00/20.9M [00:00<?, ?B/s] Downloading data: 20% 4.19M/20.9M [00:00<00:02, 5.68MB/s] Downloading data: 60% 12.6M/20.9M [00:00<00:00, 16.8MB/s] Downloading data: 100% 20.9M/20.9M [00:01<00:00, 17.2MB/s] Downloading data files: 50% 1/2 [00:01<00:01, 1.22s/it] Downloading data: 0% 0.00/1.11M [00:00<?, ?B/s] Downloading data: 100% 1.11M/1.11M [00:00<00:00, 6.36MB/s] Downloading data files: 100% 2/2 [00:01<00:00, 1.44it/s] Extracting data files: 100% 2/2 [00:00<00:00, 1884.66it/s] Generating train split: 9846 examples [00:00, 60811.90 examples/s] Generating test split: 518 examples [00:00, 61780.92 examples/s] 🚀 INFO | 2024-02-22 22:44:19 | main:process_input_data:109 - Train data: Dataset({ features: ['text'], num_rows: 9846 }) 🚀 INFO | 2024-02-22 22:44:19 | main:process_input_data:110 - Valid data: None tokenizer_config.json: 100% 676/676 [00:00<00:00, 3.59MB/s] tokenizer.model: 100% 500k/500k [00:00<00:00, 264MB/s] tokenizer.json: 100% 1.84M/1.84M [00:00<00:00, 17.0MB/s] special_tokens_map.json: 100% 411/411 [00:00<00:00, 1.70MB/s] ❌ ERROR | 2024-02-22 22:44:20 | autotrain.trainers.common:wrapper:91 - train has failed due to an exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/autotrain/trainers/common.py", line 88, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/autotrain/trainers/clm/main.py", line 171, in train tokenizer.chat_template = utils.DEFAULT_CHAT_TEMPLATE AttributeError: module 'autotrain.trainers.clm.utils' has no attribute 'DEFAULT_CHAT_TEMPLATE'

❌ ERROR | 2024-02-22 22:44:20 | autotrain.trainers.common:wrapper:92 - module 'autotrain.trainers.clm.utils' has no attribute 'DEFAULT_CHAT_TEMPLATE'

Additional Information

https://github.com/huggingface/autotrain-advanced/blob/593ca6276243b6940d8111a6456ae7188da26828/src/autotrain/trainers/clm/main.py#L170 refers to variable DEFAULT_CHAT_TEMPLATE which does not exist in clm/utils.py