[Usage] train() got an unexpected keyword argument 'attn_implementation'

[ATing0203]

Describe the issue

Issue: when I run the pretrain.sh, it reports an error. Command:

nohup sh scripts/v1_5/pretrain.sh > pretrain_log.txt 2>&1 &

Log:

Traceback (most recent call last):
  File "/data/LLaVA/llava/train/train_mem.py", line 4, in <module>
  File "/data/LLaVA/llava/train/train_mem.py", line 4, in <module>
  File "/data/LLaVA/llava/train/train_mem.py", line 4, in <module>
Traceback (most recent call last):
  File "/data/LLaVA/llava/train/train_mem.py", line 4, in <module>
            train(attn_implementation="flash_attention_2")train(attn_implementation="flash_attention_2")train(attn_implementation="flash_attention_2")

TypeError    TypeError: TypeErrortrain(attn_implementation="flash_attention_2"): train() got an unexpected keyword argument 'attn_implementation': 
train() got an unexpected keyword argument 'attn_implementation'
train() got an unexpected keyword argument 'attn_implementation'
TypeError
: train() got an unexpected keyword argument 'attn_implementation'

with CUDA11.7, pytorch=2.0.1, torchvision=0.15.2, transformers=4.37.2, llava=1.2.2.post1, flash-attn=2.5.5

[LALBJ]

In my environment, this problem is caused by invoking the llava package in the pip environment. So, I solved this problem by adding the local file llava to the python environment through the addition of the PYTHONPATH parameter when executing bash in the terminal.

Here is an example: PYTHONPATH=/path/to/your/llava:$PYTHONPATH bash your_script.sh

[JJJYmmm]

In my environment, this problem is caused by invoking the llava package in the pip environment. So, I solved this problem by adding the local file llava to the python environment through the addition of the PYTHONPATH parameter when executing bash in the terminal.

Here is an example: PYTHONPATH=/path/to/your/llava:$PYTHONPATH bash your_script.sh

Excume me, I have a little problem. I find attn_implementation is used here in train.py.

https://github.com/haotian-liu/LLaVA/blob/3e337ad269da3245643a2724a1d694b5839c37f9/llava/train/train.py#L827-L830

But it seems that the argument is not used in LlavaLlamaForCausalLM.from_pretrained. Could you point out somewhere the argument is actually used?