jeff31415

运行预训练脚本发生AttributeError

3

### 详细描述问题 尝试进行预训练，训练数据加载和处理阶段发生错误。 以下是run_pt.sh的配置： ``` lr=2e-4 lora_rank=8 lora_alpha=32 lora_trainable="q_proj,v_proj,k_proj,o_proj,gate_proj,down_proj,up_proj" modules_to_save="embed_tokens,lm_head" lora_dropout=0.05 pretrained_model=/root/autodl-tmp/openllama-3b-350bt/config.json chinese_tokenizer_path=/root/autodl-tmp/openllama-3b-350bt/tokenizer.model dataset_dir=/root/autodl-tmp/Dataset data_cache=/root/autodl-tmp/Data_cache per_device_train_batch_size=1 per_device_eval_batch_size=1 training_steps=10 gradient_accumulation_steps=1 output_dir=/root/autodl-tmp/Output ``` 看起来和`run_clm_pt_with_peft.py`中的`group_texts(examples):`(ln434)之下的`lm_datasets`以及处理代码有关，导致其缺少'train_test_split'方法 ### 运行截图或日志 ``` [INFO|tokenization_utils_base.py:1809] 2023-05-23 00:45:34,889 >> loading...

Seems unable to use the "Dynamic High Resolution" feature of llava1.6 (aka llava-next)

3

https://llava-vl.github.io/blog/2024-01-30-llava-next/ Thanks for supporting llava1.6, but Ollama currently seems still unable to use the "Dynamic High Resolution" feature, which is important for llava1.6 to achieve state-of-the-art OCR accuracy and hallucination...