attnmamba

@emmaking-smith Did you manage to fix this?

Same issue, looking for fix. Following the pip install instructions in README, I have `transformers==4.36.2` installed, which is the version pinned by `pyproject.toml` How does everyone else manage to run...

@haotian-liu Thank you! The new commit solved the above issue, but there are warning message is constantly being printed out during finetuning: First warning (printed once): > /home/attnmamba/anaconda3/envs/llava/lib/python3.10/site-packages/torch/utils/checkpoint.py:31: UserWarning: None...

@haotian-liu Of course, here is my bash script ```bash #!/bin/bash deepspeed llava/train/train_mem.py \ --lora_enable True --lora_r 128 --lora_alpha 256 --mm_projector_lr 2e-5 \ --bits 4 \ --deepspeed ./scripts/zero2.json \ --model_name_or_path liuhaotian/llava-v1.5-13b...

@haotian-liu Sorry false alarm, had some bad training examples in my dataset. No more tokenization mismatch warnings once those were removed. Thank you very much for your help!

@Lee-ray-a @chrisx599 Did you manage to solve this?