Yu-won Lee
Yu-won Lee
Okay I'll give it a try. Thanks for the comment.
I don't know what exactly is a gme version. Also, what probelm did you encounter?
Yes, vllm and sglang could be a good one to speed up the inference. Also you could use static_cache and compile for it, but you should use a fixed batch...
@yangfy2023 You could make an Dataset class similar to training dataset. or You could just make an pipeline for it. It's not so difficult.
You could directly adjust in he training_args in the `train.py`.
Sorry for the inconvinience. It could be a little confused. If the `--vision_lora` is not set to true, then the code automatically adds the keyword `visaul` in the list (`merger`...
It looks like 8bit has this problem but the answer is not that useful. https://github.com/bitsandbytes-foundation/bitsandbytes/issues/454#issuecomment-1636964951 I'll find some other way for this. Thanks for letting me know.
It seems like this is occured when I've updated the library versions. It may work when downgrading some of the libraries but the code won't work well. I'll find some...
No it dosen't load all datas at once. The log for qwen-vl-utils would show once in the first step. The memory warning is literally a warning that it cause oom...
Sorry I haven't tried multi-node that I have only one machine. So, I couldn't try to solve the problem. Sorry again for the inconvinience.