Bo Li

llama3-llava-next-8b 全参微调报错

3

### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info llamafactory 已经更新至最新版本 - `llamafactory` version: 0.9.2.dev0 - Platform: Linux-5.10.0-1.0.0.32-x86_64-with-glibc2.17 - Python version:...

llava-llama3-8b 微调过程中 loss nan

2

微调llava-llama3-8b的时候 从几个step后就开始loss=nan了 这个可能是什么原因呢？我看github issue也有人遇到类似问题 官方回复是改lr 我现在设置的 ``` # Scheduler & Optimizer batch_size = 4 # per_device accumulative_counts = 32*4 dataloader_num_workers = 32 max_epochs = 1 optim_type = AdamW lr =...