Max Khanov

Have you ever seen a cup of coffee this happy?

Results 6 comments of


                                            Max Khanov

Issue with Recreating RAFT Llama-7b Lora Benchmarks

Thanks so much for the quick reply! I'm pretty sure we did SFT the model before we did RM. Our SFT train loss was about 1.539 and the test loss...

Issue with Recreating RAFT Llama-7b Lora Benchmarks

Also would it be possible to share the checkpoint files for the LLaMA-SFT-7B or the reward model?

Issue with Recreating RAFT Llama-7b Lora Benchmarks

@WeiXiongUST Any updates?

Issue with Recreating RAFT Llama-7b Lora Benchmarks

Thanks so much for following up @WeiXiongUST, we used Lora for all the steps (sft, and rm)

Issue with Recreating RAFT Llama-7b Lora Benchmarks

Also is Lora used during the SFT training?

Issue with Recreating RAFT Llama-7b Lora Benchmarks

Thanks so so much, we'll look into this!