Bailey Kuehl comments

Repositories
Issues
Comments

Results 42 comments of


                                            Bailey Kuehl

Tensor Parallelism with the current qk norm.

Hi there, thanks for the question! Our 7B model was trained on our old trainer, which unfortunately doesn't have an option for tensor parallelism. Our new trainer has this option,...

Add MuonW optimizer: Muon with AdamW fallback for non-matrix parameters

Hi there, thanks for your contribution and interest! We apologize for the delay in response to your PR - we are indeed at a busy time of year. We will...