Yuncong Chen comments

Repositories
Issues
Comments

Results 3 comments of


                                            Yuncong Chen

[WIP] Gemma3 support.

For gemma-3-27b-it, I could not get the logits of this implementation to match the HF version, so I begin to track down where they deviate. This leads me to a...

Any plans to support DPO training?

context parallelism is one of the major features missing in torchtune.

device_map='auto' coupled with tp_plan='auto'

I'm still getting an error related to this. Please see https://github.com/pytorch/pytorch/issues/155463#issuecomment-3060504993. > I still got this error `AssertionError: found no DeviceMesh from dtensor args for c10d.broadcast_.default!` if I use `tp_plan='auto'`...