Dongjie Yang

Shanghai Jiao Tong University Shanghai, China CS in Shanghai Jiao Tong University

Results 2 comments of


                                            Dongjie Yang

[BUG] zero2 and zero3 has different behavior using the same hyperparameter to train a large model

This issue still exists in 0.14.0. Zero2 has a much larger grad norm than Zero3.

A new video-text dataset which may help

It is non-commercial and academic only.