Wenliang Peng

Results 3 comments of Wenliang Peng

but I found both used "mitchell" init method. correct me if I am wrong ![image](https://github.com/allenai/OLMo/assets/38959389/d2dda3f5-7644-4994-aeb3-f403f2f2d128) ![image](https://github.com/allenai/OLMo/assets/38959389/b82bede0-8c95-4818-a504-eca12b0401a9)

@dumitrac thanks for your explanation, but I just find the ![image](https://github.com/allenai/OLMo/assets/38959389/da4350e2-ad1a-410e-a3f7-68d15ad398d1) with ![image](https://github.com/allenai/OLMo/assets/38959389/101939b5-a12c-4368-ad68-56aaf83cc504) and both for twin and non-twin version is false ![image](https://github.com/allenai/OLMo/assets/38959389/22180f28-369f-42e3-8861-b6ff896df6c6) so where can I find the right...

> di I cannot remember well, but how to set different value for "mitchell" initialization method. I think I found the same init method for both training setup