YifeiWang issues

Repositories
Issues
Comments

Results 2 issues of


                                            YifeiWang

[Bug Report] RMSNormPre in Transformer_lens is maybe different from Llama source code?

In LlamaModeling.py, the LlamaRMSNorm function outputs the weights * scaled hidden_states like below ![image](https://github.com/TransformerLensOrg/TransformerLens/assets/82020401/2bc96528-be6b-4383-b245-4ca963a3f754) **RMSNormPre definition in Transformer_lens: it seems that this function just outputs the scaled hidden_states** ![image](https://github.com/TransformerLensOrg/TransformerLens/assets/82020401/0188f4b6-fcdd-441b-ad7c-b4b0ec0165b3) **The...

complexity-moderate

needs-investigation

where to download the dataset of COCO 2017 dataset ?

I download COCO 2017 dataset from huggingface (a repo named phiyodr/coco2017) and the structures of data is different . I wonder where the coco 2027 is downloaded?