Huang Haiduo

icon indicating an email [email protected]

icon company Xi'an Jiaotong University icon location XI'an icon location student of Xi'an Jiaotong University, @Xi'an Jiaotong University

Results 14 comments of Huang Haiduo

attention map visualization code

> @haiduo 当然，代码没有好好整理，不过应该可以直接跑。 https://drive.google.com/drive/folders/1okWf2noIrnDdOBNYUfl_iZe3LLjb_59n?usp=share_link > > @ysj9909 Very sorry about the late reply! The visualization code can be downloaded here: https://drive.google.com/drive/folders/1okWf2noIrnDdOBNYUfl_iZe3LLjb_59n?usp=share_link It is not well re-organized, but should work well....

No module named 'transformers.models.qwen_parallel.utils_qwen'

You can comment the two lines https://github.com/microsoft/LMOps/blob/daf972124f0699af18acee85473fece80fb405c2/minillm/tools/convert_mp.py#L11-L20 `11 and 20`

No module named 'transformers.models.qwen_parallel.utils_qwen'

> > You can comment the two lines > > https://github.com/microsoft/LMOps/blob/daf972124f0699af18acee85473fece80fb405c2/minillm/tools/convert_mp.py#L11-L20 > > > > `11 and 20` > > well, If I want to use Qwen model，where to find...

AB matrix initialization in layers.py does not conform to the description of the paper

> @edwardjhu can you please tell us why at least one of A or B has to be non-zero? May be the paper say that? Ensure that at the beginning...