Hao Mi

icon indicating a website link https://scholar.google.com/citations?user=RxULDkAAAAAJ&hl=en icon indicating an email [email protected]

icon company Media Synthesis and Forensics Lab, ICT-CAS icon location Beijing, China icon location MS @ ICT-CAS

Results 2 issues of Hao Mi

How to get the Activation cache while the LLM is generating new tokens?

2 comment

### Question My prompt is "Which instrument does Henry Halstead mainly play? Please answer an instrument name. Answer: ", which is a question to LLM. I want to get the...

complexity-moderate

About the cached layernorm scale factors

2 comment

### Question About the 'apply_ln_to_stack' function in 'ActivationCache.py' file, what's the meaning about the following sentence？ "The layernorm scale is global across the entire residual stream for each layer, batch...