Zhang Kehao

icon indicating a website link https://www.ucas.ac.cn icon indicating an email [email protected]

icon company University of Chinese Academy of Sciences icon location University of Chinese Academy of Sciences icon location I am Zhang Kehao, a Ph.D. student in University of Chinese Academy of Sciences. I want to study in Github and try to make some contributions to the community.

Results 2 comments of Zhang Kehao

An error in .\src\models\MIME\VAE_noEmo_posterior.py line 98

Also for line 75

Sink Cache Attention Scores are strange. CausalMask seems not working.

In cache_utils.py, I noticed that ` keys_to_keep = self.key_cache[layer_idx][ :, :, -self.window_length + self.num_sink_tokens + key_states.shape[-2] : ] ` might go wrong when -self.window_length + self.num_sink_tokens + key_states.shape[-2] >= 0...