aoom comments

Repositories
Issues
Comments

Results 2 comments of


                                            aoom

关于A = square(relu(qk / seq_len + bias))

> 改成seqlen的输出： > > ```python > pytorch: 天气预报说今天的天[台+0.2037||的+0.0798||定+0.0446||好+0.0422||以+0.0386]很好，那么我[大+0.1093||的+0.0697||本+0.0629||以+0.0559||一+0.0518]一起去公园玩吧！ > ``` > > 使用max_length的输出： > > ```python > pytorch: 天气预报说今天的天[气+0.9948||空+0.0011||色+0.0007||候+0.0004||势+0.0003]很好，那么我[就+0.4915||们+0.4186||也+0.0753||还+0.0021||都+0.0016]一起去公园玩吧！ > ``` 谢谢分享，从结果上看max_length的效果更好！

Llama.generate: prefix-match hit is very slow.

Encountered a similar problem, which manifested itself as loading the model abnormally slow under gpu (unified memory for arm platforms) and only using a single core single thread for the...