Pier-Jean Malandrino comments

Repositories
Issues
Comments

Results 2 comments of


                                            Pier-Jean Malandrino

Batch Inference with Llama 3.2 Generate Function: Only the First Result is Correct

I need it too but it seems that it is not recommended for the moment, and still a WIP from my understanding .. Some similar issues : - [Hugging face...

How should I understand the `low_freq_factor` and `high_freq_factor` in `rope_theta`, and what impact will they have?

Hi, From my understanding, these parameters could allow to improve model context window size. They adjust rotation angle for high/low frenquencies components. I found usage of these here [model.py](https://github.com/meta-llama/llama-models/blob/main/models/llama3/reference_impl/model.py)