LLaMa 2 inference

[jubick1337]

I tried to benchmark LLaMa 2 chat model from HuggingFace and got a ValueError: ValueError: temperature(=0) has to be a strictly positive float, otherwise your next token scores will be invalid. It is caused by this line

[jindongwang]

Hi @jubick1337 , we are rewriting the code for PromptBench to support more models and make the code easier to use. Check the dev branch: https://github.com/microsoft/promptbench/tree/dev

[Immortalise]

Hi Jubick,

To address this issue, you can adjust the temperature setting to an extremely small value, such as temp=0.000001. This should be sufficient for the required adjustments :)

[github-actions[bot]]

Stale issue message