Mooler0410

How do you get the tree? Curious about the tool.

1

Hi! Nice work and the tree is really impressive ! Just curious about how do you plot the tree? Did you do it manually or use any tools? Thanks!

Thanks to @dwzhu-pku for pointing this out.

Unexpected low performance of mistral-v0.1

4

Hi! I tried to run BANKING77 with mistral-7b-v0.1. However, I got a nearly zero performance. I have no idea about how this could happen. Have you ever tested mistral-7b-v0.1? Or,...

After carefully reading your huggingface repo, by my understanding, the LongAlpaca model is directly tuned from LLama-2 within 1000 steps on the LongAlpaca dataset. Is that correct?

Prompt format for different models

2

Hi! I have read the codes for open source model evaluation. I noticed that, different from some existing benchmarks such as LongBench or L-Eval, there is not prompt customization part...