SJTU-IPADS
Question about the perplexity
I am Reproducing the results of perplexity. And I get confused about the following comment lines in examples/perplexity/perplexity.cpp:
// Download: https://s3.amazonaws.com/research.metamind.io/wikitext/wikitext-2-raw-v1.zip?ref=salesforce-research
// Run ./perplexity -m models/7B/ggml-model-q4_0.bin -f wiki.test.raw
// Output: perplexity: 13.5106 [114/114]
First, I can't open the website mentioned above, so I download the wikitext-2-raw-v1 from the hugginface. Second, I didn't find the "models/7B/ggml-model-q4_0.bin",so I use ReluLLaMA-7B-PowerInfer-GGUF/llama-7b-relu.powerinfer.gguf instead, which I also download from hugginface SpareLLM Thrid, I didn't get the perplexity 13.5106. I got the crazy "Final estimate: PPL = 1241.7730 +/- 8.93354" , which makes no sense at all. I really appreciate your reply and tips. 3Q for the extraordinary work again. By the way, i also want to reproduce the accuracy result using the 4 SuperGlue dataset. But I have no idea how to make use of the dataset. Can u provide some scripts or demo, etc?
