chameleon-llm icon indicating copy to clipboard operation
chameleon-llm copied to clipboard

Published 20 hours ago • verified publisher icon lupantech

What this mean?

Open ogoflowgo opened this issue 2 years ago • 3 comments

../results/scienceqa/chameleon_chatgpt_minitest.json Result file exists: ../results/scienceqa/chameleon_chatgpt_minitest.json Count: 100, Correct: 44, Wrong: 56

ogoflowgo avatar Apr 24 '23 10:04 ogoflowgo

Hello,

It means that the experiment with identical arguments has already been run, and the results can be found in the file located at ../results/scienceqa/chameleon_chatgpt_minitest.json.

The script subsequently displays the following statistics:

  • Count: 100: A total of 100 examples were processed.
  • Correct: 44: Out of these, 44 examples had accurate predictions.
  • Wrong: 56: Conversely, 56 examples had incorrect predictions.

lupantech avatar Apr 24 '23 19:04 lupantech

soooo.....that's what your run.py does. cool. You should write (or get Chameleon LLM to write) a tutorial on how to get this thing running. dotenv etc. so we can do something special with it. Write it for beginners like me. I had to get gpt-4 to tell me how to put in all the dotenv stuff because i know nothing... Maybe you're not interested, and believe newbs can look elsewhere, but it would be cool as your project looks promising. Thanks for making it.

ogoflowgo avatar Apr 24 '23 23:04 ogoflowgo

Hi @ogoflowgo , I was wondering where did you find this file: ../results/scienceqa/chameleon_chatgpt_minitest.json?

jameszhou-gl avatar Oct 08 '23 06:10 jameszhou-gl