About Needle in a Haystack test

Open YUECHE77 opened this issue 9 months ago • 1 comments

Hi,

Could you share how many GPUs and for how long it will take to run the Needle in a Haystack test?

I saw someone says it requires 4*80G A100, and takes about/ less than 24 hours to finish. Is that true?

Thank you!

Apr 07 '25 05:04 YUECHE77

Also, could you please share the script used for running the Needle in a Haystack evaluation? Thanks!

Apr 08 '25 01:04 YUECHE77