openai
assert ( AssertionError: Eval match_mmlu_anatomy not found. Available: ['coqa-closedqa',
hi, The following message appears when running " !oaieval gpt-3.5-turbo match_mmlu_anatomy", and "/tmp/evallogs" directory cannot generated.
[2023-03-21 10:09:35,502] [registry.py:145] Loading registry from /home/coder/.local/lib/python3.9/site-packages/evals/registry/evals
[2023-03-21 10:09:35,533] [registry.py:145] Loading registry from /home/coder/.evals/evals
Traceback (most recent call last):
File "/home/coder/.local/bin/oaieval", line 8, in
How should I fix it? thank you !
Ja się ma tym kompletnie nie znam
wt., 21 mar 2023, 11:35 użytkownik 123456xxn @.***> napisał:
hi, The following message appears when running " !oaieval gpt-3.5-turbo match_mmlu_anatomy", and "/tmp/evallogs" directory cannot generated.
[2023-03-21 10:09:35,502] [registry.py:145] Loading registry from /home/coder/.local/lib/python3.9/site-packages/evals/registry/evals [2023-03-21 10:09:35,533] [registry.py:145] Loading registry from /home/coder/.evals/evals Traceback (most recent call last): File "/home/coder/.local/bin/oaieval", line 8, in sys.exit(main()) File "/home/coder/.local/lib/python3.9/site-packages/evals/cli/oaieval.py", line 225, in main run(args) File "/home/coder/.local/lib/python3.9/site-packages/evals/cli/oaieval.py", line 124, in run assert ( AssertionError: Eval match_mmlu_anatomy not found. Available: ['coqa-closedqa', 'coqa-closedqa.dev.v0', 'coqa-fact', 'coqa-fact-expl', 'coqa-fact-expl.dev.v0', 'coqa-fact.dev.v0', 'coqa-match', 'coqa.match.dev.v0', 'diversity', 'diversity.dev.v0', 'joke-animals', 'joke-animals-likert', 'joke-animals-likert.dev.v0', 'joke-animals-vs-fruits', 'joke-animals-vs-fruits.dev.v0', 'joke-animals.dev.v0', 'joke-fruits', 'joke-fruits-ans-meta', 'joke-fruits-ans-meta.dev.v0', 'joke-fruits-expl-meta', 'joke-fruits-expl-meta.dev.v0', 'joke-fruits-meta', 'joke-fruits-meta.dev.v0', 'joke-fruits.dev.v0', 'logic-fact', 'logic-fact.dev.v0', 'rap-animals-vs-fruits', 'rap-animals-vs-fruits.dev.v0', 'rap-people-vs-fruits', 'rap-people-vs-fruits.dev.v0', 'rap-people-vs-people', 'rap-people-vs-people.dev.v0', 'test-fuzzy-match', 'test-fuzzy-match.s1.simple-v0', 'test-includes', 'test-includes.s1.simple-v0', 'test-match', 'test-match.s1.simple-v0']
How should I fix it? thank you !
— Reply to this email directly, view it on GitHub https://github.com/openai/evals/issues/378, or unsubscribe https://github.com/notifications/unsubscribe-auth/A6PJHHTVY3VB66KPF5U5PYLW5GAAJANCNFSM6AAAAAAWCHU27M . You are receiving this because you are subscribed to this thread.Message ID: @.***>
@123456xxn You have to run pip install .. See https://github.com/pisrcio/evals-mahjong/blob/main/docs/custom-eval.md
As mentioned by @zxhmike, pip install should fix it. You can also use pip install -e . to install in editable so that you don't need to rerun pip install after modifying your local repo.
