Any way to run without an OpenAI key?

[mohummedalee]

Hi, I have access to Llama through Huggingface and am trying to run FactScore through the CLI: python -m factscore.factscorer --input_path data/unlabeled/InstructGPT.jsonl --model_name retrieval+llama+npm

Shouldn't this use llama to do atomic fact generation? I am getting an error: AssertionError: Please place your OpenAI APT Key in api.key. Is there a way to use FactScore without an OpenAI key?

[martiansideofthemoon]

Hi @mohummedalee, OpenAI API is still needed for atomic fact generation. We welcome any contributions that replace it with LLAMA-based atomic fact generation.

[wenxueru]

@martiansideofthemoon Hi, I was wondering if you have any experience in replacing text davinci with open-source models for atomic fact generation? Will llama2-65b be a viable option? Thanks a lot🙏

[martiansideofthemoon]

Hi @Rosenberg37, a collaborator of @shmsw25 has seen it work well enough with the LLAMA-2 models. However adding them to the codebase for atomic fact extraction may need some non-trivial refactoring. We welcome any contributions here.