LLMs-Planning
LLMs-Planning copied to clipboard
karthikv792
An extensible benchmark for evaluating large language models on planning
I was trying to go through the code and wanted to add a new domain to evaluate my LLM on that domain. So my question was how have you generated...
`python3 llm_plan_pipeline.py --task TASK --config CONFIG --engine ENGINE [--ignore_existing] [--run_till_completion RUN-TILL-COMPLETION] [--specific_instances SPECIFIC-INSTANCES] [--random_example RANDOM-EXAMPLE] [--verbose VERBOSE] [--seed SEED]` Also, is there a default config.yaml example you might share? Thanks.
Hi, I fixed some error in the code (e.g. yaml instead of json, domain_name instead of domain and other things that just dont work othewise)
Metadata
Owner
Metadata
An extensible benchmark for evaluating large language models on planning