Yemin Yu

Thanks for the quick reply! I'm aiming to evaluate the top-k exact-match metric (check if there's a ground-truth hit in top-k lowest-perplexity candidates generated using beam-search + num_return_sequences=k). In that...

same error

Found the issue: (For me)If you are generating your product_pattern.txt files for a new dataset, you need to set arg.extract_pattern = True and run extract_semi_template_pattern.py (Note there is a sys.exit()...