evaluate icon indicating copy to clipboard operation
evaluate copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Add text2text evaluator

Open lvwerra opened this issue 2 years ago • 2 comments

This PR adds three evaluators following the pipeline philosophy:

  • text2text
  • translation
  • summarization

While they are functionality similar they use slightly different defaults. The parity tests for the Trainer are missing but we can add those in a separate PR I think.

(for some reason I can't add you as reviewer @fxmarty but I think you can comment anyway)

cc @philschmid

lvwerra avatar Aug 18 '22 17:08 lvwerra

The documentation is not available anymore as the PR was closed or merged.

HuggingFaceDocBuilderDev avatar Aug 18 '22 17:08 HuggingFaceDocBuilderDev

That's great! We should include the new subclasses in https://github.com/huggingface/evaluate/blob/main/docs/source/package_reference/evaluator_classes.mdx I think, and probably mention them in the how-to guide.

fxmarty avatar Aug 19 '22 13:08 fxmarty

Thanks for your feedback, I think I integrated everything. Let me know if you think anything else needs to be changed!

lvwerra avatar Oct 10 '22 10:10 lvwerra