Yoav Katz

Looking at running 100 example of Rouge, we see the metric being processed 101 times as expected (1 for each example, and then all the whole 100). This lead to...

In the #1011 , we fix Rouge and Metor by moving away from global metrics when it's not need. For f1 for example, it may be important to use global...

In anycase, using a different CI method, should be something we consider to user automatically (users can not be involved with this decision).

Since we take the target_prefix from the main instance and not the demos, if the instance is verbalized the calculation of 1+2 and the demo need to verbalize 4+5, it...

You are right, In the past, unitxt did not support the "instruction" field and these are older templates. There are templates that use the instruction field like "templates.qa.multiple_choice.with_context.with_topic.title" BTW, it's...

I think it should be called ApplyRemoteCode - and have as similar API to Apply. ApplyRemoteCode("a", function="koko", to_field="b", source_urls = ['https:... '], source_urls_content_hash="XXXXX")

Yes. Type is currently a reserved name. Maybe we need to change it to __ type __ ( @elronbandel )?

Also, I think it's best that from the users perspective all operators will be in a single import file (and not collection_operators, dialog_operators). This could be done by adding an...

How could it work in practice? Would the card not have templates? Or both the card and task will have templates? Would it be specified via the template param or...

There seems to be a cache of artififacts: @classmethod def get_artifact(cls, artifact_identifier: str) -> Artifact: if artifact_identifier not in cls.cache: artifact, artifactory = fetch_artifact(artifact_identifier) cls.cache[artifact_identifier] = artifact return cls.cache[artifact_identifier] It...