Demo pool is returned as a stream and goes through demo verbalization.

[yoavkatz]

Example:

dataset = load_dataset( card="cards.wnli", template="templates.classification.multi_class.relation.default", num_demos=2, demos_pool_size=10, loader_limit=20, ) print(dataset) print(dataset["demos_pool"][0]['source'])

DatasetDict({ validation: Dataset({ features: [‘metrics’, ‘data_classification_policy’, ‘target’, ‘references’, ‘postprocessors’, ‘source’, ‘task_data’, ‘group’], num_rows: 3 }) test: Dataset({ features: [‘metrics’, ‘data_classification_policy’, ‘target’, ‘references’, ‘postprocessors’, ‘source’, ‘task_data’, ‘group’], num_rows: 20 }) demos_pool: Dataset({ features: [‘metrics’, ‘data_classification_policy’, ‘target’, ‘references’, ‘postprocessors’, ‘source’, ‘task_data’, ‘group’], num_rows: 10. <--- The demo pools is returned }) train: Dataset({ features: [‘metrics’, ‘data_classification_policy’, ‘target’, ‘references’, ‘postprocessors’, ‘source’, ‘task_data’, ‘group’], num_rows: 7 }) })

Each demo, in itself is verbalized, with its own demos. I’m not sure it’s needed:

---

Given a premise and hypothesis classify the entailment of the hypothesis to one of entailment, not entailment. premise: Steve follows Fred’s example in everything. He influences him hugely. hypothesis: Steve influences him hugely. The entailment class is entailment premise: George got free tickets to the play, but he gave them to Eric, because he was particularly eager to see it. hypothesis: George was particularly eager to see it. The entailment class is entailment premise: I stuck a pin through a carrot. When I pulled the pin out, it had a hole. hypothesis: The carrot had a hole. The entailment class is