unitxt
unitxt copied to clipboard
IBM
🦄 Unitxt: a python library for getting data fired up and set for training and evaluation
Hi @michal-jacovi and @elronbandel , Just to kickoff, I tweaked test_card, making it invoke load_dataset_builder (rather than LoadHF), and printed the description, citation, homepage, and whatever that load_dataset_builder harvested for...
Added modifications to loaders and the `RemoteMetric` class so that a metric can check if it's allowed to handle data which is sent to a remote service as per #646
The new feature of Metric, sample-from-groups-scores, employs the CI over instances generated ad-hoc, one per group (a group is a subset of the input instances, whose member instances are those...
Suggest a scheme of "aggregator" to be similarly employed by all three types of MetricsWithConfidenceInterval.
Hi @elronbandel , I had this in mind re the issue you assigned to me. Is this in the direction you had in mind? If yes, I will clean the...
If all the base operators had eager mode it would be easier to debug. This addition can be simple: adding to every operator an option to run on all the...
Fixed YesNoTemplate used to assume 'class' field was a list. Diverse LabelSampler, assumed the choices are a list. However, in binary classification, the "choices" field is "class", which is a...
Currently in unitxt there are wrong names and compositions for few importatnt core components in unitxt ## Main Classes * `TaskCard` -> `DataToTask` or `DataToTaskCard` `preprocess_steps` -> `data_to_task_steps` * `FormTask`...