unitxt

unitxt copied to clipboard

When using HF APIs only the HF version code and catalog will be used

yoavkatz

Different rouge scores in 1.11.1 and main

1

Following the Rouge updates , there are changes in rouge scores. In most cases, the diff is less than 1 point, , but it could be 1-2 point in extreme...

yoavkatz

Right now , metrics score overwrite each other (Not referring to the 'score' and 'score_name' , but two metrics can have the same 'f1' score. Users find it confusing. We...

yoavkatz

Currently, the full clapNQ dataset contains thousands of documents. It is not usable for testing a simple rag end-to-end flow. we would like to create a subset version of these...

benjaminsznajder

elronbandel

elronbandel

Add end-to-end RAG example

1

I created a branch: and PR in draft state - https://github.com/IBM/unitxt/pull/953 * The task is there, but it fails to run because rouge is missing. I believe the metric is...

assaftibm

Also, we don't see the print out to see what takes the time. https://github.com/IBM/unitxt/actions/runs/10042751612/job/27753878493?pr=1034

yoavkatz

currently we pass data_classification_policy in 2 places in the recipe As a field and in the task_data. We need to think if and how to consolidate them. for example: ```...

OfirArviv

assaftibm