unitxt

unitxt copied to clipboard

lilacheden

for example when trying to create llm-as-judge using llama3 on hugging ace you will get the following error because you need to set an HF token ``` 403 Client Error....

OfirArviv

This would allow us an easier management and usage of templates and cards in 'produe'

OfirArviv

OfirArviv

Added new NER metric using fuzzywuzzy logic

3

Added a new metric using fuzzywuzzy logic for NER evaluation. This will help to compare the predictions with actual using string match based on predefined threshold.

sarathsgvr

Support disambiguation of scores from metrics that returns same score names

2

Today, If metrics, returns the same score (e.g. "f1") then the first metric score is returned, and the second metric is ignored. metrics ["metrics.bert_score.distilbert_base_uncased","metrics.bert_score.deberta_base_mnli"], It would be good to be...

yoavkatz

so the type checker knows that if we passed that function the type of the var is as mentioned. instanceof function have that effect.

OfirArviv

For example to support `List[Tuple[Literal["user", "assistant", "system"], str]] `

OfirArviv

For example, consider the empty format:  Which today is: format = SystemFormat( demo_format="{source}\\N{target_prefix}{target}\n\n", model_input_format="{system_prompt}\\N{instruction}\\N{demos}{source}\\N{target_prefix}", )

yoavkatz

Currently a specific template in unitxt have few versions for different languages. For example: English sentiment template: ```python template = InputOutputTemplate(input_format="Classify the sentiment of this text: {text}") ``` Deutch sentiment...

elronbandel