COMET

COMET copied to clipboard

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.2 to 3.1.3. Release notes Sourced from jinja2's releases. 3.1.3 This is a fix release for the 3.1.x feature branch. Fix for GHSA-h5c8-rqwp-cp95. You are affected if...

dependabot[bot]

[QUESTION] Train UnifiedMetric/XCOMET with word level predictions.

1

#### What is your question? I was trying to train a model both at sentence and word level, but was unable to find any examples of it. I believe XCOMET...

zolastro

Different scores from different COMET package versions 1.1.2 and 2.2.1

2

## 🐛 Bug When the same source, target, reference files are evaluated using the same wmt22-comet-da checkpoint, `unbabel-comet 2.2.1` under `python3.9` and `unbabel-comet 1.1.2` under `python3.7` gave me dramatically different...

PinzhenChen

## ❓ Questions and Help ### Before asking: 1. Search for similar [issues](https://github.com/Unbabel/COMET/issues). 3. Search the [docs](https://unbabel.github.io/COMET/html/index.html). #### What is your question? what is the precision of the model when...

haorannlp

Minimizing cpu RAM vs only use GPU RAM

1

## 🚀 Feature Load model directly on GPU when available instead of 1) CPU 2) GPU ### Motivation Trying to use comet-score with cometkiwi-xl on Colab Currently, the load_checkpoint method...

vince62s

Sparsemax not actually used in COMET-KIWI, XCOMET-XL/XXL

4

Hi, I have been playing around with re-implementing some of your models in Marian and while progressing through the code I noticed that you are not actually using sparsemax for...

emjotde

504 Server error when running comet-score using multiple machines

10

## 🐛 Bug Hi! A 504 server error is encountered when running multiple `comet-score` scripts. See below: `Traceback (most recent call last): File /home/stan1/anaconda3/envs/prefix_mt/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py, line 261, in hf_raise_for_status response.raise_for_status() File...

Smu-Tan

[QUESTION] Train Your Own Metric

2

#### What is your question? I am trying to train my own metric. However, I continue to run into some version of the same problem (Just with a different validation...

ecroxford

what aspects of language get considered while evaluating the translation by COMET. as far as I am aware of MQM metrics are covered in it. Interested in knowing other terminology/features of language gets in COMET. I am looking for metric along with COMET that can assess the translated content for cosmetic domain data. please suggest.

1

suyogkute

Segmentation error when tring to reproduce wmt22 results

1

## 🐛 Bug ### To Reproduce ``` pip install unbabel-comet comet-score -s ../mt-metrics-eval-v2/wmt22/sources/en-zh.txt -t ../mt-metrics-eval-v2/wmt22/system-outputs/en-zh/HuaweiTSC.txt -r ../mt-metrics-eval-v2/wmt22/references/en-zh.refA.txt > log.comet Global seed set to 1 Fetching 5 files: 100%|█████████████████████████| 5/5 [00:00

SefaZeng