COMET
COMET copied to clipboard
Unbabel
A Neural Framework for MT Evaluation
## ❓ Questions and Help #### What is your question? I have seen in the sample and documentation that the reference text is translation of the source text. Is that...
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.6. Release notes Sourced from jinja2's releases. 3.1.6 This is the Jinja 3.1.6 security release, which fixes security issues but does not otherwise change behavior...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Hi, I have some questions about the dataset provided here: https://github.com/Unbabel/COMET/tree/master/data 1) If I understand correctly, the training data for each year's WMT is accumulated from all previous WMTs. Does...
## ❓ Questions and Help I used Reference Free Model [Unbabel/wmt23-cometkiwi-da-xxl](https://huggingface.co/Unbabel/wmt23-cometkiwi-da-xxl) My source langauge is English, target language is Chinese. However there is some cases that the source(also target) text...
## ❓ Questions and Help #### What is your question? Hello, I conducted an experiment here: the ref and target were set to exactly the same text, and the Comet...
Hi, I find that COMET is trained on the z-score of DA. However, I am not sure about the implementation. **Is it rescaled on the translation direction level or something...
Hi @kepler https://github.com/Unbabel/COMET/blob/770c2900c93ad6ed30bd43227531325e76d9b019/explainable-metrics/explain_unite.py#L277 I got an error in this line while running comet explainable: `python explainable-metrics/explain_comet.py -m COMET/wmt22-comet-da/checkpoints/model.ckpt -t ende-2021-concat.csv --batch_size 8` **raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'") AttributeError:...
## 🚀 Feature Models artifacts on HuggingFace should adhere to a more conventional structure including serialized model binary and json config files ### Motivation This would allow to directly the...
## 🐛 Bug Hi! I am using Unbabel/XCOMET-XL for reference-based MT evaluation and observed a negative score. Is this expected? I am aware from [Issue #38](https://github.com/Unbabel/COMET/issues/38) that older COMET models...
## 🐛 Bug When using the COMET to evaluate sentence pairs, there are debug messages that cannot be suppressed: ### To Reproduce Here is the minimal verifiable example: ``` import...
