Dan Saattrup Nielsen
Dan Saattrup Nielsen
### Dataset name hogskoleprovet ### Dataset link https://www.hogskoleprovet.nu/gamla-hogskoleprov/ ### Dataset languages - [ ] Danish - [X] Swedish - [ ] Norwegian (Bokmål or Nynorsk) - [ ] Icelandic -...
### 🐛 Describe the bug Error message when running `scandeval -m 01-ai/Yi-6B-Chat -t named-entity-recognition -l da`: > 01-ai/Yi-6B-Chat could not be benchmarked on the truncated version of the Danish named...
Consider adding the IceSum Icelandic summarisation dataset. Maybe merging it with RRN, or adding it as a separate dataset: https://repository.clarin.is/repository/xmlui/handle/20.500.12537/285
The [DBRD dataset](https://huggingface.co/datasets/dbrd) is a Dutch sentiment classification dataset which has substantially better quality than the Dutch Social dataset, according to our Dutch colleagues in the TrustLLM project, so this...
Mistral has an API, where they host their closed source model Mistral-medium as well: https://console.mistral.ai/user/
The `vllm_models` is missing unit tests. There is currently a placeholder test script in `tests/test_vllm_models.py`.
The `protocols` is missing unit tests. There is currently a placeholder test script in `tests/test_protocols.py`.
The `model_cache` is missing unit tests. There is currently a placeholder test script in `tests/test_model_cache.py`.
The `openai_models` is missing unit tests. There is currently a placeholder test script in `tests/test_openai_models.py`.
The `model_config` is missing unit tests. There is currently a placeholder test script in `tests/test_model_config.py`.