MSE not accepting (n_samples, n_ouptuts) despite docs stating so

[bauersimon]

The evaluation card for MSE states:

---

Mandatory inputs:

• predictions: numeric array-like of shape (n_samples,) or (n_samples, n_outputs), representing the estimated target values.
• references: numeric array-like of shape (n_samples,) or (n_samples, n_outputs), representing the ground truth (correct) target values.

---

So it should be usable similar to pytorch's MSELoss with multiple dimensions. But using it with (batch_size, multiple_outputs) doesn't work:

import evaluate
import numpy as np

m = evaluate.load('mse')
print(m.compute(predictions=np.random.rand(16,2), references=np.random.rand(16,2)))

⬇️

ValueError: Predictions and/or references don't match the expected format.
Expected format: {'predictions': Value(dtype='float32', id=None), 'references': Value(dtype='float32', id=None)},
Input predictions: ...
Input references: ...

---

Version: evaluate==0.4.1

[shenxiangzhuang]

Hi @bauersimon , the quick answer for your question is:

import evaluate
import numpy as np

- m = evaluate.load('mse')
+ m = evaluate.load('mse', 'multilist')
print(m.compute(predictions=np.random.rand(16,2), references=np.random.rand(16,2)))

And, the error message is also too vague to address the problem.

[bauersimon]

Thanks! Should probably be mentioned in the docs then 😅.