Max Berrendorf

One more thing I noticed: https://pykeen.readthedocs.io/en/stable/api/pykeen.training.callbacks.EvaluationLoopTrainingCallback.html also needs the factory on which to evaluate, i.e., ```python config["pipeline"]["training_kwargs"]["callback_kwargs"] = { "prefix": "validation", "factory": dataset.validation, "additional_filter_triples": dataset.training, } ```

Okay, this seems to be a bug in `EvaluationLoop`, which does not properly forward this argument to instantiate the `LCWAEvaluationDataset` [here](https://github.com/pykeen/pykeen/blob/ebb5b52b36dec7194c526b68094f82774b058ffc/src/pykeen/training/callbacks.py#L314-L318). I used this smaller snippet to reproduce your error...

Hm, so we would need to re-run the collation every time we want to have the plots for a new metric?

I encountered the same issue for https://github.com/pykeen/pykeen/actions/runs/9667143013/job/26668595669?pr=1405 Raw Log ``` 2024-06-25T17:50:31.5756234Z Current runner version: '2.317.0' 2024-06-25T17:50:31.5780240Z ##[group]Operating System 2024-06-25T17:50:31.5780876Z Ubuntu 2024-06-25T17:50:31.5781216Z 22.04.4 2024-06-25T17:50:31.5781643Z LTS 2024-06-25T17:50:31.5781988Z ##[endgroup] 2024-06-25T17:50:31.5782387Z ##[group]Runner Image 2024-06-25T17:50:31.5782859Z...

I think the current `SampledRankBasedEvaluator` is actually doing an sLCWA style evalation in its `evaluate`!

Hi @g-yit , this is unfortunately to be expected since ConvE's interaction function is designed to allow fast 1:n tail scoring while being quite inefficient for head scoring. In the...

`ConvKB` is also very slow in 1:n scoring (even worse than `ConvE` where at least one direction is fast). The default evaluation protocol is to use 1:n scoring. You could...

> The training results differ significantly from those reported in the paper. Did you first train a TransE model and use its weights to initialize the ConvKB ones? That one...

> When I trained using conve and set create_inverse_triples to False, I can understand that the evaluation would be slow, but it is only 2.83 triples/s. For each triple it...

btw, from you method name `replicate_conve` it looks like you are trying to reproduce ConvE? If so, you may want to take a look at https://github.com/pykeen/pykeen/blob/master/src/pykeen/experiments/conve/dettmers2018_conve_fb15k237.json. In particular, reproducing the...