Max Berrendorf

The default is Hits@10. The documentation is hidden here: https://github.com/pykeen/pykeen/blob/7a87578f9bdb8fef28307bac1c037e0c177eee7a/src/pykeen/evaluation/rank_based_evaluator.py#L228-L240 @cthoyt maybe we can link this method to the documentation of the early stopper?

https://github.com/ibalazevic/multirelational-poincare/tree/master/data has some additional derived datasets.

Once the remaining models from https://github.com/pykeen/pykeen/projects/1 are upgrade to `ERModel`, all models will support this; so maybe it is time update the `{hpo_}pipeline` afterwards.

Hi @jas-ho / @MatthewCorney , sorry for the delay. This might not be exactly what you are looking for, but if you provide a path to the parameter `save_model_directory` of...

> I think we should continue blocking this for the OGB release Agreed. With the increased minimum version in `setup.cfg` (https://github.com/pykeen/pykeen/pull/948/commits/4f66f33db33cd09572e2097d72a69defedbc304a), all CI stages which require installation of the package...

> Do you already want to upgrade the PyKEEN code to use the internal OGB code and not duplicate then? there shouldn't be duplication of internal OGB code :thinking: The...

Would [sklearn.metrics.average_precision_score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.average_precision_score.html) + `SklearnEvaluator` suffice?

I have not benchmarked them. However, the rank based evaluator can run on GPU (for rank computation), and only the rank aggregation happens on CPU.

> I refuse to add a GO or Drugbank dataset directly to pykeen. There’s no way a set of computer science authors did anything scientifically meaningful with them. You mean...

@cthoyt before I continue, any feedback on the general direction?