pykeen
pykeen copied to clipboard
pykeen
😮🪑 Add OGB Evaluator
Add an evaluator to call OGB evaluation from within the pipeline.
Currently blocked by Evaluator.evaluate receiving multiple kwargs (including additional_filter_triples) to be passed to evaluate, which are not supported by the OGB evaluate method.
Dependencies
- [x] #1088
I think we should continue blocking this for the OGB release
Agreed. With the increased minimum version in setup.cfg (https://github.com/pykeen/pykeen/pull/948/commits/4f66f33db33cd09572e2097d72a69defedbc304a), all CI stages which require installation of the package will fail anyway, until the new version becomes available through pypi.
Do you already want to upgrade the PyKEEN code to use the internal OGB code and not duplicate then?
Do you already want to upgrade the PyKEEN code to use the internal OGB code and not duplicate then?
there shouldn't be duplication of internal OGB code :thinking:
The evaluator mostly wraps the existing implementation and prepares the input in a suitable format / processes the output to PyKEEN's format.
Currently blocked by Evaluator.evaluate receiving multiple kwargs (including additional_filter_triples) to be passed to evaluate, which are not supported by the OGB evaluate method.
@mberr is this still true?
Currently blocked by Evaluator.evaluate receiving multiple kwargs (including additional_filter_triples) to be passed to evaluate, which are not supported by the OGB evaluate method.
@mberr is this still true?
Yes, this leads to a failing tests/test_evaluation/test_evaluators.py::OGBEvaluatorTests::test_pipeline test.
Update: the error is also related to https://github.com/pykeen/pykeen/pull/1214 (but not solved by it).