Implement importing of custom metrics from python module in configuration file

[yurakuratov]

Want to contribute to DeepPavlov? Please read the contributing guideline first.

What problem are we trying to solve?: In DeepPavlov user can define its own DatasetReaders/DatasetIterators/Components and use them in configuration files. There are two ways:

1. use @register decorator in the source code
2. add "imports": ["my_code"] to metadata in configuration file, where my_code is a python module with custom components

But, when we want to define custom metrics the only available way is the first one with @register_metric!

How can we solve it?: Make imports of custom metrics and components in a consistent way.

Are there other issues that block this solution?: no that I am aware of

[blaze-fire]

Hello @yurakuratov, I have tried to understand the problem as follows (correct me if I am wrong): if we create a custom metric we, check whether the metric is already registered or not in metrics_registry.json, if it exists we replace the old metric function with the new one, otherwise the new metric gets registered. Now, what we want is to do the above using " imports:". ps: I am new to open source

[danielkornev]

Hello @yurakuratov, I have tried to understand the problem as follows (correct me if I am wrong): if we create a custom metric we, check whether the metric is already registered or not in metrics_registry.json, if it exists we replace the old metric function with the new one, otherwise the new metric gets registered. Now, what we want is to do the above using " imports:". ps: I am new to open source

hi @blaze-fire!

This repo is for main product issues. If you want to pick a GSoC idea, feel free to pick the one you find interesting in https://github.com/deepmipt/deeppavlov-contrib-drafts/issues

[oserikov]

@blaze-fire If you feel like this Idea can enhance you GSoC proposal or be your coding challenge, rise an Issue in our GSoC tracker describing how it fits into your GSoC contribution!

[blaze-fire]

@oserikov, I would like to raise it as an issue and also solve it as a coding challenge, but as I mentioned in my above comment, I can only do so if I understand the problem statement properly. Can you help me with that?

I am also working on other open issues

[IgnatovFedor]

Importing of custom metrics from python module in configuration file is implemented. Like shown in example, one can use metrics from already installed module or import own module with metadata.import and use metrics from there.