open-metric-learning
open-metric-learning copied to clipboard
OML-Team
Log metrics as bar charts
We may compute metrics on different categories, see docs.
The problem appears when we log and plot this metrics. They are shown on different plots, so it's hard to compare different categories. It would be great to see those metrics as bar chars for easier side-by-side comparison.
Let's update a categories example so we have a function that creates a bar chart (as an image in np format) based on provided metrics dictionary, so we can visualize it.
After it's done, we can add this functionality to Pipelines Logging.
Hi @AlekseySh
This is my first time contributing to open source. I wanted to work on this. Can you give me a brief on what needs to be done?
@Rei-Moriaty Yes, sure. If you run one of the examples (for instance, training model on INSHOP dataset) you will see a lot of graphs with metrics by different categories on Neptune. The problem now is that it's hard to find the best/worst category based on these graphs. So, if we can log them as barplot images, it would be easier to navigate, where the x-axis is the category and the y-axis is the metric. And we need a series of these images with respect to epoch number.
PS. Probably Neptune provides a way of working with bar plots, so we can avoid using images.
Some random image from the internet:
@AlekseySh im happy to work on this if its still an active problem
but before I started was playing around on colab and ran into a problem with adding from oml.lightning.entrypoints.train import pl_train I get ModuleNotFoundError
I checked out #146 and was able to run visualization.ipynb just fine am i missing a step for oml.lightning.entrypoints.train or is that no accessible via pip install git+
@isehmi Hi, it would be fine, because it's still an active problem
As for imports, ideally, all of the have to be accessible in collab env
- Do other imports work in collab?
- Can you do this import locally without collab?
@AlekseySh sorry for taking so long to get back to you
when i install locally i have the same problem when i just use pip install -U open-metric-learning but i found that installing pytorch and pytorch lightning works. if this is a step that the users needs to do i recommend its added somewhere in the installation guide.
i still can't get it to work on google colab even with installing the extra modules
note this problem only exist with lightning modules from what i can tell everything else works without installing those modules
@isehmi Hm, after what you said, I created an empty docker container with python and successfully installed the library from PIP. Which OS do you have?
@isehmi Got it. I think this is a problem. We have not tested the installation of our library for windows. Only unix-like systems. As for lightning and colab, yes, it's a known issue. That is why I added colab examples without the usage of lightning. We will investigate later how to make work lightning in colab.
I opened the dedicated issue regarding this: https://github.com/OML-Team/open-metric-learning/issues/222
@isehmi But I guess, it's not a blocker for you If you want to work on this issue. Because you will install the repo via git clone from your fork, not from the PIP index. I hope, in this setup, you will be able to contribute. But yep, we need to figure out what is wrong with Windows.
@AlekseySh as I'm still working on windows I found another small bug that effects windows users should i be making issue for these or will this all be done when testing on windows systems starts
@isehmi I think after we start testing on windows we find more, so, the creating one particular issue is not needed now
thats what i was thinking a bit pointless adding issue one at a time. when the time comes we will need to build a robust system to catch all possible issue. i will still keep track of any problems i encounter as it might come in handy