Metrics in multiple categories

[rasbt]

📚 Documentation

Okay, this is a bit of a tricky one. I received some feedback about the organization:

For example in categories for module metrics like Audio, Classification, Image, Detection ..., some of the metrics could be included in multiple categories, for example Average Precision.

I think we have it organized that each metric is currently only in one category (audio, image, etc.). I think this is useful for not bloating the page. At the same time, if there is someone looking for metrics for e.g., an image context, they might miss some relevant ones because those are general metrics.

I don't know if we can/should do something about this.

[ethanwharris]

One option here (at least in the docs, but import paths would still be limited) would be to use the tag filtering view we have in PL / Flash (e.g. https://lightning-flash.readthedocs.io/en/stable/?badge=stable).

This could allow for metrics to have potentially multiple tags for each dtype they are relevant for.

[rasbt]

Yeah, that's actually pretty good. Maybe we can have a list of all metrics in alphabetical order by default, and then via the tags, we can narrow it down to the tag-specific ones

[SkafteNicki]

I do like the idea of tagging each metric. However, one potential problem is that we currently have some subsections like this one: https://torchmetrics.readthedocs.io/en/latest/references/modules.html#input-details that basically explains what we expect the input to look like for those specific metrics.

[Borda]

just for the record, I think that the docs deserve some more care and refactoring as we are still using structure from the time we had just 15 metrics, and now we have more the 60... so I like tagging idea, and/but I would personally join it with some larger docs revamp... :rabbit:

[ethanwharris]

@SkafteNicki I guess you would break it down by metric. So you can filter all metrics by data-type / task (e.g. find all metrics that can be used for image classification). Then in each metrics page it could explain the input format it requires

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.