panoptica icon indicating copy to clipboard operation
panoptica copied to clipboard

[FEATURE] Metric system Overhaul

Open Hendrik-code opened this issue 10 months ago • 0 comments

Currently, we do not support arguments for Metrics. And we distinguish as evaluator arguments into global and instance metrics

How about we overhaul the system to make it smoother:

  • Make Instance-Wise and Global versions of each metric
  • Then, we can just pass one Metric list into the evaluator (so that it would look something like metrics=[Metric.Global_IoU, Metric.Instance_IoU]
  • It would make it easier to define metrics that only make sense for one of the global/instance-wise versions.
  • And I propose to make Metrics have an initialization step, so you instantiate the metric objects you want to use, so we could add something like "Boundary_IoU(d=4)" so that metrics have parameters that are fixed for the evaluator. We can dynamically use those parameters for reporting the results.

This should also make it way easier and consistent for users, as well as more understandable.

Hendrik-code avatar Feb 18 '25 15:02 Hendrik-code