BlendingToolKit icon indicating copy to clipboard operation
BlendingToolKit copied to clipboard

Change the template of measure functions

Open atorchylo opened this issue 3 years ago • 2 comments

When I first encountered BTK measure function, I found it a little bit unintuitive because it required a very specific output, a dictionary with keys catalog, deblended_images, and segmentation, and I wasn't sure how to set the default parameters when passing a function to a MeasureGenerator. Currently, we can pass the parameter measure_kwargs to the MeasureGenerator, but it is unclear how this parameter works if we want to pass a list of different functions. The documentation says:

Each dictionnary is passed one time to each function, meaning that each function which be ran as many times as there are different dictionnaries.

This could be very unintuitive and confusing for the users.

I propose that we use the format of PyTorch Image transforms, which uses classes to implement similar functionality. In this format, every measure function will be a class, with a __call__ method that implements what the function does, and __init__ that sets default parameters. So, in this format, we will rewrite the measure function into:

class Measure_smth()
   def __init__(self, default1=True, default2=True, default3=0.5):
      self.default1=default1
      self.default2=default2
      self.default3=default3

   def __call__(self, batch, idx):
      # do stuff
      return output_dict

So we can make an instance of this class and pass it to the MeasureGenerator as:

btk.measure.MeasureGenerator([Measure_smth(default1=False), Measure_smth(default1=True)], draw_blend_generator)

Also, we can write a small parent class for measure functions that would validate measure functions written by users, and give warnings if the user implemented the function correctly. In any case, this is just an idea, and I am happy to discuss it further!

atorchylo avatar Jul 27 '22 18:07 atorchylo

I like this idea, what do you think @thuiop ?

ismael-mendoza avatar Jul 28 '22 19:07 ismael-mendoza

This basically what we do for sampling functions anyway so I guess it would make sense ; however the idea behind the measure_kwargs was more about being able to run the same measure function with a whole array of parameters ; for instance for tuning a threshold. It was also meant to make the AUC metric work, although it has continuously been clunky. I agree that it is not super intuitive currently. I will try to think of a better way.

thuiop avatar Jul 29 '22 16:07 thuiop

Implemented in #412

ismael-mendoza avatar Aug 13 '23 18:08 ismael-mendoza