vak icon indicating copy to clipboard operation
vak copied to clipboard

ENH: Convert post-processing to Transform classes that can be specified in config

Open NickleDave opened this issue 3 years ago • 1 comments

e.g. MajorityVote

[vak.model.TweetyNet]

[vak.model.TweetyNet.output-transform.MajorityVote]
[vak.model.TweetyNet.output_transform]
MinSegmentDur = {min_dur = 2}

:thinking: not sure if this is valid TOML, but ... something like this

NickleDave avatar Jul 08 '22 03:07 NickleDave

I think something like this will be necessary to achieve #472 without adding a ton of very specific logic to the _eval function

Basically the _eval function should not have to know anything about the inputs and outputs of the model, and how metrics get computed from those. It should just be something like:

def _eval():
    self.metrics.reinit()
    for batch in loader:
        output = self.model(input)
        self.metrics.add(output)
    return self.metrics.vals

One way to achieve this would be to have Networks return an Output class with attributes, and to have metrics look for those attributes. But that starts to also feel very constrained, i.e. we might regret what we chose for the expected attributes down the road

NickleDave avatar Jul 08 '22 03:07 NickleDave