core-bioimage-io-python icon indicating copy to clipboard operation
core-bioimage-io-python copied to clipboard

How to set statistics for per_dataset processing?

Open constantinpape opened this issue 2 years ago • 2 comments

@FynnBe how do we currently deal with per_dataset normalization? I could not find this documented anywhere. And if I run prediction with a per_dataset model naively I get the following error:

/home/pape/Work/bioimageio/spec-bioimage-io/bioimageio/spec/shared/schema.py:45: ValidationWarning: weights:pytorch_state_dict: missing 'pytorch_version'
  warnings.warn(msg, category=ValidationWarning)
torch-em.git: 227kiB [00:00, 13.1MiB/s]
Traceback (most recent call last):
  File "/home/pape/Work/playground/bioimageio/processing/per_dataset.py", line 11, in <module>
    pred = pp(data)
  File "/home/pape/Work/bioimageio/core-bioimage-io-python/bioimageio/core/prediction_pipeline/_prediction_pipeline.py", line 100, in __call__
    return self.forward(*input_tensors)
  File "/home/pape/Work/bioimageio/core-bioimage-io-python/bioimageio/core/prediction_pipeline/_prediction_pipeline.py", line 128, in forward
    preprocessed, sample_stats = self._processing.apply_preprocessing(*input_tensors)
  File "/home/pape/Work/bioimageio/core-bioimage-io-python/bioimageio/core/prediction_pipeline/_combined_processing.py", line 105, in apply_preprocessing
    tensors[proc.tensor_name] = proc.apply(tensors[proc.tensor_name])
  File "/home/pape/Work/bioimageio/core-bioimage-io-python/bioimageio/core/prediction_pipeline/_processing.py", line 257, in apply
    mean = self.get_computed_dataset_statistics(self.tensor_name, Mean(axes))
  File "/home/pape/Work/bioimageio/core-bioimage-io-python/bioimageio/core/prediction_pipeline/_processing.py", line 59, in get_computed_dataset_statistics
    raise RuntimeError(f"Missing computed {measure} for {tensor_name} dataset.")
RuntimeError: Missing computed Mean(axes=('c', 'y', 'x')) for input0 dataset.

constantinpape avatar Feb 14 '22 19:02 constantinpape

@FynnBe any chance you could improve the error message and prepare an example on how to pass the statistics for the per_dataset mode?

constantinpape avatar Feb 24 '22 12:02 constantinpape

will be fixed by https://github.com/bioimage-io/core-bioimage-io-python/pull/241

FynnBe avatar Mar 11 '22 20:03 FynnBe