scvi-tools icon indicating copy to clipboard operation
scvi-tools copied to clipboard

adding validation ELBO loss for pyro models

Open vitkl opened this issue 4 years ago • 2 comments

Would be great to have validation set ELBO loss for pyro models. As discussed with Adam here https://github.com/YosefLab/scvi-tools/pull/1059#discussion_r633935024 @adamgayoso I can implement this, but could you point me where this belongs?

vitkl avatar May 24 '21 11:05 vitkl

A related warning:

  /Users/vk7/anaconda3/envs/scvi-tools-dev/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py:69: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
    warnings.warn(*args, **kwargs)

vitkl avatar May 24 '21 11:05 vitkl

We need to add a function to this class

https://github.com/YosefLab/scvi-tools/blob/14ac97718c7d50470bf0db25aa7dd6d4f4a245c6/scvi/train/_trainingplans.py#L581

with this signature

https://github.com/YosefLab/scvi-tools/blob/14ac97718c7d50470bf0db25aa7dd6d4f4a245c6/scvi/train/_trainingplans.py#L517

that

  1. runs evaluate_loss and then self.log(..) it, or
  2. returns the loss evaluation then additionally write a method with signature

https://github.com/YosefLab/scvi-tools/blob/14ac97718c7d50470bf0db25aa7dd6d4f4a245c6/scvi/train/_trainingplans.py#L563

That sums over the accumulation of the loss. This would be analogous to what we discussed with averaging/summing during minibatching

adamgayoso avatar May 24 '21 15:05 adamgayoso