how can i get validation loss value in eval_hooks.py and use it in Epoch-based Runner ?

[hoya-cho]

hello. I want to do early stopping based on the validation loss. I have seen the questions and answers through the following link. https://github.com/open-mmlab/mmcv/issues/1701

so you referenced

https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/evaluation/eval_hooks.py

I'm looking at the link

---

key_score = self.evaluate(runner, results)
        # the key_score may be `None` so it needs to skip the action to save
        # the best checkpoint
        if self.save_best and key_score:
            self._save_ckpt(runner, key_score)

---

This part seemed to be the part to get validation_loss and store the best weight.

The question is, is the validation loss expressed by key_score? And where else can I see the self.evaluate(runner, results) method? I am using custom self.evaluate()

My goal is to take the validation loss value and put it as a parameter in the early_stopping method implemented in epoch_based_runner.

[zhouzaida]

Hi @hoya-cho , you can calculate the loss in the method (https://github.com/open-mmlab/mmdetection/blob/3e2693151add9b5d6db99b944da020cba837266b/mmdet/core/evaluation/eval_hooks.py#L117) and add it to key_score.

[hoya-cho]

I know that the loss value is not returned in single_gpu_test or multi_gpu_test when in val mode. How can I calculate the losses by taking them..?

result = model(return_loss=False, rescale=True, **data)

If return_loss is False, doesn't it return loss? However, even if I change it to True, an error occurs.

[zhouzaida]

I know that the loss value is not returned in single_gpu_test or multi_gpu_test when in val mode. How can I calculate the losses by taking them..?

result = model(return_loss=False, rescale=True, **data)

If return_loss is False, doesn't it return loss? However, even if I change it to True, an error occurs.

If return_loss is False, it does not return loss. What error will be raised?

[hoya-cho]

Ah sorry, i misunderstood. What i was going for was get losses using return_losses = True whena val mode in " train after epoch " hooks.

But model pipeline is not train_mode, so occurs error that i dont have a value like gt_bbox, gt_labels.