Is the validation loss computed in the mmsegmentation 1.2.2? If so, how can i plot it?

[HUANGL1NJIE]

Hello, I am using mmseg version 1.2.2.

I sincerely wonder how to get the validation loss and the accuracy score in the training step.

I found the val_step function was not in the base.py in this version. I am not sure whether the computation of validation loss is

moved to other modules.

Thanks for any help.

[AI-Tianlong]

https://github.com/open-mmlab/mmsegmentation/blob/main/docs/en/advanced_guides/evaluation.md#ioumetric Is this useful to you?

[CastleDream]

看这个文档 https://github.com/open-mmlab/mmsegmentation/blob/c685fe6767c4cadf6b051983ca6208f1b9d1ccb8/docs/zh_cn/advanced_guides/data_flow.md

上图所示数据流仅适用于当用户没有自定义 Runner 中的 TrainLoop、ValLoop 和 TestLoop，并且没有在其自定义模型中覆写 train_step、val_step 和 test_step 方法时。MMSegmentation 中 loop 的默认设置如下：使用IterBasedTrainLoop 训练模型，共计 20000 次迭代，并且在每 2000 次迭代后进行一次验证 ......

蓝色线表示 val_step 和 test_step。这两个过程的数据流除了模型输出与 train_step 不同外，其余均和 train_step 类似。由于在评估时模型参数会被冻结，因此模型的输出将被传递给 Evaluator。 来计算指标。

也就是说，val的时候不会输出loss，会直接调用Evaluator计算iou等指标

[eighteenlin]

Hello, I am also using mmseg version 1.2.2. I sincerely wonder how to get the validation loss and the accuracy score in the training step. I also encountered this problem, how did you solve it? Thanks for any help.

[syf66666666]

你好，这个问题解决了吗，我也想知道如何获得val loss

[CastleDream]

you can look this flow image in document:

and when you in test and val stage, you just evaluator model, don't receive loss, and can't get val loss

if you want to get val loss, maybe you can see this issue: Logging validation loss without library code hacks #1396

[HUANGL1NJIE]

https://github.com/open-mmlab/mmsegmentation/blob/main/docs/en/advanced_guides/evaluation.md#ioumetric Is this useful to you?

Sorry for the late reply. The tutorial from the link actually is more related to the evalution metric calculation. Maybe this comment https://github.com/open-mmlab/mmsegmentation/issues/3494#issuecomment-2136444244 understands my request. Still, sincerely thanks for your comment.

[HUANGL1NJIE]

你好，这个问题解决了吗，我也想知道如何获得val loss

还没有，感觉会很麻烦。

[HUANGL1NJIE]

Hello, I am also using mmseg version 1.2.2. I sincerely wonder how to get the validation loss and the accuracy score in the training step. I also encountered this problem, how did you solve it? Thanks for any help.

I am still searching for a effective solution, regret for not providing any useful help to you.

[HUANGL1NJIE]

you can look this flow image in document:

and when you in test and val stage, you just evaluator model, don't receive loss, and can't get val loss

if you want to get val loss, maybe you can see this issue: Logging validation loss without library code hacks #1396

Thanks for your patient comments. From the flow image, it seems like the val loss is unable to be calculated in the mmseg 1.2.2. The solutions discussed in the issue may only work with earlier versions of mmseg.