How to tune the loss weights?

[twmht]

Hi,

there are some of the loss weights, like https://github.com/yzd-v/FGD/blob/master/mmdet/distillation/losses/fgd.py#L27

How did you tune these loss weights? Any guideline to tune these hyper parameters?

[yzd-v]

It's used to balance the losses and FGD is not very sensitive to them. You can change the hyper-parameters according to the loss scale.

[twmht]

what did you mean loss scale? Is it related to classification loss and regression loss?

[yzd-v]

The smallest distillation loss scale is similar to the original classification loss or regression loss may be fine.

[twmht]

so you mean that if the classification loss is 0.3, then smallest distillation loss (one of the fpn level) should approach to 0.3 as well?

[yzd-v]

yes, 0.1-2 may be fine too

[twmht]

I found it may be sensitive to these parameters in my own dataset, since there is no improvement and even worse than the baseline.

I would follow your suggestion and see whether it works or not.

[yzd-v]

You can keep the ratio between different loss and change them together to balance loss. You may also check whether the teacher load the weight successfully.

[twmht]

@yzd-v

Do you mind to upload your training log?

[twmht]

Here are my distiller loss

Are these losses too small? the loss value for different fpn levels differ a lot.

[twmht]

update

[yzd-v]

There might be something wrong in your distillation environment. You can test your environment on COCO to compare the results.

[twmht]

yup I am going to test with coco.

[RY-97]

Do you use FGD to distill your own data?

[RY-97]

I am researching this direction, can I exchange some questions about FGD with you?

[twmht]

@renyu-renyu

Are you talking with me?

[RY-97]

@renyu-renyu

你在跟我说话吗？

Yes, if we can, we can communicate together

[twmht]

Yup.

What is your opinion about FGD?

[RY-97]

是的。

您对 FGD 有什么看法？

Are you training your own dataset or?

[twmht]

Yup.

I was training widerface.

but did not get good result, what about you?

[RY-97]

是的。

我正在训练更宽的脸。

但没有得到好的结果，你呢？

I'm reproducing the results of FGD and haven't trained my own network and data

[RY-97]

I'm reproducing the results of FGD and haven't trained my own network and data

[twmht]

did you reproduce?

[RY-97]

你繁殖了吗？ It is reproduced, but the accuracy is not ideal, and it may need to be modified. Have you implemented your own data set?

[twmht]

what do you mean reproduced but the accuracy is not ideal? does the accuracy not match the paper?

[RY-97]

你是什​​么意思复制但准确性不理想？精度点是否与纸张匹配？

yes

[twmht]

Can you share your reproduced accuracies and the paper accuracies?

[RY-97]

你能分享你的复制精度和纸张精度吗？

I'm not quite done yet, I've emailed you

[dd1005]

Can you share your reproduced accuracies and the paper accuracies?

Will you reproduce this article? How about the accuracy of reproduction? I'd like to discuss it with you

[dd1005]

你能分享你的复制精度和纸张精度吗？

I'm not quite done yet, I've emailed you

Will you reproduce this article? How about the accuracy of reproduction? I'd like to discuss it with you

[peiyingxin]

你能分享你的复制精度和纸张精度吗？

I'm not quite done yet, I've emailed you

Will you reproduce this article? How about the accuracy of reproduction? I'd like to discuss it with you

Have you reproduce this article? I'd like to discuss it with you.