[BUG] Graph of Validation Loss does not work with Hetero-nn

[PaulKMandal]

trafficstars

Describe the bug Hetero-nn does not calculate the validation loss. It only runs the validation set during eval.

To Reproduce Run Hetero-nn with a validation set.

Expected behavior A graph of the validation loss should show up on FATE Board like it does with Homo-nn

Screenshots N/A

FATE Version: FATE 1.11.2

Additional context We tried to fix this ourselves. When we tried to calculate loss on the validation batch, we end up getting a collision error with a hash. This issue has not been fixed in 1.11.3 and 1.11.4. Tagging @talkingwallace since he was the last to commit to hetero-nn.

[talkingwallace]

Hi! In the FATE-1.X version, validation loss is not supported. In Homo-NN the guest&host Fateboards display local train loss and the arbiter Fateboard displays aggregated train loss.

[PaulKMandal]

Gotcha, are there any plans on implementing this in 2.0? Should I turn this into a feature request?

[talkingwallace]

Hi, In the 2.0-RC version(which is going to be released next month), the hetero-nn and homo-nn are developed based on the transformer trainer and can run without fateflow and fateboard. I think it will be easy to get a validation loss.