GradNorm

This in my Demo of Chen et al. "GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks" ICML 2018

Input: Two synthetic regression tasks according to Ma et al. (KDD 2018) "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts"

Network: One shared layer and two task specific towers

Framework: Pytorch

Update (Nov 2021): This repository is not maintained anymore. The unresolved issues are left open in case someone has a revised implementation.