Qingyun Wang

icon indicating a website link EagleW.github.io icon indicating an email [email protected]

icon company University of Illinois at Urbana-Champaign icon location Champaign, Illinois icon location Ph.D. student, Computer Science Department, University of Illinois at Urbana-Champaign

Results 1 issues of Qingyun Wang

Question about ITM pretraining

Hi, @dandelin I have some questions about ITM pre-training. For the pretraining ITM, how did you use itm loss and wpa loss? It seems that you use them separately: https://github.com/dandelin/ViLT/blob/762fd3975c180db6fc88f577cf39549983fa373a/vilt/modules/vilt_utils.py#L127-L139...