DeBERTa
DeBERTa copied to clipboard
DeBERTa-MT and code for NLG experiments
Hi, in appendix 4 of the paper you describe results on NLG tasks and a DeBERTa-MT base model pretrained on wikitext-103, using unilm task formulations.
Are you going to release this model and the code for the associated experiments?
Thanks
I'm also interested in deberta-mt implementation, and there is grey area for unilm implementation for example how we can implement disentangled attention, how did author dealt with relative position bias for incremental model, etc.