wdyxwzyh

How to pre-train? has anyone start pre-train successful?

4

1. the repo contains pretrain_glm.py and scripts/ds_pretrain_nvidia.sh, and it got an args $1, so what is this $1？ is it the config_tasks/seq_blank.sh? 2. I want to know if the model...

1. the first is using GLM as constrained text generation, such as [CLS] [M1][x3][M2][x5][M3][S][x6][S][x1, x2][S][x4]. i recommend {Lexically Constrained Neural Machine Translationwith Levenshtein Transformer} AND {POINTER: Constrained Progressive Text Generation...

Hi There, i got some Q and suggestions.

1

thanks for your code =.= 1. Q: the model faces the problem with Error Accumulation, which i suggest use '[DEL]' token to let model know when to delete which words....

slot nums

1

i didn't understand that for sequences {sos , x1, x2, ..., xt, eos}, shouldn't it be t+1 slots as described in INSERTION TRANSFORMER? which means (sos ,x1) (x1,x2),...(xt, eos), but...