generate.py: error: argument --encoder: invalid choice: 'gpt'

[SefaZeng]

I try to use the MixedCorpus+GptEncoder+LmTarget model for generate some text like GPT2. And I followed the example scripts like

python3 scripts/generate.py --pretrained_model_path models/gpt_model.bin --vocab_path models/google_zh_vocab.txt 
                            --input_path story_beginning.txt --output_path story_full.txt --config_path models/bert_base_config.json 
                            --encoder gpt --target lm --seq_length 128  

Then it raise an error generate.py: error: argument --encoder: invalid choice: 'gpt' It seems there are some codes mismatch because there is no such encoder type called gpt in UER code. In addition to this, there are also something wrong with the pretrain scripts for GPT and the release GPT model from MixedCorpus+GptEncoder+LmTarget. Some features in the pretrain scripts like --embedding word_pos --remove_embedding_layernorm --layernorm_positioning pre need to be droped if you wanna finetune the MixedCorpus+GptEncoder+LmTarget model. And you need add another args --has_lmtarget_bias Maybe you guys make some other scripts in the pretrain stage.

[hhou435]

We have updated the structure of UER. In the latest version ,the gpt encoder has been replaced by the transformer encoder --mask causal. We will adjust the scripts code and readme in the near future to ensure that it can run correctly. Hope you keep paying attention to UER. Now you can use the following script to generate text : python3 scripts/generate.py --pretrained_model_path models/gpt_model.bin --vocab_path models/google_zh_vocab.txt --input_path story_beginning.txt --output_path story_full.txt --config_path models/bert_base_config.json --encoder transformer --mask causal --target lm --layernorm_positioning pre --remove_embedding_layernorm --embedding word_pos --seq_length 128