MrSworder

加载electra-base模型时，self.layers[layer]报错找不到“Emebedding-Mapping"。发现在构建模型时，prepare_variable()构建和variable_mapping()构建的结构有出入

5

def prepare_embeddings(self, inputs): """BERT的embedding是token、position、segment三者embedding之和 """ x, s, layer_norm_conds = inputs z = layer_norm_conds[0] x = self.call(inputs=x, layer=Embedding, input_dim=self.vocab_size, output_dim=self.embedding_size, embeddings_initializer=self.initializer, mask_zero=True, name='Embedding-Token') s = self.call(inputs=s, layer=Embedding, input_dim=2, output_dim=self.embedding_size, embeddings_initializer=self.initializer, name='Embedding-Segment')...

这是我运行run_pretrain.py时的运行报错，显示在temperature_sampling函数中pred_ids = probs.cpu().multinomial(probs.size()[1],replacement=False)这一句存在问题，由于multinomial输入的weights只能是1维或2维，而这里的probs的shape是【32，128，21128】所以出错。请问这里应该如何修改？ >python run_pretraining.py --data_dir=dataset/ --vocab_path=prev_trained_model/electra_tiny/vocab.txt --data_name=electra --config_path=prev_trained_model/electra_tiny/config.json --output_dir=outputs/ 03/31/2022 11:38:38 - INFO - root - samples_per_epoch: 173385 03/31/2022 11:38:38 - INFO - root - device: cuda , distributed...

In readme, there is a command : python train.py --dataset review --split_ratio 0.25 --seed 0 \ --train_type base \ --backbone bert --classifier_type softmax --optimizer adam_ood \ But in the code,...