DialogBERT
DialogBERT copied to clipboard
guxd
Source Code for DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances (https://arxiv.org/pdf/2012.01775.pdf)
I am training on the DailyDialog dataset with the same hyperparameters as described in the paper. I cannot seem to get the model to perform to the standards described in...
Hi, I'm trying to reproduce the results you reported in the paper and unable to do so with the set of current hyperparameters. One notable problem is with per_gpu_eval_batch_size=1. Keeping...
hi, the dataset in "data/dailydial/*.h5", how can I get this file form raw txt file? Thanks in advance!
self.context_mlm_trans and self.context_order_trans are expecting a different key-structure RuntimeError: Error(s) in loading state_dict for BertPredictionHeadTransform: Missing key(s) in state_dict: "dense.weight", "dense.bias", "LayerNorm.weight", "LayerNorm.bias". Unexpected key(s) in state_dict: "utt_encoder.bert.embeddings.position_ids", "utt_encoder.bert.embeddings.word_embeddings.weight", "utt_encoder.bert.embeddings.position_embeddings.weight",...
test error
def load(self, args): # Load a trained model and vocabulary that you have fine-tuned assert args.reload_from>=0, "please specify the checkpoint iteration in args.reload_from" output_dir = os.path.join(f"./output/{args.model}/{args.model_size}/models/", f'checkpoint-{args.reload_from}') self.model = DialogBERT.from_pretrained(output_dir)...
Hi, thanks for making your work available. Are there any checkpoints available for DialogBERT to experiment with, or does it have to be trained from scratch using main.py? Can you...
Hi again, I am curious about what methods the paper authors used for context with DialogBERT development? Did you use context prepending of input tokens for that? And how many...
gradient checkpointing would be super helpful for training.
I failed to install tokenizers because the pip version is too low. Does Python have to be 3.6?