zhezhaoa

Question about the loss of Masked LM

5

Thank you very much for this great contribution. I found the loss of masked LM didn't decrease when it reaches the value around 7. However, in the official tensorflow implementation,...

Hello, Thank you for the great work and nice implementation. It really helps me! I know that i can obtain distances through emd( (X[i], BOW_X[i]), (X[j], BOW_X[j]), distance). But how...

The command: ``` python3 finetune/run_classifier_prompt.py --pretrained_model_path models/google_zh_model.bin \ --vocab_path models/google_zh_vocab.txt \ --config_path models/bert/base_config.json \ --train_path datasets/douban_book_review/train.tsv \ --dev_path datasets/douban_book_review/dev.tsv \ --test_path datasets/douban_book_review/test.tsv \ --prompt_path models/prompts.json --prompt_id chnsenticorp_char \ --learning_rate 3e-5...

Conflicting option error encountered

1

Encounter conflicting option error when using scripts including *scripts/generate_lm.py*, *scripts/extract_features.py*, *scripts/cloze_test*, and etc.

关于在您的模型上继续预训练

2

您好打扰您了 我用您的roberta模型进行微调效果很好 但是我用您的模型进行预训练，一上来的mlm准确率是0。我也用完形填空试了一下，确实模型不能进行准确的预测。所以我猜测您的预训练模型的最上层（单词预测）是不是有一些问题，期待您的回复～

请问您们可以简单描述一下在WSC上进行数据增强的细节么 非常感谢哈

Thank you for this great implementation. I have a question initialize_parameters funtion in glove.c. Since you have add 1 to the vector_size (vector_size++) , why do you still add 1...

Tensorflow version

3

I am really happy seeing question generation work like this. However, i am not familiar with torch, is there any resource written in tensrflow that can reproduce your work?