BERT-NER-Pytorch
BERT-NER-Pytorch copied to clipboard
lonePatient
预训练模型
请问你这里的bert预训练模型用的哪里的，我用的huggingface的bert-base-chinese，不但训练效果差，而且代码也一直报vocab的索引对不上的问题。
`Some weights of the model checkpoint at C:\BaiduNetdiskDownload\bert-base-chinese were not used when initializing BertCrfForNer: ['cls.predictions.decoder.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight']
- This IS expected if you are initializing BertCrfForNer from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertCrfForNer from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of BertCrfForNer were not initialized from the model checkpoint at C:\BaiduNetdiskDownload\bert-base-chinese and are newly initialized: ['classifier.bias', 'crf.start_transitions', 'crf.transitions', 'crf.end_transitions', 'classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.`
我用的 transformers 版本是 4.12.5 然后加载模型的时候就会报出上面的问题, 我还没往下跑代码, 发现这个问题就先来找找看, 不知道会不会有题主说的情况出现, 总之就是说, premodel 并没有被完全加载进来, 还说我可能需要微调一下这个 pre model 才能继续用. 我换了其它的 pre model 也是一样的问题. 所以我不觉得是模型的问题, 应该是代码里处理 NER 任务的这个类: BertCrfForNer 有问题才会这样. 我准备再看看吧, 好烦, 就是想确定一个合适的任务 baseline, 但是 baseline 总是跑不出一个相对合适的结果. 很烦 ...
我在hugging face上下载了bert-base-chinese到prev_trained_model这个目录是报下面这个错
Traceback (most recent call last): File "run_ner_crf.py", line 496, in
main() File "run_ner_crf.py", line 428, in main model = model_class.from_pretrained(args.model_name_or_path, config=config) File "/Users/bytedance/Projects/BERT-NER-Pytorch/bert_pytorch/lib/python3.7/site-packages/transformers/modeling_utils.py", line 2604, in from_pretrained state_dict = load_state_dict(resolved_archive_file) File "/Users/bytedance/Projects/BERT-NER-Pytorch/bert_pytorch/lib/python3.7/site-packages/transformers/modeling_utils.py", line 450, in load_state_dict with safe_open(checkpoint_file, framework="pt") as f: safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge
修复办法： scripts/run_ner_crf.sh 里的BERT_BASE_DIR=bert-base-chinese，就可以自动下载模型了，看样子git clone和这种默认下载下来的模型可能有略微的不同。
