训练问题

[jiu-leo]

你好，我直接运行你的代码会出现以下问题： Traceback (most recent call last): File "/content/UER-py/finetune/run_text2text_csl.py", line 348, in <module> main() File "/content/UER-py/finetune/run_text2text_csl.py", line 277, in main model = Text2text(args) File "/content/UER-py/finetune/run_text2text_csl.py", line 24, in __init__ self.embedding = str2embedding[args.embedding](args, len(args.tokenizer.vocab)) TypeError: unhashable type: 'list'

当我将self.embedding = str2embedding[args.embedding[0]](args, len(args.tokenizer.vocab))修改之后，又出现以下维度不匹配问题，请问是哪里有问题呢？

Traceback (most recent call last): File "finetune/run_text2text_csl.py", line 360, in <module> main() File "finetune/run_text2text_csl.py", line 338, in main loss = train_model(args, model, optimizer, scheduler, src_batch, tgt_in_batch, tgt_out_batch, seg_batch) File "finetune/run_text2text_csl.py", line 133, in train_model loss, _ = model(src_batch, (tgt_in_batch, tgt_out_batch, src_batch), seg_batch) File "/home/ljx/anaconda3/envs/csl/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/ljx/anaconda3/envs/csl/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward outputs = self.parallel_apply(replicas, inputs, kwargs) File "/home/ljx/anaconda3/envs/csl/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 178, in parallel_apply return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)]) File "/home/ljx/anaconda3/envs/csl/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply output.reraise() File "/home/ljx/anaconda3/envs/csl/lib/python3.8/site-packages/torch/_utils.py", line 425, in reraise raise self.exc_type(msg) IndexError: Caught IndexError in replica 0 on device 0. Original Traceback (most recent call last): File "/home/ljx/anaconda3/envs/csl/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker output = module(*input, **kwargs) File "/home/ljx/anaconda3/envs/csl/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "finetune/run_text2text_csl.py", line 66, in forward loss = self.target(hidden, tgt_out, seg)[0] File "/home/ljx/anaconda3/envs/csl/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/home/ljx/CSL/UER-py-master/uer/targets/lm_target.py", line 84, in forward loss, correct, denominator = self.lm(memory_bank, tgt, seg) File "/home/ljx/CSL/UER-py-master/uer/targets/lm_target.py", line 41, in lm memory_bank = memory_bank[seg > loss_mask, :] IndexError: The shape of the mask [3072] at index 0 does not match the shape of the indexed tensor [288, 768] at index 0

[banianzr]

The source code is modified based on https://github.com/dbiir/UER-py/blob/master/finetune/run_text2text.py, and it depends on UER-py, so you can further fix it based on the original code.