text-similarity icon indicating copy to clipboard operation
text-similarity copied to clipboard

Published 20 hours ago • verified publisher icon DengBoCong

多卡 attention

Open skynet-jzm opened this issue 2 years ago • 0 comments

bert 模型多卡训练计算attention的时候有问题， Traceback (most recent call last): File "xxxxxxxx/text-similarity/sim/pytorch/common.py", line 291, in scaled_dot_product_attention attention_scores += (mask * -1e9) RuntimeError: The size of tensor a (256) must match the size of tensor b (512) at non-singleton dimension 3

skynet-jzm avatar Aug 15 '22 10:08 skynet-jzm