mesh icon indicating copy to clipboard operation
mesh copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

bias in selfAttention

Open wintersurvival opened this issue 4 years ago • 0 comments

when running transformer, bias is not existed in selfAttention. mesh_tensorflow/bert has bias in selfAttention. what's the meaning of relative_attention_type transformer_layer.SelfAttention? how could I get the bias in transformer_layer.SelfAttention?

wintersurvival avatar Dec 03 '20 02:12 wintersurvival