SimCSE icon indicating copy to clipboard operation
SimCSE copied to clipboard

在计算output pooling时不用mask吗?

Open shaomai00 opened this issue 3 years ago • 1 comments

if pooling == 'first-last-avg': outputs = [ keras.layers.GlobalAveragePooling1D()(outputs[0]), keras.layers.GlobalAveragePooling1D()(outputs[-1]) ] output = keras.layers.Average()(outputs) elif pooling == 'last-avg': output = keras.layers.GlobalAveragePooling1D()(outputs[-1]) elif pooling == 'cls': output = keras.layers.Lambda(lambda x: x[:, 0])(outputs[-1]) elif pooling == 'pooler': output = bert.output

平时不用keras, 但是查了一下GlobalAveragePooling1D是有mask入参的,这个不加attention_mask进来不是会有问题吗? https://keras.io/api/layers/pooling_layers/global_average_pooling1d/

shaomai00 avatar Aug 10 '21 12:08 shaomai00

自带有mask

bojone avatar Aug 11 '21 02:08 bojone