Clarity in tfa for F1 score

[amjass12]

Hi,

I am integrating the F1 score from tensorflow addons for a multi-label classification problem and there is a part of the documentation tat confuses me. Just to be completely clear, this is a snippet of code demonstrating the gist of the task..

model = tf.keras.models.Sequential()
    model.add(tf.keras.layers.Dense(800, input_shape=(20003,), activation='relu'))
    layers in between.....
    model.add(tf.keras.layers.Dense(20, activation='sigmoid'))

    model.compile(loss='binary_crossentropy', optimizer=optimizer, 
                metrics=[tf.keras.metrics.BinaryAccuracy(), 
                tfa.metrics.F1Score(num_classes=20, average='macro',threshold=0.5)])

Each sample contains 2 known labels based on 2 bigger classes within sample set (i.e. shape and color).

I am monitoring F1 and will also use this for reporting later on. As you can see, the threshold is set to 0.5 (what i thought the threshold was in the binary setting) however the documentation states:

Elements of y_pred above threshold are considered to be 1, and the rest 0. If threshold is None, the argmax is converted to 1, and the rest 0.

Please can this be clarified (maybe I have misunderstood) but the argmax to me would make sense in a softmax probabilistic setting, it certainly doesn't work in the multi-label setting where the argmax would single out one sample in a row of one-hot labels, where multiple labels can be correct.

Having said this, if i use the sklearn classification report as follows:

print(classification_report(trainingtargets[test], np.around(model.predict(input[test]))))

The macro F1 score matches that seen in model.evaluate(input[test])

Is it the case then that for multi-label a default threshold is used for each class?