f_scores & sparse_f_scores

[markub3327]

Hello,

@bhack @LukeWood

this is solution of #393. Please check it and I'll make a test for SparseFBetaScore.

Thanks.

[bhack]

What is the status of this? We are closing another request on TF https://github.com/tensorflow/tensorflow/issues/56358

[LukeWood]

What is the status of this? We are closing another request on TF tensorflow/tensorflow#56358

I've been prioritizing other issues over this for a bit, also deciding how components that support multiple label types:

i.e. classification, object detection, and image segmentation should be structured

I will do a pass on this now.

[innat]

Why is it required to have f_1/2_score if we can get them with a general f_beta_score? It doesn't seem to need various beta=n versions and thus will reduce implemented cost and clean API.

[markub3327]

@innat

It's easier for the newbie users to understand the CV.

For example, F1 is common metrics described in wide range of tutorials.

[bhack]

Why is it required to have f_1/2_score if we can get them with a general f_beta_score? It doesn't seem to need various beta=n versions and thus will reduce implemented cost and clean API.

It could be similar to passing single RGB values or Colormapping: https://github.com/keras-team/keras-cv/pull/433

[markub3327]

Hi @bhack @LukeWood, all tests are passed. You can merge it. Thanks!

[bhack]

Do we have a review here?

[innat]

@markub3327 Could you please test your implementation with this https://github.com/keras-team/tf-keras/issues/251 ? (As this implementation is quite similar to tfa.f1 score.)

[markub3327]

@innat I have available PC with only one GPU.

[innat]

You can still try with mirror or single strategy scope. Or, use kaggle env, it provides multi gpu for free and tf 2.11.

[markub3327]

@innat @bhack

All is working for me. Here is notebook: https://www.kaggle.com/code/martinkubovksk/notebooke5641a9136

Here is the result:

Number of devices: 2

Epoch 1/5 2/2 [==============================] - 15s 608ms/step - loss: 0.8687 - f1_score: 0.6667 - val_loss: 0.9360 - val_f1_score: 0.6667 Epoch 2/5 2/2 [==============================] - 0s 177ms/step - loss: 0.8406 - f1_score: 0.6667 - val_loss: 0.9011 - val_f1_score: 0.6667 Epoch 3/5 2/2 [==============================] - 0s 173ms/step - loss: 0.8150 - f1_score: 0.6667 - val_loss: 0.8684 - val_f1_score: 0.6667 Epoch 4/5 2/2 [==============================] - 0s 173ms/step - loss: 0.7930 - f1_score: 0.6667 - val_loss: 0.8386 - val_f1_score: 0.6667 Epoch 5/5 2/2 [==============================] - 0s 165ms/step - loss: 0.7698 - f1_score: 0.6667 - val_loss: 0.8110 - val_f1_score: 0.6667 [8]: <keras.callbacks.History at 0x7fe44859cfd0>

[LukeWood]

These metrics are a bit more generic than just "CV" and as such probably belong in Keras core if they are valuable. Closing for now, but perhaps @fchollet would think these are a good fit for core Keras? If there is a strong user need I think we should re-open this PR to introduce F-scores in core, but for now lets close until such a need arises.