[Previous Feature Request] Throw an error if `softmax` is used incorrectly

[Frightera]

I had a PR (#18105 - closed due to merging process) which was adding a utility function to the codebase if softmax is used with a 1 neuron as this will only produce a vector of 1s (ones).

This is a mistake mostly beginners do, and error should be raised IMHO. There are too many Stack Overflow questions because of this logical error.

Consider:

import keras
import numpy as np

keras.__version__ # 3.0.1

model = keras.Sequential([
    # ... some layers ... #
    keras.layers.Dense(1, activation = "softmax")
])

x = np.random.randn(100, 1)
if np.allclose(model.predict(x), 1.0):
    print("1 neuron with softmax only outputs 1s")

It's actually about the applied axis of the softmax, so this is valid for any layer.

If the PR above sounds reasonable, I can adapt the changes for Keras 3.0

Thanks.

[divyashreepathihalli]

@Frightera based on the context of the old PR, do you want a open a new PR?

[Frightera]

Hi @divyashreepathihalli,

Yes, I can create a new PR if you can confirm that this feature seems reasonable and that the Keras team would be willing to merge it.

Thanks.

[divyashreepathihalli]

yes, please go ahead.

[Frightera]

The work is currently in progress 👍🏼