Check if softmax activation is used correctly

[Frightera]

Feature request was made in #18909

This is a utility function to check if the usage of softmax makes sense (new users make this mistake a lot). Applying softmax on a single neuron will make the model output ones everytime, there are too many Stackoverflow posts about this.

This applies for any other layers (Conv2D etc.) where the applied axis (axis=-1 default) of softmax has only one unit.

[codecov-commenter]

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (c8700f4) 80.14% compared to head (443d784) 69.79%. Report is 37 commits behind head on master.

Files	Patch %	Lines
keras/models/functional.py	73.91%	3 Missing and 3 partials :warning:

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #19189       +/-   ##
===========================================
- Coverage   80.14%   69.79%   -10.35%     
===========================================
  Files         341      362       +21     
  Lines       36163    39071     +2908     
  Branches     7116     7571      +455     
===========================================
- Hits        28982    27271     -1711     
- Misses       5578    10238     +4660     
+ Partials     1603     1562       -41     

Flag	Coverage Δ
keras	69.71% <76.92%> (-10.28%)	:arrow_down:
keras-jax	59.48% <76.92%> (-3.58%)	:arrow_down:
keras-numpy	53.93% <73.07%> (-3.16%)	:arrow_down:
keras-tensorflow	60.79% <76.92%> (-3.86%)	:arrow_down:
keras-torch	?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[fchollet]

This is something that would make more sense in the softmax op, actually. Just check if ops.shape[axis] == 1 and print a warning if so.

[Frightera]

This is something that would make more sense in the softmax op, actually. Just check if ops.shape[axis] == 1 and print a warning if so.

So you mean completely removing it from model construction process and put it under ops/nn.py? Intuitively it makes more sense to me throw an error if something is not just correct or valid. However happy to go with the warning too.

[fchollet]

I just added a warning for incorrect softmax usage. Thanks for the suggestion!