keras
keras copied to clipboard
keras-team
Check if softmax activation is used correctly
Feature request was made in #18909
This is a utility function to check if the usage of softmax makes sense (new users make this mistake a lot). Applying softmax on a single neuron will make the model output ones everytime, there are too many Stackoverflow posts about this.
This applies for any other layers (Conv2D etc.) where the applied axis (axis=-1 default) of softmax has only one unit.
Codecov Report
Attention: 6 lines in your changes are missing coverage. Please review.
Comparison is base (
c8700f4) 80.14% compared to head (443d784) 69.79%. Report is 37 commits behind head on master.
| Files | Patch % | Lines |
|---|---|---|
| keras/models/functional.py | 73.91% | 3 Missing and 3 partials :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## master #19189 +/- ##
===========================================
- Coverage 80.14% 69.79% -10.35%
===========================================
Files 341 362 +21
Lines 36163 39071 +2908
Branches 7116 7571 +455
===========================================
- Hits 28982 27271 -1711
- Misses 5578 10238 +4660
+ Partials 1603 1562 -41
| Flag | Coverage Δ | |
|---|---|---|
| keras | 69.71% <76.92%> (-10.28%) |
:arrow_down: |
| keras-jax | 59.48% <76.92%> (-3.58%) |
:arrow_down: |
| keras-numpy | 53.93% <73.07%> (-3.16%) |
:arrow_down: |
| keras-tensorflow | 60.79% <76.92%> (-3.86%) |
:arrow_down: |
| keras-torch | ? |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
This is something that would make more sense in the softmax op, actually. Just check if ops.shape[axis] == 1 and print a warning if so.
This is something that would make more sense in the
softmaxop, actually. Just check ifops.shape[axis] == 1and print a warning if so.
So you mean completely removing it from model construction process and put it under ops/nn.py? Intuitively it makes more sense to me throw an error if something is not just correct or valid. However happy to go with the warning too.
I just added a warning for incorrect softmax usage. Thanks for the suggestion!