[NOMERGE] Nomogram tests - investigating unreproducible random error

[janezd]

Playing around.

Tests for tooltip of nomogram's marker fail randomly, but only on Ubuntu with Python 3.7 or, sometimes, 3.8. Interestingly either both tests, for LR and NB fail or both pass. This only happens on CI; I can't reproduce this on Ubuntu machine at home.

[codecov[bot]]

Codecov Report

Merging #6018 (d97e2a7) into master (c495fba) will decrease coverage by 0.08%. The diff coverage is 75.00%.

@@            Coverage Diff             @@
##           master    #6018      +/-   ##
==========================================
- Coverage   86.58%   86.50%   -0.09%     
==========================================
  Files         315      315              
  Lines       67302    67307       +5     
==========================================
- Hits        58275    58222      -53     
- Misses       9027     9085      +58     

[janezd]

@VesnaT, see prints I added to the code.

Test for naive Bayes

My machine (and, probably yours) prints

val=12.173463707365983, p=None, k=[11.72533296  7.25521762], cls_index=0, d_=30.796719820629992
nnp 0.6101947116490711
val=374.90275104235775, p=None, k=[11.72533296  7.25521762], cls_index=0, d_=30.796719820629992
nnp 0.6101947116490707
val=374.90275104235775, p=None, k=[11.72533296  7.25521762], cls_index=0, d_=30.796719820629992
nnp 0.6101947116490707

val=0.0, p=[0.54098361 0.45901639], k=[-0.16430305  0.16430305], cls_index=0, d_=59.93836724461009
nnp 0.5409836065573771
val=0.0, p=[0.54098361 0.45901639], k=[-0.16430305  0.16430305], cls_index=0, d_=59.93836724461009
nnp 0.5409836065573771
val=500.0, p=[0.54098361 0.45901639], k=[-0.16430305  0.16430305], cls_index=0, d_=59.93836724461009
nnp 0.9997978311828017

It thus passes two tests and fails on the third. I can't confirm it's always like this.

All numbers are equal - except val. val is position of the circle with the sum on the nomogram (probably computed from the sum, or maybe not). When this fails on gh, the circle is (I suppose) all the way left or right (500 is probably horizontal size).

Test for logistic regression

My machine (and, probably yours) prints

val=12.173463707365983, p=None, k=[11.72533296  7.25521762], cls_index=0, d_=30.796719820629992
nnp 0.6101947116490711

When it fails on Github, it prints

val=6.602235731061593, p=None, k=[11.72603981  7.25598097], cls_index=0, d_=30.79538751568359
nnp 0.005918100399121266

This failed on first test already. I don't know if it's always like this.

Notes

• It is failing on Ubuntu, never Windows or MacOS. I've seen it fail on Python 3.7-3.9, which doesn't mean it can't fail on 3.10.
• Most interestingly: it fails for none or for both (nb and lr). I've never seen it fail on only one of them.