The score decreases when the number of statements decrease

[PCManticore]

Steps to reproduce

Use the following example to get a negative score:

from typing import Optional

a, b = 1

a.test

Notice that removing the unused import will result in a lower score.

Current behavior

The removal of the unused import will result in a lower score.

Expected behavior

Removing a fault should result in an increased score. Unfortunately we rely on the number of statements as a factor which will affect the result in the aforementioned example as the number of statements decreased after fixing the issue.

pylint --version output

Any version has this behaviour including 2.4.2

Related is https://github.com/PyCQA/pylint/issues/2399 which aims to remove the negative score as it's not very friendly.

[clavedeluna]

Testing with pylint 2.15.3, no negative numbers are output, instead, both outputs receive 0.

from typing import Optional

a, b = 1

a.test

Your code has been rated at 0.00/10 (

and

a, b = 1

a.test

Your code has been rated at 0.00/10

The spirit of the issue continues to be valid: you'd think that as a user removes one code issue, the score would increase. However, given the few line statements and the computation done 'max(0, 0 if fatal else 10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10))'

the math turns both examples to 0 because everything after 10 - ... is negative.

Any thoughts to what the calculation should now be? I tried a few things like removing the *10 but I"m not sure if this will completely blow out all calculations going forward.

[Pierre-Sassoulas]

At this point using a log for values ranging from infinite negative to 1 so it's instead in the 0 to 1 range, then the normal formula would work imo.

[clavedeluna]

At this point using a log for values ranging from infinite negative to 1 so it's instead in the 0 to 1 range, then the normal formula would work imo.

Let's see if I understood your suggestion. Trying out

max(0, 0 if 0 else 10.0 - math.log((float(5 * 1 + 2 + 0 + 0) / 3) * 10)) ==> 6.85 max(0, 0 if 0 else 10.0 - math.log((float(5 * 1 + 1 + 0 + 0) / 2) * 10)) ==> 6.59

looks better but results in the same issue in which we've reduced the warnings count (and hence the number of statements) and so the score goes down, which isn't what we want.

[Pierre-Sassoulas]

I was thinking of something like math.log(x) if x < 1 else x with x = 10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)). To be fair this is getting really complicated and at this point we might want to do a special case internally if the current option is not user defined.