grate icon indicating copy to clipboard operation
grate copied to clipboard

Fixed reading of decimal numbers when they are saved as int

Open mbgonicus opened this issue 1 year ago • 3 comments

The xls standard allows to save a decimal number like 1.75 as an integer, multiplied by 100. The current implementation did neither detect this as an integer nor as a float and subsequently returns the default value 0.

This fix adds the case to RKNumber's Float64() method in order to retrieve the value correctly as float.

It also adds a test and a new xls file in testdata for this case. I did not dare to touch the existing xls files to not corrupt the saved data (I only have LibreOffice and it might do funky things). So if you do not like a new file there, please feel free to edit this PR to your liking.

mbgonicus avatar Nov 15 '23 16:11 mbgonicus

Hmm thanks for catching that! I think we should cover the remaining case also, can you add that in?

if (r&1) == 0 && (r&2) != 0 {
    return float64(val)
}

pbnjay avatar Nov 16 '23 04:11 pbnjay

Hmm thanks for catching that! I think we should cover the remaining case also, can you add that in?

if (r&1) == 0 && (r&2) != 0 {
    return float64(val)
}

But that case would mean it is a signed integer and should be handled via the Int() method there? IsInteger() would be true in that case anyway.

mbgonicus avatar Nov 16 '23 08:11 mbgonicus

@pbnjay btw the failing tests are failing already, i.e. are not failing due to my changes.

mbgonicus avatar Nov 22 '23 14:11 mbgonicus