Programmer View: Wrong Result: (1<<46) / 8380417 = 0

[rensushan]

Describe the bug

I am doing some cryptography research and its convinent for me to quickly validate some proofs .

When dividing integers in Programmer view the result of (1<<46) /8380417 is 0 instead of 8396807.

The results for dividing by 8380416 and 8380418 are correct.

Tested on two different machines running Windows 11.

Steps To Reproduce

Steps To Reproduce

Go to Programmer view Select DEC input Input 1<<46 Hit "=" Output should be 70,368,744,177,664 (Correct) Hit "/", Then input 8380417 ( to divide 70,368,744,177,664 by 8380417 ) Result is 0

Expected behavior

Result shoud be 8396807

Screenshots

Device and Application Information

• OS Build: Microsoft Windows 11(Home Edition);
• Architecture: AMD Ryzen 7 5825U with Radeon Graphics 2.00 GHz;
• Application Version: Calculator 11.2210.0.0
• Region:
• Dev Version Installed: HP ZHAN 66 Pro A, 14" G5 Notebook.

Additional context

Requested Assignment

I'm just reporting this problem. I don't want to fix it.

[rillig]

The wrong value is calculated in scidisp.cpp:73, in result &= GetChopNumber().

For some reason, Rational(8396807.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000019316) & 0xffff_ffff_ffff_ffff results in 0.

Thinking further, the many zeroes are already wrong, the rational number should never have m_mantissa.size() == 16, it should simply be Rational(1 << 46, 8380417).

According to Perl and Python, 1 << 46 / 8380417 is 8396807.00586427 or 8396807.005864266.

Please don't expect further help or input from me, I just did this as an exercise to try out the Visual Studio debugger.

[rillig]

Minimal reproducer:

auto nom = CalcEngine::Rational(1ULL << 46);
auto den = CalcEngine::Rational(8380417);
auto quot = nom / den;

auto mask = CalcEngine::Rational(0ULL - 1);
auto result = quot & mask;

[MicrosoftIssueBot]

This is your friendly Microsoft Issue Bot. I've seen this issue come in and have gone to tell a human about it.

[rillig]

At the end of intrat, there is an assertion missing that the result is actually an integer.

The whole approach of doing limited-precision arithmetics including rounding when dealing with perfectly exact rational numbers, especially in programmer mode, is complete crap.