Incorrect result after simplification

[satya-das]

I am surprised to see a wrong simplification computation with mathics. Following instructions lead to incorrect result. Steps to follow. Enter the following:

1. F[K_, n_] := (2K+1) 2 ^ (1+n) + 2^n-1 F[K, n]

2. (3 * F[K, n] + 1) / 2

3. Simplify[%]

Result after step 2 is correct but the final step which simplifies the previous result is wrong. See the screenshot for the detail. There is no way 9 can appear in the result but it does when mathics is used.

My version of Mathics reports the following:

Mathics 1.1.0
on CPython 3.6.9 (default, Oct  8 2020, 12:12:24) 
using SymPy 1.6.2, mpmath 1.1.0, django 3.1.4

[rocky]

Mathics uses sympy to handle symbolic manipulation.

As best as I can tell though, Mathics conversion of its WL expressions to Sympy is very buggy.

For the above:

(1 + 3(-1 + 2^n + (1 + 2K) 2^(1+n))) / 2

is getting translated for Sympy Python to simplfy:

3*2**n/2 + 3*2**(n + 1)*(2*K + 1)/2 - 1/2

This translation looks incorrect to me.

[satya-das]

Any workaround I can use to get it right? I am fine if modifying the expression makes mathics do the right translation.

[rocky]

If you use https://github.com/mathics/Mathics/tree/SympyForm you can use SympyForm[expr] or expr // SympyForm to see how Mathics is converting the expression entered.

With that you might be able to figure out and narrow the kinds of things that aren't being correctly translated into sympy for it to work on.

Report back the more targeted information on what's wrong. Even better, please just go in there and investigate and fix whatever the problems are and put in a PR.

[suhr]

Some bugs could be caught by generating random sympy expression and ensuring that the following diagram commutes: https://q.uiver.app/?q=WzAsNCxbMCwwLCJTeW1weUV4cHIiXSxbMiwwLCJNYXRoaWNzRXhwciJdLFsyLDIsIlN5bXB5RXhwciJdLFswLDIsIlN5bXB5RXhwciJdLFswLDEsInRvTWF0aGljcyJdLFsxLDIsInRvU3ltcHkiXSxbMiwzLCJzaW1wbGlmeSJdLFswLDMsInNpbXBsaWZ5IiwyXV0=

Also we can test it inductively, like checking that e1 / e2 is a valid expression if e1 and e2 are valid expressions and so on.

[rocky]

A bug with this bug report is that there is missing input not shown.

So I don't have to retype a lot of stuff when trying to replicate this, start out with:

x = (1 + 3 (-1 + 2^n + (1 + 2K) 2^(1 + n))) /  2

@satya-das there is a general principle here: the simpler the bug report, and the easier it is to replicate, the more likely it is going to get looked at and fixed.

Should also note that kind of a big conceptual problem has been uncovered with to_sympy(). The bug fix in #1096 doesn't address this but the conceptual flaw in that may also appear here as well.

[satya-das]

A bug with this bug report is that there is missing input not shown.

So I don't have to retype a lot of stuff when trying to replicate this, start out with:

x = (1 + 3 (-1 + 2^n + (1 + 2K) 2^(1 + n))) /  2

@satya-das there is a general principle here: the simpler the bug report, and the easier it is to replicate, the more likely it is going to get looked at and fixed.

Should also note that kind of a big conceptual problem has been uncovered with to_sympy(). The bug fix in #1096 doesn't address this but the conceptual flaw in that may also appear here as well.

I dont think there is any bug in the bug report. I have given exact three steps that you can literally copy paste into mathics and see the result. Didn't copy paste those three steps work for you?

[rocky]

Didn't copy paste those three steps work for you?

No.

mathics

$ Mathics 2.0.0dev
on CPython 3.7.9 (default, Nov  3 2020, 05:22:01) 
using SymPy 1.7.1, mpmath 1.1.0

Copyright (C) 2011-2020 The Mathics Team.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions.
See the documentation for the full license.

Quit by pressing CONTROL-D

In[1]:= F[K_, n_] := (2K+1) 2 ^ (1+n) + 2^n-1 F[K, n]
In[2]:= (3 * F[K, n] + 1) / 2
$RecursionLimit::reclim: Recursion depth of 200 exceeded.
Out[2]= $Aborted

F[K_, n_] is defined interms of F[K, n].

For this recursive definition, I don't see any base cases defined.

Perhaps this does not give the above results in Mathics 1.1.0, which was a while ago. 1.1.1 was released before Christmas and the current development version is 2.0.0dev.

However since you report that this part works, it is irrelevant.

Your job as a bug reporter is to figure out what is the smallest and simplest test case that doesn't work. Of course, you don't have to do that. But then others don't have to bother with this either.

[satya-das]

Hi @rocky, Sorry but you are copy-pasting it wrong. Here is my output:

In[1]:= F[K_, n_] := (2K+1) 2 ^ (1+n) + 2^n-1
In[2]:= F[K, n]
Out[2]= -1 + 2 ^ n + (1 + 2 K) 2 ^ (1 + n)

In[3]:= (3 * F[K, n] + 1) / 2
Out[3]= (1 + 3 (-1 + 2 ^ n + (1 + 2 K) 2 ^ (1 + n))) / 2

In[4]:= Simplify[%]
Out[4]= -1 + 6 K 2 ^ n + 9 2 ^ n / 2

I have copy pasted from the bug report and it demonstrates the exact problem I wanted to report with a smallest possible example. You are mixing the first two lines and that is why it is causing problem for you. They are two separate lines. I see my style of using mathics is probably different than yours and that might have caused confusion. I tend to print the function that I just defined so that I can see if the definition was correctly understood by mathics.

I tried my best to report the problem as clearly and easily reproducible as possible. And so I also attached a screenshot because many a times that helps in removing ambiguity. But still it appears my effort did not have desirable effect. :)

I hope my explanation is helpful in resolving confusion.

Thanks, Satya

[rocky]

Ok. Thanks for the clarification. I guess a lesson here is that although screen shots are nice, giving the bug as text input is preferred and for that use mathics or mathicsscript and cut and paste the text input rather (which reports In's and Out's) rather than try to translate that that in your own format.

So the simplify part I don't I don't think you've got that down. As indicated before, is something if working, you don't need to report it. So entering the formula with F isn't needed and as we see it gets in the way. Instead of describing the problem in 3 steps. you can describe the problem in one simpler step.

The point is that there is a simpify expression that isn't working. Is that the simplest simplify expression that isn't working? No. How about removing the -1 for example? Does that make things work? If not, then the -1 can be removed and we have a simpler expression that demonstrates there is a problem.

Finally I should say that the "9" isn't necessarily incorrect. https://www.wolframalpha.com/input/?i=Simplify%5B%281+%2B+3+%28-1+%2B+2%5En+%2B+%281+%2B+2K%29+2%5E%281+%2B+n%29%29%29+%2F++2%5D has as solution with 9 in it and that looks close. But in the solution with 9, there is a 3 instead of 6.

The fact that we have 2^n / 2 instead of 2^(n-1) or as Mathics currently puts it 2^(-1 + n) (see issue #1098) does indicate this is a bug in translation of input that goes into Sympy similar to #1096 and possibly the fix there is in needing evaluation of sub-expressions in more often.

Notice in reporting the bug I used x - 1 as my example rather than say 2^(n-1). This is an example of what I mean by simple: the 2^ part is irrelevant so it is omitted.