ValueError: `LQMarkov` with `beta=1`

[oyamad]

Π = np.array([[0.8, 0.2],
              [0.2, 0.8]])
Qs = np.array([[[0.9409]], [[0.870489]]])
Rs = np.array([[[1., 0., 1.],
                [0., 0., 0.],
                [1., 0., 1.]],
               [[1., 0., 1.],
                [0., 0., 0.],
                [1., 0., 1.]]])
Ns = np.array([[[-0.97, 0., -0.97]],
               [[-0.933, 0., -0.933]]])
As = np.array([[[0., 0., 0.],
                [0., 1., 0.],
                [0., 5., 0.8]],
               [[0., 0., 0.],
                [0., 1., 0.],
                [0., 5., 0.8]]])
B = np.array([[1., 0., 0.]]).T
Bs = [B, B]
C = np.array([[0., 0., 1.]]).T
Cs = [C, C]

lq_markov_mat1 = qe.LQMarkov(Π, Qs, Rs, As, Bs,
                             Cs=Cs, Ns=Ns, beta=1.)
lq_markov_mat1.stationary_values()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-15-022c46ef29a8> in <module>
----> 1 lq_markov_mat1.stationary_values()

/usr/local/lib/python3.7/site-packages/quantecon/lqcontrol.py in stationary_values(self)
    506 
    507         # == Solve for P(s) by iterating discrete riccati system== #
--> 508         Ps = solve_discrete_riccati_system(Π, As, Bs, Cs, Qs, Rs, Ns, beta)
    509 
    510         # == calculate F and d == #

/usr/local/lib/python3.7/site-packages/quantecon/matrix_eqn.py in solve_discrete_riccati_system(Π, As, Bs, Cs, Qs, Rs, Ns, beta, tolerance, max_iter)
    290 
    291         if iteration > max_iter:
--> 292             raise ValueError(fail_msg.format(max_iter))
    293 
    294         else:

ValueError: Convergence failed after 1000 iterations.

[duncanhobbs]

I tested this same example with different values of beta below. I still get a convergence error when beta is larger than 0.972.

beta error after 1000 iterations:

1 51.302186197106494

0.99 0.0022754969468223862

0.98 1.0506482794880867e-07

0.975 7.403286872431636e-10

0.974 2.764863893389702e-10

0.973 1.0368239600211382e-10

0.972 converged in 966 iterations

[bktaha]

This should be addressed by PR #550 I did not link the PR here, so this has remained open.

[jstac]

Many thanks for your contributions @bktaha .

@shizejin , could you please confirm and close when ready?

[oyamad]

@shizejin Is the model well defined with beta >= 1? Shouldn't the input with beta >= 1 be rejected?

Refer to the discussion at #506.

[shizejin]

Hi @bktaha we are concerned about the nonexistence of stable solution when beta >= 1 and this shall not be solved by simply adding more iterations. You may check that the example @oyamad raised in this thread fails to converge after 10000 iterations (thanks to your work, we can now freely choose the number of iterations). But again, PR #550 shall have nothing to do with this issue.

@jstac @oyamad

[bktaha]

Oh, I assumed that non-convergence for beta >= 1 was expected behavior. In this snippet from the test file for lqcontrol;

https://github.com/QuantEcon/QuantEcon.py/blob/ca8fb06b52b3cb9b5a53702fbd1090eb7a6dcbd5/quantecon/tests/test_lqcontrol.py#L225-L229

the stationary_values method is expected to raise a ValueError for lq_markov_mat2 which is initialized with beta = 1.05.