Sampling fails due to degenerate covariance matrix in parameter proposal step.

[vwiela]

I have a model with some unidentifiabilities and want to perform sampling after optimization. Using the AdaptiveParallelTemperingSampler with an AdaptiveMetropolisSampler as an internal sampler does work for some hundred thousands of iterations but suddenly, it fails when proposing a new parameter vector in the singular value decomposition of the covariance matrix. For me this indicates, that the covariance matrix might converge to some weird degenerate case or to a case where SVD does not work due to numerical issues.

Any ideas why this might happen in the adaptive samplers and how to prevent it? Because just let it fail after hundred of thousands of iterations is not a good solution in my opinion.

Traceback (most recent call last):
  File "sampling_selected_models.py", line 85, in <module>
    result = sample.sample(
             ^^^^^^^^^^^^^^
  File "new_env/lib/python3.12/site-packages/pypesto/sample/sample.py", line 88, in sample
    sampler.sample(n_samples=n_samples)
  File "new_env/lib/python3.12/site-packages/pypesto/sample/parallel_tempering.py", line 111, in sample
    sampler.sample(n_samples=1, beta=beta)
  File "new_env/lib/python3.12/site-packages/pypesto/sample/metropolis.py", line 82, in sample
    x, lpost, lprior = self._perform_step(
                       ^^^^^^^^^^^^^^^^^^^
  File "new_env/lib/python3.12/site-packages/pypesto/sample/metropolis.py", line 114, in _perform_step
    x_new: np.ndarray = self._propose_parameter(x)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "new_env/lib/python3.12/site-packages/pypesto/sample/adaptive_metropolis.py", line 107, in _propose_parameter
    x_new = np.random.multivariate_normal(x, self._cov)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "numpy/random/mtrand.pyx", line 4251, in numpy.random.mtrand.RandomState.multivariate_normal
  File "new_env/lib/python3.12/site-packages/numpy/linalg/_linalg.py", line 1812, in svd
    u, s, vh = gufunc(a, signature=signature)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "new_env/lib/python3.12/site-packages/numpy/linalg/_linalg.py", line 113, in _raise_linalgerror_svd_nonconvergence
    raise LinAlgError("SVD did not converge")
numpy.linalg.LinAlgError: SVD did not converge

I used

• Python 3.12
• pyPESTO 0.5.5
• numpy 2.2.4

[arrjon]

Checking the np.random.multivariate_normal function you most likely encounterd a nan value in the covariance matrix. Could you check, if this is the case and also the value of self._cov_scale?