QCEngine icon indicating copy to clipboard operation
QCEngine copied to clipboard

7 tests fail

Open yurivict opened this issue 3 years ago • 10 comments

Describe the bug qcengine-test-failures.txt

Python 3.9 FreeBSD 13.1 STABLE

yurivict avatar Sep 30 '22 05:09 yurivict

The first (and most alarming looking) failure, the scf ene: computed value (-1.7055776) does not match (-1.6561385) has to do with nwchem versions. The root that nwc finds varies btwn versions, so it fails for me locally, too, but passes with the c-f version in CI. Not a great test case, I admit, but nothing to be concerned about.

The geometric or rdkit fails look like error msg formatting of some kind. Those would need investigation.

Thanks for report.

loriab avatar Oct 10 '22 14:10 loriab

For the version 0.25.0 one test fails: qcengine-test-failures.txt

yurivict avatar Nov 14 '22 20:11 yurivict

0.27.0 has 31 test failures: qcengine-0.27.0-tests.log

yurivict avatar Aug 03 '23 09:08 yurivict

Thanks for the update. Most or all are gcp errors? Is there any chance it's detecting the wrong gcp -- there's a system one (https://manpages.ubuntu.com/manpages/trusty/man1/gcp.1.html). Of the proper "geometric counter poise" project, there's two implementations -- the original with executable gcp and the newer preferred one with executable mctc-gcp. Does that provide any clues as to what's going wrong?

loriab avatar Aug 04 '23 04:08 loriab

Yes, the gcp executable was unrelated.

Now with the mctc-gcpexecutable there are 2 failures. The first failure was a runaway process that I killed after many hours. The second failure is a regular test failure.

log

yurivict avatar Aug 06 '23 18:08 yurivict

Is the "regular test failure" the nwchem one with scf ene: computed value (-1.7055776) does not match (-1.6561385) to atol=3e-06 by difference (-0.0494391).? That one I know about -- the SCF root that nwchem finds differs by nwchem version.

loriab avatar Aug 06 '23 21:08 loriab

Now I am rerunning the tests and

  • test_hf_energy_module[hf uhf ae: psi4-dz-ene0] hangs for a very long time so I have to kill it
  • the ``test_atom_labels[nwchem] failure happens (scf ene: computed value (-1.7055776) does not match (-1.6561385) to atol=3e-06 by difference (-0.0494391).```)

yurivict avatar Aug 07 '23 21:08 yurivict

I don't know about the first (psi4) one -- it's pretty quick for me.

> pytest -v qcengine -k "psi4-dz-ene0"
============================================================================================= test session starts =============================================================================================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0 -- /psi/toolchainconda/envs/asdfadcc/bin/python3.11
cachedir: .pytest_cache
rootdir: /psi/gits/QCEngine
configfile: setup.cfg
plugins: cov-4.1.0, xdist-3.3.1
collected 1022 items / 1003 deselected / 3 skipped / 19 selected                                                                                                                                              

qcengine/programs/tests/test_alignment.py::test_hf_alignment[hf  rhf ae: psi4-dz-ene0-fixed-SRM9] PASSED                                                                                                [  5%]
qcengine/programs/tests/test_alignment.py::test_hf_alignment[hf  rhf ae: psi4-dz-ene0-free-SRM9] PASSED                                                                                                 [ 10%]
qcengine/programs/tests/test_alignment.py::test_hf_alignment[hf  uhf ae: psi4-dz-ene0-fixed-SRM9] PASSED                                                                                                [ 15%]
qcengine/programs/tests/test_alignment.py::test_hf_alignment[hf  uhf ae: psi4-dz-ene0-free-SRM9] PASSED                                                                                                 [ 21%]
qcengine/programs/tests/test_standard_suite.py::test_hf_energy_module[hf  rhf ae: psi4-dz-ene0] PASSED                                                                                                  [ 26%]
qcengine/programs/tests/test_standard_suite.py::test_hf_energy_module[hf  uhf ae: psi4-dz-ene0] PASSED                                                                                                  [ 31%]
qcengine/programs/tests/test_standard_suite.py::test_hf_energy_module[hf rohf ae: psi4-dz-ene0] PASSED                                                                                                  [ 36%]
qcengine/programs/tests/test_standard_suite.py::test_mp2_energy_module[mp2  rhf ae: psi4-dz-ene0] PASSED                                                                                                [ 42%]
qcengine/programs/tests/test_standard_suite.py::test_mp2_energy_module[mp2  rhf fc: psi4-dz-ene0] PASSED                                                                                                [ 47%]
qcengine/programs/tests/test_standard_suite.py::test_mp2_energy_module[mp2  uhf ae: psi4-dz-ene0] PASSED                                                                                                [ 52%]
qcengine/programs/tests/test_standard_suite.py::test_mp2_energy_module[mp2  uhf fc: psi4-dz-ene0] PASSED                                                                                                [ 57%]
qcengine/programs/tests/test_standard_suite.py::test_mp2_energy_module[mp2 rohf ae: psi4-dz-ene0] PASSED                                                                                                [ 63%]
qcengine/programs/tests/test_standard_suite.py::test_mp2_energy_module[mp2 rohf fc: psi4-dz-ene0] PASSED                                                                                                [ 68%]
qcengine/programs/tests/test_standard_suite.py::test_ccsd_energy_module[ccsd  rhf ae: psi4-dz-ene0] PASSED                                                                                              [ 73%]
qcengine/programs/tests/test_standard_suite.py::test_ccsd_energy_module[ccsd  rhf fc: psi4-dz-ene0] PASSED                                                                                              [ 78%]
qcengine/programs/tests/test_standard_suite.py::test_ccsd_energy_module[ccsd  uhf ae: psi4-dz-ene0] PASSED                                                                                              [ 84%]
qcengine/programs/tests/test_standard_suite.py::test_ccsd_energy_module[ccsd  uhf fc: psi4-dz-ene0] PASSED                                                                                              [ 89%]
qcengine/programs/tests/test_standard_suite.py::test_ccsd_energy_module[ccsd rohf ae: psi4-dz-ene0] PASSED                                                                                              [ 94%]
qcengine/programs/tests/test_standard_suite.py::test_ccsd_energy_module[ccsd rohf fc: psi4-dz-ene0] PASSED                                                                                              [100%]

=============================================================================== 19 passed, 3 skipped, 1003 deselected in 47.34s ===============================================================================

the second (nwchem) one is the one I was thinking of -- the calculation can simply land on either of two roots. I get it "wrong" locally, too, b/c I leave the repo set to the answer that our CI nwc yields. Maybe I should make it pass for either correct value ...

loriab avatar Aug 07 '23 21:08 loriab

0.30.0 has 1 test failure:

========================================================================================= FAILURES ==========================================================================================
_____________________________________________________________________________________ test_dftd3_error ______________________________________________________________________________________

    @using("dftd3")
    def test_dftd3_error():
        json_data = {
            "molecule": qcng.get_molecule("eneyne"),
            "driver": "energy",
            "model": {"method": "b3lyp-d3(bj)"},
            "keywords": {},
        }
    
        # Test driver
        with pytest.raises(qcng.exceptions.InputError) as exc:
            input_data = json_data.copy()
            input_data["driver"] = "properties"
            ret = qcng.compute(input_data, "dftd3", raise_error=True)
    
>       assert "Driver properties not implemented" in str(exc.value)
E       AssertionError: assert 'Driver properties not implemented' in 'Driver DriverEnum.properties not implemented for DFTD3.'
E        +  where 'Driver DriverEnum.properties not implemented for DFTD3.' = str(InputError('Driver DriverEnum.properties not implemented for DFTD3.'))
E        +    where InputError('Driver DriverEnum.properties not implemented for DFTD3.') = <ExceptionInfo InputError('Driver DriverEnum.properties not implemented for DFTD3.') tblen=4>.value

qcengine/programs/tests/test_dftd3_mp2d.py:47: AssertionError
===================================================================================== warnings summary ======================================================================================

Python-3.11 FreeBSD 14.1

yurivict avatar Jun 26 '24 15:06 yurivict