problems with ChiantiPy and pytesting

[kdere]

with the new core/test/test_spectrum_api.py, pytest is finding a lot of errors. the main problem is that the tests are trying to use the ChiantiPy methods in ways that were unanticipated, mainly because I never explicitly stated how the constraints on the sizes of temperature, density and emission measure, especially how they would act in different methods.

So, I need to put some sort of statement into the docs and in the mean time I will try to sort things out.

One of the questions is whether to make the test more appropriate to the way things are or to adapt the methods to these unanticipated ways of calling things.

One example of this is the fact that the radiative losses (radLoss), especially the free-free and free-bound continuum, do not depend too much on density so that the specification of the density was not anticipated.

[svaberg]

This is a really good point and indeed the tests may be a bit too aspirational! I just tested things that I would expect to work from the 'principle of least surprise'. Any test that is expected to fail I can just test for the particular exception instead or the test case can be removed.

One things that I have noticed is that the test of spectrum and ipymspectrum don't always give the same result, for example these two

cd <CHIANTIPY>  # Here I run the tests from the project root folder
pytest ChiantiPy/core/tests/test_spectrum_api.py::test_spectrum_container_length_compatibility[1-1-1-container_type0-spectrum]
pytest ChiantiPy/core/tests/test_spectrum_api.py::test_spectrum_container_length_compatibility[1-1-1-container_type0-ipymspectrum]

has one succeeding and the other one failing.

[svaberg]

If a value is expected or required to be a scalar I guess you can cast it to a float at the start of the method as a sort of input validation

density = float(density)

the same way that you use np.atleast_1d for anything that is not a scalar. In that way you only need to deal with numpy arrays and float values inside the functions.

[kdere]

I suspect that the problem with ipymspectrum is that you need to start ipcluster ipcluster3 start --n=3

[kdere]

also saw a warning the ipyparallel was not installed on your machine.

[svaberg]

Hi Ken,

If when you run this

pytest -v -k "ipymspectrum"  # Run all the tests with ipymspectrum 

you see this message at the end of the result

============================================================= warnings summary =============================================================
/Users/u1092841/Documents/PHD/sw_tools/ChiantiPy/ChiantiPy/core/IpyMspectrum.py:9
  /Users/u1092841/Documents/PHD/sw_tools/ChiantiPy/ChiantiPy/core/IpyMspectrum.py:9: UserWarning: ipyparallel not found. You won't be able to use the ipymspectrum module
    warnings.warn("ipyparallel not found. You won't be able to use the ipymspectrum module")

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================================== 24 failed, 101 deselected, 1 warnings in 8.14 seconds ===========================================

I think it means that ipyparallel is not found by pytest on your machine somehow.

My environment

Some ipymspectrum tests pass in my python environment so I think it is correctly configured with ipyparallel. When I run this

ipcluster start –n=4
pytest -v -k "test_spectrum_container_length_compatibility[5-5-5-container_type0-ipymspectrum]"

The test passes, so my configuration seems to be okay.

My current results

Below is a cropped version of the test output from running

pytest -v -k "not -mspectrum" test_spectrum_api.py::test_spectrum_container_length_compatibility

[1-1-1-container_type0-spectrum] PASSED                            [  2%]
[1-1-1-container_type0-ipymspectrum] FAILED                        [  8%]
[1-1-5-container_type0-spectrum] FAILED                            [ 11%]
[1-1-5-container_type0-ipymspectrum] FAILED                        [ 16%]
[1-5-1-container_type0-spectrum] PASSED                            [ 19%]
[1-5-1-container_type0-ipymspectrum] PASSED                        [ 25%]
[1-5-5-container_type0-spectrum] FAILED                            [ 27%]
[1-5-5-container_type0-ipymspectrum] FAILED                        [ 33%]
[5-1-1-container_type0-spectrum] FAILED                            [ 36%]
[5-1-1-container_type0-ipymspectrum] FAILED                        [ 41%]
[5-1-5-container_type0-spectrum] PASSED                            [ 44%]
[5-1-5-container_type0-ipymspectrum] PASSED                        [ 50%]
[5-5-1-container_type0-spectrum] FAILED                            [ 52%]
[5-5-1-container_type0-ipymspectrum] FAILED                        [ 58%]
[5-5-5-container_type0-spectrum] PASSED                            [ 61%]
[5-5-5-container_type0-ipymspectrum] PASSED                        [ 66%]

Notice how not all my ipymspectrum tests are failing:

• 1-1-1-container_type0-spectrum passes,
• 1-1-1-container_type0-ipymspectrum fails, but
• e.g. 5-5-5-container_type0-ipymspectrum passes.

This indicates that the API is different between spectrum and ipymspectrum.

[kdere]

still going through the various tests. One thing I just realized was the ipymspectrum must be run in a Jupyter environment. The mspectrum can be run in a Python or IPython shell. That is probably why ipyparallel is not being found

[kdere]

for the spectrum tests, the 1-1-5 tests failed. That is because it does not make sense to have more emission-measures than temperatures or densities. in the ionTrails.argCheck() method, the size of the emission-measure is set to that of the temperature and the tests pass