EDIT

I just noticed the info on the system requirements for pycsw which states that python 3.5 is supported. Strangely the PEP479 has been introduced on python3.5.

EDIT-EDIT

Even more strangely, your travis server runs the python 3.7 tests absolutely fine.

Description

While I was working on some code related to #617 I tried to run the already existing tests using

py.test

I faced some errors unrelated to the tests themself, but rather some strange StopIteration and NoneType issues:

=========================================================================================================== test session starts ===========================================================================================================
platform linux -- Python 3.7.3, pytest-3.0.3, py-1.8.1, pluggy-0.4.0
rootdir: /home/****/PycharmProjects/pycsw, inifile: 
plugins: timeout-1.2.0, flake8-0.8.1, cov-2.4.0
collecting 474 items / 1 errors
================================================================================================================= ERRORS ==================================================================================================================
_________________________________________________________________________ ERROR collecting venv/lib/python3.7/site-packages/importlib_metadata/tests/test_api.py __________________________________________________________________________
venv/lib/python3.7/site-packages/_pytest/python.py:172: in pytest_pycollect_makeitem
    raise StopIteration
E   StopIteration

The above exception was the direct cause of the following exception:
venv/lib/python3.7/site-packages/_pytest/runner.py:163: in __init__
    self.result = func()
venv/lib/python3.7/site-packages/_pytest/main.py:460: in _memocollect
    return self._memoizedcall('_collected', lambda: list(self.collect()))
venv/lib/python3.7/site-packages/_pytest/main.py:331: in _memoizedcall
    res = function()
venv/lib/python3.7/site-packages/_pytest/main.py:460: in <lambda>
    return self._memoizedcall('_collected', lambda: list(self.collect()))
venv/lib/python3.7/site-packages/_pytest/python.py:404: in collect
    return super(Module, self).collect()
venv/lib/python3.7/site-packages/_pytest/python.py:318: in collect
    res = self.makeitem(name, obj)
venv/lib/python3.7/site-packages/_pytest/python.py:330: in makeitem
    collector=self, name=name, obj=obj)
venv/lib/python3.7/site-packages/_pytest/vendored_packages/pluggy.py:745: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
venv/lib/python3.7/site-packages/_pytest/vendored_packages/pluggy.py:339: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
venv/lib/python3.7/site-packages/_pytest/vendored_packages/pluggy.py:334: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
venv/lib/python3.7/site-packages/_pytest/vendored_packages/pluggy.py:613: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
venv/lib/python3.7/site-packages/_pytest/vendored_packages/pluggy.py:250: in _wrapped_call
    wrap_controller.send(call_outcome)
E   RuntimeError: generator raised StopIteration

Many articles or posts identified the problem's source in an incompatibility between 'old python style' and python 3.7due to the PEP479 change from 2014:

• Python 3.7: StopIteration could be your next headache
• Github Bmap-Tools Issue
• Stackoverflow

As seen in the error message above, the error source is located in the pytest package, which is still sticking to version 3.0.3 according to the requirements-dev.txt.

Removing this version constraint/changing it to the latest release, as well as for the related plugins underneath, fixes this error for me and the tests are starting.

Environment

• operating system: Deb10 Buster
• Python version: 3.7
• pycsw version: 2.4.2
• source/distribution
  • [x] git clone
  • [ ] DebianGIS/UbuntuGIS
  • [ ] PyPI
  • [ ] zip/tar.gz
  • [ ] other (please specify):
• web server
  • [ ] Apache/mod_wsgi
  • [ ] CGI
  • [x] other (please specify): simple wsgi for dev purposes

Steps to Reproduce

1. Set up venv using python 3.7
2. Set up fresh cloned source code
3. Try to run py.test

Additional Information

None

The harvesting pytest checks gave me a nice headache. There are 23 tests - all from the harvest test suite - which do not run properly.

Since I knew from your travis server, that the tests have to run fine I took a look on your test calls over there and you seem to skip the harvesting tests using

 -k 'not harvesting'

Not gonna critizice, I just would like to know why is that? I get some 404 or Bad Gateway errors on the tests you are skipping there.

The reason why we are skipping the harvesting tests is because they deal with remote resources, which, from a continuous integration perspective, may or may not be available. We can/should mock out the harvesting tests IMHO to run them with consistency.

So, while tests are running fine on Travis (for Python 3.5, 3.6 and 3.7), they fail on your local dev machine, using the same requirements-dev.txt? Indeed seems weird.

From what I can see, Travis is running Python 3.7.6 while you have v3.7.3 - I would think that would not be so relevant though.

I've made a quick pytest run on my local machine (Ubuntu 20.04, with Python 3.8.2) and I could run all non-harvest tests without any problem.

cd ~/dev/pycsw
python3 -m venv ./test_venv
source test_venv/bin/activate
pip install -r requirements-dev.txt
pip install -e .
cd tests
pytest -v -x -k 'not harvesting'
...
=== 438 passed, 36 deselected in 30.59 seconds ===

# running all tests, including harvesting
pytest -v
...
=== 21 failed, 453 passed in 144.53 seconds ===

Could this be something on you local env?

Regarding not running the harvesting tests on Travis (which we should track as a different issue), as @tomkralidis says, it is not something we are too proud of, and the eventual plan is to do a proper mock of these external services. Not having these constantly running in CI is a perfect way to let them rot, as can be attested by my own local test run, which shows 21 fails, all are harvesting-related. In fact, the same could be said about us not running Travis tests against a postgresql database, which I think is more representative of a typical production environment.

Seems like a good place for some extra help, from someone who is test-savvy, if you catch my drift :wink:

Anyway, jokes aside, I think this is definitely something that should be improved upon.

Hey guys,

thanks for your replies! Since I moved on a while after opening this issue, I do not have the same setup right now to check again, sorry. But when it runs on all your machines fine - beside the harvesting tests - , it surely must be something on my machine.

Seems like a good place for some extra help, from someone who is test-savvy, if you catch my drift wink

Oh dear, look at the time, I will miss that really important thing that I have to attend if I won't go now. :smile:

Thanks for your time and good luck on solving the remote test data task ;)