pykokkos
pykokkos copied to clipboard
TST: pytest-xdist failures
As part of the effort to add multi-process testing support to runtests.py
, I encounted another issue that seems related to pykokkos
import machinery and main
blocks, even if I only tell pytest-xdist
to use a single process:
`runtests.py` diff below the fold
diff --git a/runtests.py b/runtests.py
index b1fbe36..d8afc7d 100644
--- a/runtests.py
+++ b/runtests.py
@@ -1,3 +1,4 @@
+import argparse
import os
import shutil
import pytest
@@ -9,6 +10,19 @@ cwd = os.getcwd()
shutil.rmtree(os.path.join(cwd, "pk_cpp"),
ignore_errors=True)
+
+# try to support command line arguments to
+# runtests.py that mirror direct usage of
+# pytest
+pytest_args = []
+
+parser = argparse.ArgumentParser()
+parser.add_argument('-n', '--numprocesses', type=int)
+args = parser.parse_args()
+
+if args.numprocesses:
+ pytest_args.extend(["-n", str(args.numprocesses)])
+
# force pytest to actually import
# all the test modules directly
-pytest.main()
+pytest.main(pytest_args)
Sample traceback from the testsuite via python runtests.py -n 1
:
________________________________________________________________________________________________________________________ ERROR collecting tests/test_views.py _________________________________________________________________________________________________________________________
tests/test_views.py:6: in <module>
import pykokkos as pk
pykokkos/__init__.py:17: in <module>
defaults: Optional[CompilationDefaults] = runtime_singleton.runtime.compiler.read_defaults()
pykokkos/core/compiler.py:260: in read_defaults
main: Path = self.get_main_path()
pykokkos/core/compiler.py:162: in get_main_path
return Path(self.console_main)
E AttributeError: 'Compiler' object has no attribute 'console_main'
This is not likely a high-priority matter, but worth keeping an eye on this as pykokkos
evolves. If much of the testsuite is already parallel with OpenMP we may not gain too much by distributing it along a second dimension anyway, though I was hoping to test that. I think there's an environment variable that controls the number of openmp threads used by default--not sure how/if that interacts with pykokkos
, but that variable has caused some pain in the scientific Python community in the past with stuff like OpenBLAS, etc.
@tylerjereddy can you confirm that this is working after I merged #59?
@NaderAlAwar parallel support not quite there yet I don't think: https://github.com/kokkos/pykokkos/pull/60, though single core with xdist
is "ok" I think, python runtests.py -n 1
passes for me.
So, you could probably close this, and have a separate issue for genuine parallelism vs. the specific xdist
single core support here.