psydac icon indicating copy to clipboard operation
psydac copied to clipboard

Pyccelize evaluate_dofs to speed up tests

Open kvrigor opened this issue 10 months ago • 3 comments

test_derham_projector_3d and test_3d_commuting_pro_3 collectively take ~10mins to run. This PR tries to drastically reduce this long test time.

#
# Run tests on 4 CPUs just like GitHub runners
#
$ pytest -n 4 psydac/feec/tests/test_global_projectors.py -k "test_derham_projector_3d"
========================== test session starts ==============================
platform linux -- Python 3.12.8, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/user/psydac
configfile: pytest.ini
plugins: profiling-1.8.1, anyio-4.6.2.post1, xdist-3.6.1
4 workers [18 items]
..................                                                                                            [100%]
=================== 18 passed in 361.08s (0:06:01) ===========================

$ pytest -n 4 psydac/feec/tests/test_commuting_projections.py -k "test_3d_commuting_pro_3"
========================== test session starts ==============================
platform linux -- Python 3.12.8, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/user/psydac
configfile: pytest.ini
plugins: profiling-1.8.1, anyio-4.6.2.post1, xdist-3.6.1
4 workers [16 items]
................                                                                                              [100%]
========================= 16 passed in 219.16s (0:03:39) =====================

These slow tests call evaluate_dofs_*() multiple times. A single call to evaluate_dofs_*() is already costly due to usage of deeply nested for loops: https://github.com/pyccel/psydac/blob/66205a5978abd2397a3ac9e2e046013993ad6857/psydac/feec/global_projectors.py#L772-L778

https://github.com/pyccel/psydac/blob/66205a5978abd2397a3ac9e2e046013993ad6857/psydac/feec/global_projectors.py#L781-L818

The evaluate_dofs_*() functions seem to be good candidates for pyccel-ization. Problem is, evaluate_dofs_*() can take a Python function as an input, and this is not yet supported in pyccel. I'll have to find another way to speed this up.

kvrigor avatar Feb 11 '25 16:02 kvrigor

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
:white_check_mark: +0.02% :white_check_mark: 100.00%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (66205a5978abd2397a3ac9e2e046013993ad6857) 30663 18399 60.00%
Head commit (3aae8b2ccb1c84afb0e04808d8d77109fa162896) 61356 (+30693) 36828 (+18429) 60.02% (+0.02%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#458) 170 170 100.00%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

codacy-production[bot] avatar Feb 11 '25 17:02 codacy-production[bot]

Pyccelized modules can call Python functions, if these are also Pyccelized...

yguclu avatar Feb 12 '25 15:02 yguclu

Pyccelized modules can call Python functions, if these are also Pyccelized...

That's the catch -- those Python functions (i.e. f-evals in our case) should remain black box from C/ForTran perspective, and thus should not be pyccel-ized. Not optimal, but I think preserving the original f-eval (i.e. not pyccel-izing) is more important. There should be a way to bind Python methods as a function pointers in C . I asked this question in the Pyccel repo: https://github.com/pyccel/pyccel/discussions/2213#discussioncomment-12169877

kvrigor avatar Feb 12 '25 16:02 kvrigor