pyFAI
pyFAI copied to clipboard
silx-kit
Debian packaging of 0.20.0
Hello, Jerome, I will put here all the issue found during the packaging.
during the build_man processing
ERROR:pyFAI.app.sparsify:Sparsfy requires a valid OpenCL stack to be installed
pybuild --build -i python{version}-dbg -p 3.9 -s custom -p 3.9 "--build-args=PYTHONPATH={build_dir} xvfb-run -a --server-args=\"-screen 0 1024x768x24\" {interpreter} setup.py build_man"
I: pybuild base:232: PYTHONPATH=/builds/science-team/pyfai/debian/output/source_dir/.pybuild/cpython3_3.9_dbg_pyfai/build xvfb-run -a --server-args="-screen 0 1024x768x24" python3.9-dbg setup.py build_man
There is no OpenCL available during the build so the manual page can not generated :)
Here a bunch of warning due to unclosed files
INFO:pyFAI.setup:Build man for entry-point target 'check_calib'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'detector2nexus'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'diff_map'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'diff_tomo'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'eiger-mask'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'MX-calibrate'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-average'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-benchmark'
WARNING:matplotlib:Matplotlib created a temporary config/cache directory at /tmp/matplotlib-pes52q0p because the default path (/home/salsaci/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-calib'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-calib2'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-drawmask'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-diffmap'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-integrate'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-recalib'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-saxs'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-waxs'
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
INFO:pyFAI.setup:Build man for entry-point target 'sparsify-Bragg'
ERROR:pyFAI.app.sparsify:Sparsfy requires a valid OpenCL stack to be installed
/builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3>
synopsis = self.get_synopsis(module_name, env)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
I do not install the third_party so I have this error.
what is this transformation module ?
INFO:matplotlib.font_manager:Generating new fontManager, this may take some time...
Traceback (most recent call last):
File "/builds/science-team/pyfai/debian/output/source_dir/.pybuild/cpython3_3.9_pyfai/build/pyFAI/third_party/transformations.py", line 30, in <module>
from transformations import *
ModuleNotFoundError: No module named 'transformations'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/builds/science-team/pyfai/debian/output/source_dir/./run_tests.py", line 536, in <module>
unittest.defaultTestLoader.loadTestsFromNames(options.test_name))
File "/usr/lib/python3.9/unittest/loader.py", line 220, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib/python3.9/unittest/loader.py", line 220, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib/python3.9/unittest/loader.py", line 205, in loadTestsFromName
test = obj()
File "/builds/science-team/pyfai/debian/output/source_dir/.pybuild/cpython3_3.9_pyfai/build/pyFAI/test/__init__.py", line 61, in suite
from . import test_all
File "/builds/science-team/pyfai/debian/output/source_dir/.pybuild/cpython3_3.9_pyfai/build/pyFAI/test/test_all.py", line 50, in <module>
from . import test_geometry
File "/builds/science-team/pyfai/debian/output/source_dir/.pybuild/cpython3_3.9_pyfai/build/pyFAI/test/test_geometry.py", line 54, in <module>
from ..third_party import transformations
File "/builds/science-team/pyfai/debian/output/source_dir/.pybuild/cpython3_3.9_pyfai/build/pyFAI/third_party/transformations.py", line 32, in <module>
from ._local.transformations import *
ModuleNotFoundError: No module named 'pyFAI.third_party._local'
E: pybuild pybuild:353: test: plugin custom failed with: exit code=1: PYTHONPATH=/builds/science-team/pyfai/debian/output/source_dir/.pybuild/cpython3_3.9_pyfai/build WITH_QT_TEST=False PYFAI_TESTIMAGES=testimages PYFAI_DATA=/builds/science-team/pyfai/debian/output/source_dir xvfb-run -a --server-args="-screen 0 1024x768x24" python3.9 ./run_tests.py -v --low-mem --installed
dh_auto_test: error: pybuild --test -i python{version} -p 3.9 -s custom "--test-args=PYTHONPATH={build_dir} WITH_QT_TEST=False PYFAI_TESTIMAGES=testimages PYFAI_DATA=/builds/science-team/pyfai/debian/output/source_dir xvfb-run -a --server-args=\"-screen 0 1024x768x24\" {interpreter} ./run_tests.py -v --low-mem --installed" returned exit code 13
make[1]: *** [debian/rules:40: override_dh_auto_test] Error 25
make[1]: Leaving directory '/builds/science-team/pyfai/debian/output/source_dir'
make: *** [debian/rules:17: build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
On Thu, 28 Jan 2021 00:21:14 -0800 picca [email protected] wrote:
Hello, Jerome, I will put here all the issue found during the packaging.
during the build_man processing
ERROR:pyFAI.app.sparsify:Sparsfy requires a valid OpenCL stack to be installed pybuild --build -i python{version}-dbg -p 3.9 -s custom -p 3.9 "--build-args=PYTHONPATH={build_dir} xvfb-run -a --server-args=\"-screen 0 1024x768x24\" {interpreter} setup.py build_man" I: pybuild base:232: PYTHONPATH=/builds/science-team/pyfai/debian/output/source_dir/.pybuild/cpython3_3.9_dbg_pyfai/build xvfb-run -a --server-args="-screen 0 1024x768x24" python3.9-dbg setup.py build_manThere is no OpenCL available during the build so the manual page can not generated :)
T'as pas PoCL ?
non par lors du build, vous ne vouliez pas que je dépende de pocl pour ne pas limiter le nombre d'architectures supportées.
non par lors du build, vous ne vouliez pas que je dépende de pocl pour ne pas limiter le nombre d'architectures supportées.
I think that was me for silx, but I am not so sure about pyFAI
non par lors du build, vous ne vouliez pas que je dépende de pocl pour ne pas limiter le nombre d'architectures supportées.
I think that was me for silx, but I am not so sure about pyFAI
C'est étrange car j'ai bien
Build-Depends: cython3,
cython3-dbg,
debhelper-compat (= 12),
dh-python,
python3-all-dbg,
python3-all-dev,
python3-fabio,
python3-fabio-dbg,
python3-h5py,
python3-h5py-dbg,
python3-matplotlib,
python3-matplotlib-dbg,
python3-numexpr,
python3-numexpr-dbg,
python3-numpy,
python3-numpy-dbg,
python3-pyopencl,
dans les dépendances, du coup je ne comprends pas trop pourquoi ca marche pas ???
On Thu, 28 Jan 2021 05:21:30 -0800 picca [email protected] wrote:
non par lors du build, vous ne vouliez pas que je dépende de pocl pour ne pas limiter le nombre d'architectures supportées.
I'll implement a workarround
On Thu, 28 Jan 2021 05:25:17 -0800 V. Armando Solé [email protected] wrote:
non par lors du build, vous ne vouliez pas que je dépende de pocl pour ne pas limiter le nombre d'architectures supportées.
I think that was me for silx, but I am not so sure about pyFAI
No the bug is that I did not find the time to perform a pure cython application for the jungfrau developement in addition to the GPU version. --> hence the OpenCL dependency.
On Thu, 28 Jan 2021 00:22:54 -0800 picca [email protected] wrote:
Here a bunch of warning due to unclosed files
INFO:pyFAI.setup:Build man for entry-point target 'check_calib' /builds/science-team/pyfai/debian/output/source_dir/setup.py:349: ResourceWarning: unclosed file <_io.BufferedReader name=3> synopsis = self.get_synopsis(module_name, env) ResourceWarning: Enable tracemalloc to get the object allocation traceback
This one is easy: line 3.6 in the setup.py: p.stdout.close()
I'll make a PR soon
Hi Fred, I double checked and this is what I got when generating the manual pages without pyopencl installed:
INFO:pyFAI.setup:Build man for entry-point target 'pyFAI-waxs'
INFO:pyFAI.setup:Build man for entry-point target 'sparsify-Bragg'
ERROR:pyFAI.app.sparsify:Sparsfy requires a valid OpenCL stack to be installed
(noopencl) jerome@antarctica:~/workspace/pyFAI$ ls build
build/ build-deb.sh
(noopencl) jerome@antarctica:~/workspace/pyFAI$ ls build/
lib.linux-x86_64-3.9/ man/ temp.linux-x86_64-3.9/
(noopencl) jerome@antarctica:~/workspace/pyFAI$ ls build/man/
check_calib.1 diff_tomo.1 pyFAI-average.1 pyFAI-calib2.1 pyFAI-integrate.1 pyFAI-waxs.1
detector2nexus.1 eiger-mask.1 pyFAI-benchmark.1 pyFAI-diffmap.1 pyFAI-recalib.1 sparsify-Bragg.1
diff_map.1 MX-calibrate.1 pyFAI-calib.1 pyFAI-drawmask.1 pyFAI-saxs.1
(noopencl) jerome@antarctica:~/workspace/pyFAI$ head build/man/sparsify-Bragg.1
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16.
.TH SPARSIFY-BRAGG "1" "January 2021" "sparsify-Bragg 0.21.0-dev0" "User Commands"
.SH NAME
sparsify-Bragg \- Sparsify 2D single crystall diffraction images by separating Bragg peaks from background signal.
Positive outlier pixels (i.e. Bragg peaks) are all recorded as they are without destruction.
Peaks are not integrated.
Background is calculated by an iterative sigma-clipping in the polar space.
The number of iteration, the clipping value and the number of radial bins could be adjusted.
So there is an error message telling pyopencl was mandatory but the man-page is properly generated.
Shall I demote the severity of the message from error to warning ?
About the transformations module ... I think this is not yet available as a debian package. So if you removed it, the dependency will be missing. It is available here: https://pypi.org/project/transformations/ Note that I have removes all other third-party libraries since they all become obsolete or packaged.