AdvancedPhotonSource
Usr meson-python to package GSAS-II
Suggest a way to compile C, Fortran and Cython files for GSAS-II and install them together with Python files.
There are no C++ files --> why is there a dependency on pybind11 ?
Not all extensions have been converted.
The way the Fortran binding is currently polluting the source tree which is not very nice.
Only linux has been tested so far.
This is very much of interest. My plan was to move to meson for py3.12, and was waiting for help from someone more experienced with meson. Could you expand on the source tree issue?
Well, it is not really an issue but it is not as clean as one would expect:
To compile/bind the fortran code, one needs to call f2py with the proper options which creates 2 additional files, one fortran and the other one C. Then to compile all 3 files together to build the python extension. This is currently done in the source folder, leaving 2 computer generated files in the source folder, moreover untracked in git.
The documentation of f2py (https://numpy.org/doc/stable/f2py/buildtools/meson.html) suggests to use the custom_target() meson function which generates the 2 binding files in the build directory and then the compilation of the 3 files (2 in build directory and one in the source directory) fails. I did not manage to get it working, or maybe I did not try long enough. One option would be to copy the source fortran file to the same place as the binding files and compile them together in the build directory.
I have opened a PR to @kif 's branch with some additional meson work.
scipy has a nice helper for building with f2py that could probably be borrowed.
I've opened a second pull request to @kif 's branch that does three things:
- removes the need for any mutation of
sys.path(most of the churn is making things that are sub-modules be imported as sub-modules) - cleansup / extends the meson config a bit
- adds pixi configuration
To try out the pixi config get a copy of pixi ( https://prefix.dev do what they say for your platform. it is in brew so brew install pixi works). If you then cd into pixi
cd pixi
pixi run build # <- this will make a conda environment with all the deps, build, and install gsas-ii
pixi run ui # <- assuming the build step is done, this will launch the UI
For users who want to stay up-to-date with the repo the workflow would look like
git pull
pixi run build
pixi run ui
and being able to notify users of updates would be using git fetch + some git cli work + UI elements.
Once we get gsas-ii building on CF, users who want to have stable versions, would use what ever they want to manage that in environments they fully control.
I think this solves all of the problem discussed in #38 using tools that someone else maintains! We have already made a big bet on conda-forge at NSLS-II and are starting to look very seriously at pixi so if these boats go down, I'm already on the hook to fix/replace them 😉 .
With a fair amount of help/hints from Ralf Gommers it will build on windows (via pixi run build-win).
to reproduce the #123 with this branch
pixi run build # build-win
pixi run python -c 'import GSASII.pyspg as pyspg; pyspg.sgforpy("P 21 21 21")'
@briantoby I sorted out how to fix the space group issue: turn off bounds checking in fortran. I do not know much fortran, but this suggests that there is a latent bug someplace, but removing the flag for now unblocks us.
@kif I discovered I can push to your branch and took the liberty of doing so to not make you a bottle neck
I don't think pybind11 should be needed. We are not using c++ and I'd prefer we use more c, but not c++. Libraries dependencies in the scons compilation were never cleaned up, but I did that here, also added a build for the LATTIC.f (NIST*LATTICE) program.
I'm confused. I pushed 06f754614764f85dd6ad71000136a1a1a7b02a95 with some minor changes, but I am not seeing it here. Where did it go?
@kif I discovered I can push to your branch and took the liberty of doing so to not make you a bottle neck
I am perfectly aware a reviewer can mangle my remote branch, it is bug (or not) of github, we discovered it with silx a long time ago.
By the way, I appreciate all the local imports. Huge refactoring indeed.
I am perfectly aware a reviewer can mangle my remote branch, it is bug (or not) of github, we discovered it with silx a long time ago.
Even though GH allows it, I tend to always ask permission first!
@briantoby It looks like you pushed a new branch named '40' with that commit, I'll move it to this branch.
I have also added environments to pixi's config so that we can easily run in different versions of python:
pixi run -e py313 build
pixi run -e py313 ui
py310, py311, py312, py313 are configured
After a phone discussion with @briantoby the short term plan is:
@tacaswell will:
- add
GSASIIxyz.pytop-level modules that re-export the correct sub-module + warns as a back compatibility for scripts - push this branch to upstrteam as
develop - will manage forward-porting any commits that land on
master - selectively backport commits from
developtomasteras required - investigate using entry points or a single (user-configurable) path for user-supplied import/export classes
@briantoby will:
- get some examples of scripts that should keep working
- start working primarily against
develop - identify examples of third-party importers/exporters that are in use
This discussed but no action now:
- move to not-Python files for writing user configuration (yaml or toml?)
- renaming the sub-modules to drop redundant
GSASIIfrom names - need installation/use documentation.
When we are satisfied that there is a nice workflow for non-programming users we will rename develop to main and then in ~6mo retire master.
How does one launch the GUI for this branch when working from git? This works but is clumsy:
(py311) G2 py311 GSASII % python
>>> import GSASII
>>> import GSASII.GSASII
>>> GSASII.GSASII.main()
GSAS-II binary directory: ...
I'm not sure the GSASII.py file is needed after this refactoring. Its purpose was to have a short file that was used to start the GUI.
With the latest round of commits I am able to run the refinement from the second (https://advancedphotonsource.github.io/GSAS-II-tutorials/CWNeutron/Neutron%20CW%20Powder%20Data.htm) tutorial end-to-end and get very close numbers to refinement at the bottom.
I think that is enough confirmation that this works to move ahead with moving this branch to upstream as develop
On Wed, 08 Jan 2025 09:56:55 -0800 Brian Toby @.***> wrote:
How does one launch the GUI for this branch when working from git? This works but is clumsy:
(py311) G2 py311 GSASII % python >>> import GSASII >>> import GSASII.GSASII >>> GSASII.GSASII.main() GSAS-II binary directory: ...
one could expose "main" from the top level GSASII module: in GSASII.init:
from .GSASII import main
like this it would become:
python -c "import GSASII;GSASII.main()"
If you are looking for a way to develop and run the code without installation, you should use editable installs: https://mesonbuild.com/meson-python/how-to-guides/editable-installs.html
Alternatively, I have a "bootstrap" script that does this in pyFAI but it is not mainstream, other alternative also exist.
pip install -ve . --no-build-isolation works as expected now. I also added pixi tasks to automate it.
I played some silly games to have a merge commit and get this PR to close as merged (rather than closed).