ADIOS2
ADIOS2 copied to clipboard
ornladios
difficulties building rpm package for adios2
I was attempting to get an ADIOS2 science package for openSUSE. I thought it could just be as simple as swapping out autoconfig for cmake, but ran into some issues.
It is still a rough work-in-progress, but I've attached a tar file with specs etc. The main issue seems to be with the different ideas of the file-system or build root. The RPM build is within a BUILDROOT/ structure, which means that stuff like /usr/lib64/cmake/... are actually found below that.
I figured there must be a kitware guy (eg, @chuckatkins) who can resolve this in an instant.
Hi @williamfgc - found any time to play with this? I would really like to get this going well before OpenFOAM-1912.
@olesenm thanks for the reminder @chuckatkins did you have a chance to look into this issue and provide a timeline for @olesenm . Thanks. @olesenm on a side note, we are working towards the 2.5.0 release due this Monday, I want to touch basis with you perhaps next week as I am trying to build OpenFoam on summit with the adios2 submodule.
Hi @olesenm , I haven't worked with RPM systems in a while (Ubuntu mostly), which system are you using?
Hi @williamfgc - I use openSUSE, but the problems will likely be similar with RedHat. The tar file attached in the original message is what I've be working with.
@olesenm I can try to reproduce it with your spec file on a suse or rhel system but if you can just post the error you're getting then I may be able to just quickly debug it here.
OK I'll add the log tomorrow morning
Here is one of the log files (after trying different things and giving up). stderr.txt
@olesenm I can see that the Adios package in the science repo is working well now for a while. Shall we close this issue?
@olesenm I can see that the Adios package in the science repo is working well now for a while. Shall we close this issue?
The science repo has adios (ie, adios1) but the problems I had were with adios2. I haven't taken an further attempts at this, so AFAIK the problems are still there unless someone confirms otherwise
@olesenm oh right! Thanks for pointing it out :smile: . Actually I could possibly help you with this, I am also part of Kitware and my main linux distro is OpenSuse. I also maintain some other Kitware projects at the Science repo.
The initial given links are now broken, any chance you could share with me the current attempt to get ADIOS2 rpm for OpenSuse?
Hi @vicentebolea
No updates, but if you grab the tar file from the first comment, that should be enough to get started. You will likely have to fiddle with the multibuild, but that's it.
Nothing here? https://github.com/ornladios/ADIOS2/files/3560422/adios2-rpm-build.tar.gz
When I open it, it appears to be an empty tar ball
On Fri, Jul 9, 2021 at 10:58 AM Mark Olesen @.***> wrote:
Nothing here? https://github.com/ornladios/ADIOS2/files/3560422/adios2-rpm-build.tar.gz
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ornladios/ADIOS2/issues/1710#issuecomment-877249616, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHFOFS2CNAOZUTD5NITOXLTW4FA5ANCNFSM4ISOIRSQ .
-- Vicente Adolfo Bolea Sanchez R&D Engineer 518-881-4444 Advancing the frontiers of understanding by developing innovative open source software platforms and integrating them into research, processes, and product
Still very weird - I've sent by email to your gmail account (address from github).
@olesenm I have spent sometime working with the adios2 rpm package, I create an experimental package for Adios2 2.7.1, I introduced many changes to the original RPM and have also created a few patches and I disabled the OpenHPC flavors (I do not have the mojo to work with that). I have also brought to the team ideas into what to provide in each of the flavors and so on.
The results is here: https://build.opensuse.org/package/show/home:vicentebolea/ADIOS2
I took the liberty to add Tumbleweed and Fedora_34, so if in the future somebody wants to submit the package to their repos he has some work done here.
We are also exploring possible ways to move this upstream (e.g. this repo) and only have in the OBS the _service file that pulls everything from the repo.
Hi @vicentebolea - haven't tested your changes (just got back from holidays), but they look like they are in the right direction. Might still need a bit of jigging for handling various openmpi major versions (eg, multi mpi build openfoam ).
The Fedora addition makes sense, but there is a major "gotcha" to watch out for: on newer Fedora (and RH8 too I think), they handle the MPI-related quite differently.
Take a look around lines 350-360 of the openfoammpi.spec file, for the system location of the mpi-dependent files. These are hooked into at line 417 with the final owership given at line 477. If you miss on these parts (Fedora and newer RH), you get a package without the properly resolved dependency of the mpi library .so tag.
Great insights @olesenm for Fedora rpm. As for the different OpenMPI versions, yeah I skipped the OpenHPC mpi flavors due to not being familiar with OpenHPC but its on my ToDo list.
I guess that after we have a consistent Fedora RPM we can possibly submit it to the Fedora official repos?
I skipped the OpenHPC mpi flavors due to not being familiar with OpenHPC but its on my ToDo list.
It looks scary at first, but is not that bad.
I guess that after we have a consistent Fedora RPM we can possibly submit it to the Fedora official repos?
The first intermediate solution is to build for Fedora/centos on the copr system which gives you a chance to hammer out any wrinkles, get things tested and properly reviewed before submitting for "regular" distribution.
- The copr builds for OpenFOAM: https://copr.fedorainfracloud.org/coprs/openfoam/openfoam/
Using the copr builds, you can verify that the generated rpms have the correct MPI dependencies resolved and iterate towards correct result. Feel free to contact me by email if you need any inputs (I'm not a Fedora/RedHat user, but have gone through the hoops).
After everything is polished, can submit for review. However, if I remember correctly, the Fedora team wants spec files that are purely for Fedora, which means stripping out all any SuSE (perhaps even RedHat) specific parts. In other words, undoing part of your current work of generalizing things :(
You might have to settle for a master version of the spec file that handles various flavours, and mark it up with comment tags (eg, #<suse> ... #</suse>) that you can use to sed out later to extract a pristine Fedora-only version.
https://build.opensuse.org/package/show/science/adios2, the spec pkg finally made it to the Science project. I will submit it to Factory soon, unfortunately we miss it to make it to the Leap 15.5
Closing as we already have the adios2 package in the science project :)
Closing as we already have the adios2 package in the science project :)
This is cool. Anything with copr, Fedora, EPEL coming? Debian/Ubuntu next?
We had an adios2 repo on Debian: https://salsa.debian.org/science-team/adios2 but no activity was reported in the last 2 years.
@williamfgc note that this issue was for the RPM package for the OpenSUSE Science project, that thing is done. We can open new issues for each of the dists.