Update build instructions for macOS

[btalamini]

Use MacPorts.

[codecov-commenter]

Codecov Report

Merging #770 (e75351c) into develop (cffd219) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #770   +/-   ##
========================================
  Coverage    94.86%   94.86%           
========================================
  Files          140      140           
  Lines         8970     8970           
========================================
  Hits          8509     8509           
  Misses         461      461           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[samuelpmishLLNL]

a built copy of these documents is available here

[samuelpmishLLNL]

I'm struggling to follow this guide. There are useful parts, but there enough missing steps that it's not clear what I should be doing, and I'm just at the point of trying to install and symlink the stuff from MacPorts.

sam@Sams-Air darwin_monterey % sudo port select mpi openmpi-clang-fortran
Selecting 'openmpi-clang-fortran' for 'mpi' failed: The specified version 'openmpi-clang-fortran' is not valid.

---

The packages.yaml also contains entries of libraries that were not instructed to be installed by MacPorts (tar, perl, netlib-lapack), where should I get these?

I'm dreading having to manually go into that packages.yaml file and manually edit the specific directories and versions of each of the 17 packages. Why do I have to do this?

---

instead of

The operating_system field should be set to bigsur for macOS 11. Refer to the spack documentation for other versions of macOS.

we should inline some of the options users are likely to need, since spack's documentation is unintuitive to navigate:

"mojave", "catalina", "bigsur", "monterey", "ventura"

These are the values scraped from spack's readthedocs, but I can't confirm if they work or not, because I can't get to that step yet.

Similarly, I think "aarch64" is the appropriate entry for "target" on the newer machines, but I couldn't find good information about this from the spack documentation or check that it works.

---

I've installed several gigabytes of compilers and tools I already have, since this guide is only written from the perspective of MacPorts (in my experience, brew is the more commonly-used tool). I was alarmed to see that port install openmpi-clang12 went on to install 3 separate versions of LLVM, several versions of clang, etc.

---

This comment isn't directed at Brandon or the content of this PR, but: man this TPL build system is infuriating. I've spent several hours today trying to follow these steps (and this is after the additional clarification provided by the PR) and have very little to show for it. I can't imagine anyone not being paid by LLNL actually jumping through all of these hoops.

[btalamini]

I'm struggling to follow this guide. There are useful parts, but there enough missing steps that it's not clear what I should be doing, and I'm just at the point of trying to install and symlink the stuff from MacPorts.

sam@Sams-Air darwin_monterey % sudo port select mpi openmpi-clang-fortran
Selecting 'openmpi-clang-fortran' for 'mpi' failed: The specified version 'openmpi-clang-fortran' is not valid.

I'm in agreement that this deserves editing for clarity. There was essentially nothing about the Mac build before, so I did my reasonable best effort at capturing everything I had to do. Let's capture the parts that gave you trouble and add as necessary. I'm willing to help on Teams to work through the rough patches.

The packages.yaml also contains entries of libraries that were not instructed to be installed by MacPorts (tar, perl, netlib-lapack), where should I get these?

On my system, getting the ports I documented got the other things as dependencies. Relying on this is probably not wise, since MacPorts could change their packages. On the other hand, filling in the others will require identifying the port names for everything, and as you note below, there's an awful lot to specify.

I'm dreading having to manually go into that packages.yaml file and manually edit the specific directories and versions of each of the 17 packages. Why do I have to do this?

Only the spack maintainers know the answer to this one. Seems like it should be possible to specify /opt/local as a global prefix, and then only to override the few that are different. May be worth opening an issue on spack.

instead of

The operating_system field should be set to bigsur for macOS 11. Refer to the spack documentation for other versions of macOS.

we should inline some of the options users are likely to need, since spack's documentation is unintuitive to navigate:

"mojave", "catalina", "bigsur", "monterey", "ventura"

These are the values scraped from spack's readthedocs, but I can't confirm if they work or not, because I can't get to that step yet.

Similarly, I think "aarch64" is the appropriate entry for "target" on the newer machines, but I couldn't find good information about this from the spack documentation or check that it works.

Agree with these, but couldn't test them, and didn't want to say more than I knew to be true.

I've installed several gigabytes of compilers and tools I already have, since this guide is only written from the perspective of MacPorts (in my experience, brew is the more commonly-used tool). I was alarmed to see that port install openmpi-clang12 went on to install 3 separate versions of LLVM, several versions of clang, etc.

I won't argue the relative merits of one Apple package manager versus another; people can use whatever they want. I just wrote down the process using the one I know how to use. Again, happy to assist with a Homebrew-based build (but I don't have the time to try a Homebrew build on my own just for the sake of the doc page).

[goxberry]

Similarly, I think "aarch64" is the appropriate entry for "target" on the newer machines

On an M1 Mac, I'd use arch=m1.

but I couldn't find good information about this from the spack documentation or check that it works.

See https://spack.readthedocs.io/en/latest/basic_usage.html#support-for-specific-microarchitectures for supported values of arch.

[goxberry]

I'm dreading having to manually go into that packages.yaml file and manually edit the specific directories and versions of each of the 17 packages. Why do I have to do this?

Only the spack maintainers know the answer to this one. Seems like it should be possible to specify /opt/local as a global prefix, and then only to override the few that are different. May be worth opening an issue on spack.

Because that's usually not enough information. In addition to packages and versions, introspection also needs to be done on variants. This issue has come up before with Spack maintainers, which is why the spack external find command exists. It doesn't work for every package -- logic must be added to a package to do the required introspection -- but this problem is known, and is being addressed for some of the packages at the ~~root~~ leaves of the dependency DAG (e.g., MPI implementations).

[samuelpmishLLNL]

I'm dreading having to manually go into that packages.yaml file and manually edit the specific directories and versions of each of the 17 packages. Why do I have to do this?

Only the spack maintainers know the answer to this one. Seems like it should be possible to specify /opt/local as a global prefix, and then only to override the few that are different. May be worth opening an issue on spack.

Because that's usually not enough information. In addition to packages and versions, introspection also needs to be done on variants. This issue has come up before with Spack maintainers, which is why the spack external find command exists. It doesn't work for every package -- logic must be added to a package to do the required introspection -- but this problem is known, and is being addressed for some of the packages at the ~root~ leaves of the dependency DAG (e.g., MPI implementations).

It seems untenable that our build system requires a human to manually specify that readline is at exactly version 8.1.2.000. The argument that this sort of thing is necessary doesn't make any sense, since every other package manager I've used figures this sort of thing out automatically.

On a related note, how am I supposed to know how to format the library version in that packages.yaml file? When I install, say, unzip with MacPorts, it tells me that it's installed version unzip-6.0_4. The packages.yaml file is formatted as [email protected]. Does the _4 matter? How am I supposed to know when to truncate the version string, and by how much? By contrast, entries like [email protected] have totally different formatting. How many trailing zeroes are necessary? Why does perl have a "v" in its version prefix and other libraries don't?

If we're expecting users to do this manually 17 times, we have to actually explain what is expected of them.

---

In contrast: the other day, I was looking for a tool to help me inspect what was taking up space in our test binaries. A quick google search led me to bloaty, which I had never heard of before. My experience with bloaty went something like this:

1. They have a github page. I clone their repository with --recursive since I saw it has a .gitmodule file in the repo.
2. The project contains a CMakeLists.txt file, so I just configure cmake and build it.
3. The dependencies and main executable build successfully in ~30 seconds.
4. bloaty --help helps me discover which flags to use in order to find the information I need.
5. Identify that serac's binaries are 95% debug symbols from TPLs installed by spack.

I went from having never heard of this tool, to building it (and all of its dependencies) without needing any documentation, to using the tool to solve the problem I had, and it all took a matter of minutes. In serac, it's taking me hours just to process one of many steps in a guide to build just the TPLs, let alone the project itself. And I'm pretty sure I've done this step wrong, because it's completely unclear what needs to actually happen.

Why is it the case that building serac is so painful, and yet bloaty builds so effortlessly?

[goxberry]

It seems untenable that our build system requires a human to manually specify that readline is at exactly version 8.1.2.000. The argument that this sort of thing is necessary doesn't make any sense, since every other package manager I've used figures this sort of thing out automatically.

Most package managers I've used don't try to integrate with software installed outside the package management system. re: version detection, that is typically the province of build configuration systems. But Spack is trying to leverage packages installed external to it, so it needs enough information to determine whether package prerequisites are satisfied by those external packages.

Attempting to use external packages is optional. You don't need to do it if you're willing to have Spack build more software.

[jamiebramwell]

This thread is getting a bit heated. I would encourage revisiting our code of conduct, particularly the statements about respectful discourse and empathy.

[white238]

@goxberry Thanks for the input! It's good to know that a lot of these could have been found with spack external find. I am trying to move to environments as quickly as possible which will clean up a lot of these difficulties i think but as usual other things pop up the priority tree.

[white238]

Please remember when reviewing these steps that this was a first cut at documenting what we had to do to get a working build at all. The intention was to document these steps so we didn't lose found knowledge then iterate to improve the process.

[goxberry]

I've tried to add some additional comments where parts of config files can be automatically generated using Spack commands. All of the comments assume use of whatever the commit of the development branch was at the time I made those comments.

I agree with @samuelpmishLLNL's point that asking users to manually construct more than small snippets of these files -- like the first several lines of packages.yaml -- is probably too much, even though I can appreciate the time and effort that goes into writing up an install guide like the one in this PR. In addition, manually writing these files is likely to be a challenge to maintain for both users and Serac developers. While I personally don't have the time to work through the directions and figure out the right combinations of flags and arguments needed to automatically generate the configuration files, I think doing so has the potential to address some of @samuelpmishLLNL's concerns, and it may make the install guide easier to maintain in the process. I also think these commands would address @btalamini's point that users should be able to supply an install prefix so that Spack can detect these externals.

For the packages that can't be detected automatically, is dropping them from packages.yaml an option? The increase in TPL build times could be worth avoiding the hassle of trying to debug manually-created configuration files.

[goxberry]

@white238 No problem! I agree; environments have the potential to clean up build instructions significantly.

[btalamini]

I'm struggling to follow this guide. There are useful parts, but there enough missing steps that it's not clear what I should be doing, and I'm just at the point of trying to install and symlink the stuff from MacPorts.

sam@Sams-Air darwin_monterey % sudo port select mpi openmpi-clang-fortran
Selecting 'openmpi-clang-fortran' for 'mpi' failed: The specified version 'openmpi-clang-fortran' is not valid.

Sam, Alex caught the error here. The command had a typo in it. I'm sorry about that.

[adrienbernede]

First of all, thank you @btalamini for this work. It is quite a tough mission to explain how to build Serac on MacOS, not to mention that there are now 2 possible architectures.

Earlier this year, I tried to install Serac on my M1 iMac, and failed miserably at clingo install. I was expecting the challenge to be within the packages.yaml, because I knew about the tough points mentioned above.

I can understand @samuelpmishLLNL frustration, although I wanted to share a quick answer to his comments:

• I agree that there should be a quick way to install a usable target of Serac on any system, for a quick getting starting at running Serac.
• Serac being an HPC code, both fine grain configuration of the installation (and dependencies) as well as reproducibility are paramount.

What I am trying to say is that we describe here the workflow targeting the second point. For pure, super simple installation, maybe another technology (container?) should be considered / described.

Now for the PR review: I am a lazy sequential reader. For a quick-start guide I am looking for actionable information for my specific context. I would like to suggest that, in a subsequent PR, this documentation has clearer partitions between say "LC", "Mac OS", "Windows" contexts and maybe sub-partitions "user quick install" and "developer controlled install".

Thanks for reading me :)

Note: admittedly, I haven’t been through the process described in this PR yet.

[btalamini]

@white238, I'd like to get your approval / additional comments on this, since it's your bailiwick. Would you mind taking another look at your earliest convenience?

[white238]

Approved.... sorry I am still catching up.

[btalamini]

No worries. We can always amend the instructions later.

From: Chris White @.> Reply-To: LLNL/serac @.> Date: Monday, September 12, 2022 at 11:12 AM To: LLNL/serac @.> Cc: "Talamini, Brandon Louis" @.>, State change @.***> Subject: Re: [LLNL/serac] Update build instructions for macOS (PR #770)

Sorry I am still catching up.

— Reply to this email directly, view it on GitHubhttps://urldefense.us/v3/__https:/github.com/LLNL/serac/pull/770*issuecomment-1244116316__;Iw!!G2kpM7uM-TzIFchu!kwu-1CAo08kt3VPWi8eFx4Qziy3-AZ0p-2JXHbhe5-wKzCOOqD7gA2FJQ3XsI81sGA$, or unsubscribehttps://urldefense.us/v3/__https:/github.com/notifications/unsubscribe-auth/AHLCWWUFFZK27FEUUCABUI3V55XBNANCNFSM56XUKQJQ__;!!G2kpM7uM-TzIFchu!kwu-1CAo08kt3VPWi8eFx4Qziy3-AZ0p-2JXHbhe5-wKzCOOqD7gA2FJQ3US_6wyvQ$. You are receiving this because you modified the open/close state.Message ID: @.***>