macOS support

[pagrubel]

Unable to build on mac laptop with OSX version 10.12.6:

Below is version information of my compiler:

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 9.0.0 (clang-900.0.37) Target: x86_64-apple-darwin16.7.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Here is the output from the make command: $ make printf '%s+%s%s\n'
$(cat VERSION)
$(git rev-parse --short HEAD)
$(git diff-index --quiet HEAD || echo '.dirty')
> VERSION.full echo "#define VERSION "$(cat VERSION.full)"" > bin/version.h echo "version () { echo 1>&2 '$(cat VERSION.full)'; }" > bin/version.sh cd bin && /Library/Developer/CommandLineTools/usr/bin/make SETUID= all cc -std=c11 -Wall -c -o ch-run.o ch-run.c ch-run.c:11:10: fatal error: 'argp.h' file not found #include <argp.h> ^~~~~~~~ 1 error generated. make[1]: *** [ch-run.o] Error 1 make: *** [all] Error 2

[reidpr]

Patricia, thanks for the new issue!

We have not previously considered OS X because it doesn't have the namespace system calls we use to run containers.

Could you tell us more about your use case and the workflow you envision? If we can be persuaded it fits Charliecloud's vision, then we'd love to see a PR.

Regarding the specific compile failure above, that's because we use the argp functions in glibc. I don't know how one might make them available on OS X.

[pagrubel]

We'd like to be able to build the container on mac and run them on the HPC systems.

[reidpr]

OK. Can you explain this use case in more detail? I'm still a little confused because even if you can successfully build images, you won't be able to run them, even for a sanity check.

Alternately, you can run Charliecloud in a virtual machine on your Mac: https://hpc.github.io/charliecloud/virtualbox.html. It works quite well, and since you're at LANL I can provide a pre-built image file if you like (e-mail me internally and I'll send you the link).

[pagrubel]

I will use a platform like Darwin to test for use case. I will look into the virtual case as well as seeing if I can solve this problem.

[le-ander]

Hey there,

I was looking into installing charliecloud on mac as well in order to create tarballs from my docker images using ch-builder2tar which I can then upload to the server running charliecloud.

As the VM solution seems like quite a bit of overhead for this purpose, I was wondering if tarballs created with docker save would be compatible with charliecloud's ch-tar2dir?

Thanks a lot in advance!

[j-ogas]

Hello @le-ander! ch-run needs a flattened filesystem tree (directory or squashfs).

docker save produces a tarred repository, which contains all parent layers and corresponding meta data (hash directories, layer tarballs, json, etc.), which wouldn't work:

$ docker save mpihello-openmpi -o mpihello.tar
$ ch-tar2dir mpihello.tar /tmp
creating new image /tmp/foo
 845MiB 0:00:01 [================================================================>] 100%
[...]
/tmp/mpihello unpacked ok
$ ls -lh /tmp/mpihello
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 09b4a904061cbcccd9ba90ad2f1755f61f4c3fa45a0ca8bc5ccbb60adf1c2a84
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 0a61e1722b1d139db28a3a20d6f9a3d6d1d9645d7844bc1ba8d72693b7cb685e
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 27d7818c590444f92d885f0c01ca42ce5b1f1ef77c146e092c29c7063131de74
-rw-r--r--.  1 jogas jogas 7.6K Aug  5 12:16 32f94b26f508604afe81caf1bcba9944ed943eca690596b512a379fa21040162.json
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 3e42179c6482b84dc0f5d9812be3ff01011a93b0a12873421463d5cdf0bb0637
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 424096783465c9032ef00f33ec6a8f08270d77764faa83fc69e77f7bbca54368
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 43a058b250cae44858c3e5f320fa7174f3a54515329c9930149a964e656034ce
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 4a23901f0227b15b1251a9785013cec0b9ec5d0d9708dee25e29752f55f84c4c
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 5f01a031b1e5eb27e34892ca394ab3f42cdcca67223b61734a5f76aad732b185
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 63a9ea169652012829470065feb7942179d81f91ae3425392a6f5f8ddeb976de
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 6d0e8910d48c8e97857e732dd1f4d79d2fe10ae0921de15726952e8fdfffb109
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 793dcf1cfdbcc7df30f5fb3177e076aa1c27b683edd1360467cab132b607c0b1
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 90d211418820b7f7fb5d8b7d3c676740f5e344aeb7365e5e3135bf3f9f9ba7d0
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 9e29d33324e16165d74d8cb694abeaf67455c7da26365ecc8a55df9f789e2b8c
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 ae1b39a037869d761dd41a3e389253b030b0ea5e765be8093bcc0627ca2a53fa
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 b45bf41abcdc64d8239ee008416d229e156930d24c82b46e185b50f8efc4d369
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 b543d5dac9968560ebb880c108b0e47b8eea57b73e0803d7627681560990f548
drwxrwxr-x.  2 jogas jogas    6 Aug 14 08:01 dev
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 e45ed4b43f57653d43d38404d0314787f68248694449fbf93c12a478657dbc84
drwxrwxr-x.  2 jogas jogas   38 Aug 14 08:01 etc
drwxr-xr-x.  2 jogas jogas   50 Aug  5 12:16 fac42474ae32068a2f8103e0fde3be219392d90ec4f4d1c45de670e69ab68402
-rw-r--r--.  1 jogas jogas 1.5K Dec 31  1969 manifest.json
drwxrwxr-x. 12 jogas jogas   96 Aug 14 08:01 mnt
drwxrwxr-x.  2 jogas jogas    6 Aug 14 08:01 proc
-rw-r--r--.  1 jogas jogas   99 Dec 31  1969 repositories
drwxrwxr-x.  2 jogas jogas    6 Aug 14 08:01 sys
-rw-rw-r--.  1 jogas jogas   40 Aug 14 08:01 WEIRD_AL_YANKOVIC

If you wanted to create charliecloud compatible tarballs (flattened filesystem images) , you'd want to use docker create in conjunction with docker export:

$ sudo docker export $(sudo docker create --read-only mpihello-openmpi) > mpihello.tar
$ mkdir mpihello && mv mpihello.tar mpihello
$ cd mpihello && tar xf mpihello.tar
$ mv mpihello.tar ../
total 12K
drwxr-xr-x.  2 jogas jogas 4.0K Aug  5 11:24 bin
drwxr-xr-x.  2 jogas jogas    6 Mar 28 03:12 boot
drwxr-xr-x.  4 jogas jogas   43 Aug 14 08:18 dev
drwxr-xr-x. 48 jogas jogas 4.0K Aug 14 08:18 etc
drwxr-xr-x.  2 jogas jogas  133 Aug  5 12:16 hello
drwxr-xr-x.  2 jogas jogas    6 Mar 28 03:12 home
drwxr-xr-x.  9 jogas jogas  125 Aug  5 11:24 lib
drwxr-xr-x.  2 jogas jogas   34 Jul  7 21:30 lib64
drwxr-xr-x.  2 jogas jogas    6 Jul  7 21:30 media
drwxr-xr-x.  3 jogas jogas   15 Aug  5 11:27 mnt
drwxr-xr-x.  2 jogas jogas    6 Jul  7 21:30 opt
drwxr-xr-x.  2 jogas jogas    6 Mar 28 03:12 proc
drwx------.  2 jogas jogas   37 Jul  7 21:30 root
drwxr-xr-x.  3 jogas jogas   30 Jul  7 21:30 run
drwxr-xr-x.  2 jogas jogas 4.0K Aug  5 11:24 sbin
drwxr-xr-x.  2 jogas jogas    6 Jul  7 21:30 srv
drwxr-xr-x.  2 jogas jogas    6 Mar 28 03:12 sys
drwxrwxr-x.  2 jogas jogas    6 Aug  5 12:16 tmp
drwxr-xr-x. 10 jogas jogas  105 Jul  7 21:30 usr
drwxr-xr-x. 11 jogas jogas  139 Jul  7 21:30 var
$ ch-run . -- ./hello/hello
0: MPI version:
Open MPI v3.1.4, package: Open MPI root@89edfcf5d38f Distribution, ident: 3.1.4, repo rev: v3.1.4, Apr 15, 2019
0: init ok, 1 ranks, userns 4026532427
0: send/receive ok
0: finalize ok

Hope this helps.

[le-ander]

This is great, thanks a ton!

[reidpr]

I'll add that there's no fundamental reason that the building-related scripts could not run on MacOS, so if you're interested in writing a patch, we should discuss. It is running containers (ch-run and the scripts that call it) that requires Linux system calls.

[le-ander]

I see, I think however that this beyond my capabilities for now.

[pagrubel]

This will only work for those who have sudo on their mac, unfortunately.

[rstyd]

@pagrubel You should be able to use ch-grow on mac. Though, I don't think that's been tested.

[tylerjereddy]

At the time of writing it looks like ./configure on the master branch will fail on mac OS because configure: error: shm_open(3) not found. It looks like the configure script assumes that -lrt is the linking source for shm_open, but that is only true on Linux, and not BSD/Mac. spack runs into the same issue rather than telling me that MacOS is not supported when I try to install charliecloud package. Probably useful to raise an informative error in spack recipe for now, when building on Mac.

If there are broader issues with performing basic builds on a Mac, I won't try to open a PR to fix that though.

[reidpr]

@tylerjereddy The namespace system calls we need aren't available on Mac, so Charliecloud definitely can't run containers on a Mac. Building containers may be plausible. I'd certainly be open to a PR that fails configure with a better error message on non-Linux systems, as well as the Spack recipe. A more extensive PR to enable building containers on a Mac would need more discussion, I think.

[reidpr]

It turns out that we could build images on MacOS (via Docker but not ch-grow because it needs ch-run, which uses Linux system calls), and all/most of the shell scripts should work. So, we can revisit this.

[rstyd]

The current plan is to add support for building images via docker and pushing images via ch-grow push on MacOS. Will also require modifying configure.ac to support MacOS builds.

[bio-la]

Hello, any update on this? is the suggested VM still the recommended route for creating a tarball from docker image on Mac? thanks!

[reidpr]

@bio-la, thanks for asking. At present a Linux VM is the only way to do Charliecloud on MacOS. The Python and shell parts of Charliecloud should work on Macs, aside from there surely being showstopper bugs that need to be fixed, but we haven't been able to prioritize it. Patches welcome, of course.