charliecloud icon indicating copy to clipboard operation
charliecloud copied to clipboard

ch-run: default binds: potential issue(s) with Cray PE images.

Open j-ogas opened this issue 2 years ago • 0 comments

The default bind list includes optional paths intended to give, in conjunction with ch-fromhost, a container access to Cray resources, e.g., Aries/Slingshot interconnect. The required binds to enable such access differs between Aries and Slingshot; however, we've had no reason to make this distinction or require explicit corresponding arguments. This will likely change with Cray PE containers.

Cray PE containers are going to do cray things, software will be installed to /opt and the various paths listed in the current default bind list. It seems like such a container will likely have issues running on different flavors of Cray when all these optional binds are picked up by default.

This issue is then to: 1) separate Cray specific binds from the defaults; 2) organize Cray binds according to host interconnect, e.g., BINDS_CRAY_UGNI, BIND_CRAY_SLINGSHOT, etc.; and 3) add corresponding explicit optional arguments, e.g., --bind-ugni, and --bind-slingshot.

Something like:


/* Cray bind-mounts. */
struct bind BINDS_CRAY[] = {
   { "/var/lib/hugetlbfs",       "/var/lib/hugetlbfs",       BD_OPTIONAL },
};

/* Cray Gemini/Aries interconnect bind-mounts. */
struct bind BINDS_CRAY_UGNI[] = {
   { "/etc/opt/cray/wlm_detect", "/etc/opt/cray/wlm_detect", BD_OPTIONAL },
   { "/opt/cray/wlm_detect",     "/opt/cray/wlm_detect",     BD_OPTIONAL },
   { "/opt/cray/alps",           "/opt/cray/alps",           BD_OPTIONAL },
   { "/opt/cray/udreg",          "/opt/cray/udreg",          BD_OPTIONAL },
   { "/opt/cray/ugni",           "/opt/cray/ugni",           BD_OPTIONAL },
   { "/opt/cray/xpmem",          "/opt/cray/xpmem",          BD_OPTIONAL },
   { "/var/opt/cray/alps",       "/var/opt/cray/alps",       BD_OPTIONAL },
};

/* Known Cray Shasta bind-mounts. */
struct bind BINDS_CRAY_SHASTA[] = {
   { "/var/spool/slurmd",        "/var/spool/slurmd",        BD_OPTIONAL },
};

j-ogas avatar Oct 18 '22 15:10 j-ogas