charliecloud
charliecloud copied to clipboard
ch-run: default binds: potential issue(s) with Cray PE images.
The default bind list includes optional paths intended to give, in conjunction with ch-fromhost
, a container access to Cray resources, e.g., Aries/Slingshot interconnect. The required binds to enable such access differs between Aries and Slingshot; however, we've had no reason to make this distinction or require explicit corresponding arguments. This will likely change with Cray PE containers.
Cray PE containers are going to do cray things, software will be installed to /opt
and the various paths listed in the current default bind list. It seems like such a container will likely have issues running on different flavors of Cray when all these optional binds are picked up by default.
This issue is then to: 1) separate Cray specific binds from the defaults; 2) organize Cray binds according to host interconnect, e.g., BINDS_CRAY_UGNI
, BIND_CRAY_SLINGSHOT
, etc.; and 3) add corresponding explicit optional arguments, e.g., --bind-ugni
, and --bind-slingshot
.
Something like:
/* Cray bind-mounts. */
struct bind BINDS_CRAY[] = {
{ "/var/lib/hugetlbfs", "/var/lib/hugetlbfs", BD_OPTIONAL },
};
/* Cray Gemini/Aries interconnect bind-mounts. */
struct bind BINDS_CRAY_UGNI[] = {
{ "/etc/opt/cray/wlm_detect", "/etc/opt/cray/wlm_detect", BD_OPTIONAL },
{ "/opt/cray/wlm_detect", "/opt/cray/wlm_detect", BD_OPTIONAL },
{ "/opt/cray/alps", "/opt/cray/alps", BD_OPTIONAL },
{ "/opt/cray/udreg", "/opt/cray/udreg", BD_OPTIONAL },
{ "/opt/cray/ugni", "/opt/cray/ugni", BD_OPTIONAL },
{ "/opt/cray/xpmem", "/opt/cray/xpmem", BD_OPTIONAL },
{ "/var/opt/cray/alps", "/var/opt/cray/alps", BD_OPTIONAL },
};
/* Known Cray Shasta bind-mounts. */
struct bind BINDS_CRAY_SHASTA[] = {
{ "/var/spool/slurmd", "/var/spool/slurmd", BD_OPTIONAL },
};