qmcpack Add components to centos CI containers for reproducibility

Is your feature request related to a problem? Please describe. In order to debug CI failures, e.g. #4510 the container images should be interactive and reproducible. After downloading and running the container it's currently missing the spack installation used to installed required packages.

Describe the solution you'd like The location /opt/spack from here is missing. Also find /opt -name "spack" returns empty.

Describe alternatives you've considered N/A

Additional context Not sure if this is intentional when using this spack image.

Mar 14 '23 11:03 williamfgc

The way spack generated the dockerfile, it uses an intermediate image spack/centos-stream that has spack installed on it to build everything, then transfers the built dependencies to the actual base image of choice, in this case quay.io/centos/centos:stream.

Is this issue asking just for the spack version used in the intermediate image or for another image to be built but one that stops short of transferring to the actual base image?

Mar 14 '23 13:03 walshmm

@walshmm I might need your guidance on this. Is spack supposed to be available in the image? /opt/spack is added in the PATH and LD_LIBRARY_PATH when running CI (see first message), so I thought it would be available if I pull the container locally. Does it make sense?

Mar 14 '23 14:03 williamfgc

@williamfgc Yes I understand, the reason it was added in run_steps.sh was to match the path setup my image displays locally. This path variable may actually just be left over from building and unnecessary to include in run_steps.sh, because as you have mentioned this path doesnt actually exist in the final image.

Spack is not intended to be accessible in the final container, they use a two stage approach, where spack should only be available in the "build" phase. This is to minimize final image size.

Mar 14 '23 15:03 walshmm

OK, thanks for clarifying. Should non-existing directories removed from PATH and LD_LIBRARY_PATH in run_step.sh?

Mar 14 '23 15:03 williamfgc

Yes, for clarity's sake. Leftovers from trying to figure out the right env for the runners.

Mar 14 '23 15:03 walshmm

@walshmm thanks, feel free to do a PR. The reason we want to keep run_step.sh clean is so we can reproduce locally potential CI failures.

Mar 14 '23 15:03 williamfgc

Please document or point to the docs where how to run this ci container is explained.

Mar 14 '23 16:03 prckent

@prckent good point, @walshmm please add the spack centos image to this page.

Mar 14 '23 16:03 williamfgc