Docker build fail & CI of OpenSeesPy

[ccaprani]

@zhuminjie we're getting ready to release https://github.com/MonashSmartStructures/ospgrillage and just completing our CI/CD pipeline to push it to PyPi. Unfortunately we've come up against a problem, which I'll try to explain below. However, the relevance to OpenSeesPy is that the solution to our problem will be useful for CI for OpenSeesPy itself, and could mean the manual steps involved in making a release could be avoided. So I think solving this is useful for everyone.

We are setting up Github Actions and cibuildwheel to automate tagged release deployments. cibuildwheel uses docker images of the target platforms inside the remote build machine runner (i.e. a box in a box). Our test suite installs and executes in the runner fine (the outer box). For building the wheels in the docker (i.e. the inner box), the OpenSeesPy wheel installs fine, but the opensees.so and opensees.pyd library has dependencies which are not in the wheel (that's ok), but critcally also not in the docker image which causes the wheel build to fail due to the import openseespy checks in the OpenSeesPy build system. These libraries are: libdl.so.2, libgfortran.so.5, libstdc++.so.6, libm.so.6, libgcc_s.so.1, libc.so.6.

Simple, right? Just install these in the docker container?! Not really though...since the opensees.so or opensees.pyd dependencies need to be built for the current platform. Hence, to build the wheel for ospgrillage, we can't just use the wheel for OpenSeesPy, but we must build OpenSees itself.

So as a first go at this, I tried to build OpenSeesPy locally, using the DockerFile. However, it doesn't build:

Step 7/105 : COPY opensees /home/opensees
COPY failed: file not found in build context or excluded by .dockerignore: stat opensees: file does not exist

So it seems the DockerFile is hardcoded to your local machine and I guess the OpenSeesPy release process is a few manual steps of building OpenSees itself locally, and then making the docker container, in order to then build the wheels for PyPi.

I think the upshot here is that for us to solve our problem for distributing ospgrillage, while we could mirror the manual workflow it seems a generic and automated build system for OpenSeesPy itself would benefit both of us.

Can you let me know if I have misunderstood something about the build process?

Edit: By the way, the build failure in docker is:

  ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /tmp/pip-build-env-nt6e4vwh/overlay/lib/python3.8/site-packages/openseespylinux/opensees.so)

even though the libraries are installed - maybe you've come across this before?

Running before_all...

    + /opt/python/cp38-cp38/bin/python -c 'import sys, json, os; json.dump(os.environ.copy(), sys.stdout)'
    + sh -c 'yum install -y libgfortran5 libgcc glibc libstdc++'
Loaded plugins: ovl
Setting up Install Process
Package libgfortran5-8.2.1-1.3.1.el6_10.x86_64 already installed and latest version
Package libgcc-4.4.7-23.el6.x86_64 already installed and latest version
Package matching glibc-2.12-1.212.el6_10.3.x86_64 already installed. Checking for update.
Package libstdc++-4.4.7-23.el6.x86_64 already installed and latest version
Nothing to do

[zhuminjie]

There is a script called build_docker.py in the same folder of the DockerFile, which I use it to do some preparation work and to call the docker to build openseespy. If you use the DockerFile in your own workflow, you might need adapt it to fit your environment.