conan-docker-tools icon indicating copy to clipboard operation
conan-docker-tools copied to clipboard
verified publisher icon conan-io

libstdc++ is different between images for gcc compilers

Open samuel-emrys opened this issue 2 years ago • 7 comments

Description of Problem, Request, or Question

I'm discovering that the glibc versions provided for the containers for the different gcc compiler version images is different. To illustrate, the maximum version on the gcc12 image is GLIBCXX_3.4.30, and on the gcc10 image it's GLIBCXX_3.4.28. This poses a problem where for the moment, all conan center images for conan 2.0 are being built against gcc 11.

Particularly for tool_requires packages, where the compiler is removed from the package id (such as doxygen, xapian-core), the resulting binary is only compatible with the gcc11 image and higher. On the gcc10 image, I'm faced with error messages similar to the following:

/builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen: /usr/local/lib64/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen)
/builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen: /usr/local/lib64/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen)
/builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen: /usr/local/lib64/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by /builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen)
CMake Warning at .conan/p/cmake5b7ee79318ee9/p/share/cmake-3.25/Modules/FindDoxygen.cmake:492 (message):
  Unable to determine doxygen version: 1
Call Stack (most recent call first):
  .conan/p/cmake5b7ee79318ee9/p/share/cmake-3.25/Modules/FindDoxygen.cmake:655 (_Doxygen_find_doxygen)
  docs/CMakeLists.txt:1 (find_package)
-- Found Doxygen: /builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen  found components: doxygen dot 
/builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen: /usr/local/lib64/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen)
/builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen: /usr/local/lib64/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen)
/builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen: /usr/local/lib64/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by /builds/user/project/.conan/p/doxyga56639b015d56/p/bin/doxygen)
CMake Error at .conan/p/cmake5b7ee79318ee9/p/share/cmake-3.25/Modules/FindDoxygen.cmake:734 (message):
  Unable to generate Doxyfile template: 1
Call Stack (most recent call first):
  docs/CMakeLists.txt:1 (find_package)
-- Configuring incomplete, errors occurred!
See also "/builds/user/project/build/Debug/CMakeFiles/CMakeOutput.log".
*********************************************************
Recipe 'conanfile.py (project/0.1.0)' cannot build its binary
It is possible that this recipe is not Conan 2.0 ready
If the recipe comes from ConanCenter check: https://conan.io/cci-v2.html
If it is your recipe, check if it is updated to 2.0
*********************************************************
ERROR: conanfile.py (project/0.1.0): Error in build() method, line 130
	cmake.configure()
	ConanException: Error 1 while executing

Because the compiler doesn't form part of the package id for these packages, this is also difficult to overwrite consistently/explicitly. It would be good to attempt to maintain a consistent glibc version across all images regardless of compiler to keep the binary interface the same.

Logs

$ docker container list
CONTAINER ID   IMAGE                                 COMMAND       CREATED             STATUS             PORTS     NAMES
76b4ddf35686   conanio/gcc12-ubuntu16.04:2.0.3       "/bin/bash"   48 seconds ago      Up 47 seconds                nostalgic_nightingale
d28c60e839f5   conanio/gcc11-ubuntu16.04:2.0.3       "/bin/bash"   2 minutes ago       Up 2 minutes                 unruffled_hodgkin
bc6376ebebe2   conanio/gcc10-ubuntu16.04:2.0.3       "/bin/bash"   8 minutes ago       Up 8 minutes                 laughing_chandrasekhar

gcc10

docker exec bc6376ebebe2 strings /usr/local/lib64/libstdc++.so.6 | grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_3.4.25
GLIBCXX_3.4.26
GLIBCXX_3.4.27
GLIBCXX_3.4.28
GLIBCXX_DEBUG_MESSAGE_LENGTH

gcc11

docker exec d28c60e839f5 strings /usr/local/lib64/libstdc++.so.6 | grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_3.4.25
GLIBCXX_3.4.26
GLIBCXX_3.4.27
GLIBCXX_3.4.28
GLIBCXX_3.4.29
GLIBCXX_DEBUG_MESSAGE_LENGTH

gcc12

$ docker exec 76b4ddf35686 strings /usr/local/lib64/libstdc++.so.6 | grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_3.4.25
GLIBCXX_3.4.26
GLIBCXX_3.4.27
GLIBCXX_3.4.28
GLIBCXX_3.4.29
GLIBCXX_3.4.30
GLIBCXX_DEBUG_MESSAGE_LENGTH

samuel-emrys avatar Apr 10 '23 14:04 samuel-emrys

. On the gcc10 image, I'm faced with error messages similar to the following:

@samuel-emrys Could you please inform which specific docker image is failing?

Also, legacy Docker images are incompatible with modern image, as documented here, so conanio/gcc10 and conanio/gcc10-ubuntu16.04 are completely different and incompatible.

uilianries avatar Apr 10 '23 14:04 uilianries

I get the above error using both gcc10-ubuntu16.04 and gcc10-ubuntu18.04. Both have the same glibc installed, and both are different from their gcc11 counterparts. The logs posted above have the specific image and glibc information

Edit: to be clear, all my references to gcc10 have been colloquial referenced to gcc10-ubuntu16.04. This also applies to other gcc versions.

samuel-emrys avatar Apr 10 '23 14:04 samuel-emrys

So your problem is not the glibc version, but the libstdc++ provided by the compiler.

Indeed it's a problem, but it should be fixed by Conan Center Index CI instead, to calibrate to build tool packages with conanio/gcc10-ubuntu16.04 only.

I'll open an internal issue to C3I to use that image. Thank you for reporting.

uilianries avatar Apr 10 '23 15:04 uilianries

So your problem is not the glibc version, but the libstdc++ provided by the compiler.

Ah yes, you're right. apologies.

Indeed it's a problem, but it should be fixed by Conan Center Index CI instead, to calibrate to build tool packages with conanio/gcc10-ubuntu16.04 only.

Is this the solution? Doesn't this just kick the ball further down the hill? Presumably one would still come up against this issue with gcc9-ubuntu16.04. Maybe the solution here (really) is to adopt a proper model of glibcxx, which might mean:

  • Avoid removing compiler and compiler.version from package id
  • Describe the compatibility with binaries using the compatability.py plugin

This would solve the problem of doxygen being incorrectly identified as compatible on the gcc10-ubuntu16.04 image, and would hopefully instead trigger a rebuild/download a compatible binary.

I'm assuming that it wouldn't be feasible to hold the libsdc++ version constant between compilers/images - this would presumably remove forward looking language features.

samuel-emrys avatar Apr 10 '23 23:04 samuel-emrys

One note, we can not keep a same version of libstdc++ for every Docker image (it could workaround the situation), because each compiler version expects new features. So, Docker are incompatible by definition.

Plus, this repository, Conan Docker Tools, is only related to building Docker images, not to building anything on ConanCenterIndex. Which means, any decision about recipes, packages, etc, are not related to this project.

Avoid removing compiler and compiler.version from package id

It's already recommended: https://github.com/conan-io/conan-center-index/blob/master/docs/adding_packages/conanfile_attributes.md#settings

And, part of the pre-built tool template: https://github.com/conan-io/conan-center-index/blob/master/docs/package_templates/prebuilt_tool_package/all/conanfile.py

For your specific case, just need to open a pull request fixing Doxygen or an issue asking for a fix 💯

uilianries avatar Apr 11 '23 07:04 uilianries

Plus, this repository, Conan Docker Tools, is only related to building Docker images, not to building anything on ConanCenterIndex. Which means, any decision about recipes, packages, etc, are not related to this project.

yep, this is the conclusion i was trying to communicate :)

It's already recommended: https://github.com/conan-io/conan-center-index/blob/master/docs/adding_packages/conanfile_attributes.md#settings

This is actually the opposite of what I'm saying. The excerpt from that link that matters (emphasis mine):

Recipes that primarily provide compiled applications (e.g. b2, cmake, make, ...), which typically applies to packages that are consumed as tool requires) must list all the settings as well, as they are required during package creation. However, it is advised that the compiler setting is removed one in the package_id() method as follows:

def package_id(self):
   del self.info.settings.compiler

This reflects those cases where tools are consumed exclusively as executables, irrespective of how they were built. Additionally, this reduces the number of configurations generated by CI.

Removing self.info.settings.compiler from the package id here is what causes this issue. Doxygen is built with gcc11, and has the compiler removed from the package id. I try to use it on an image that only has gcc10, and then it fails because it doesn't satisfy the libstdc++ requirements. I think the better solution would be to actively prohibit removal of self.info.settings.compiler and encourage the implementation of a compatibility() method to the effect of:

def compatibility(self):
    # is there a better way of reading all possible versions from settings.yml?
    settings_yml_path = pathlib.Path(pathlib.Path.home(), ".conan2", "settings.yml")
    with open(settings_yml_path, "r") as f:
        settings_yml = yaml.safe_load(f)

    if self.settings.compiler == "gcc":
        return [{"settings": [("compiler.version", v)]}
            for v in settings_yml["gcc"]["version"] if v >= self.settings.compiler.version]

Which basically would just mean that doxygen when built with gcc11 would match any version of gcc later than 11, but not match anything earlier.

samuel-emrys avatar Apr 11 '23 09:04 samuel-emrys

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Mar 17 '24 08:03 stale[bot]