Build fails in GitHub CI

[dzenanz]

Ubuntu 20.04: https://open.cdash.org/viewBuildError.php?buildid=8243884 MacOS 11: https://github.com/dzenanz/ITKIOOMEZarrNGFF/actions/runs/3348207654/jobs/5547019929

In branch: https://github.com/dzenanz/ITKIOOMEZarrNGFF/tree/tensorstore Commit: https://github.com/dzenanz/ITKIOOMEZarrNGFF/commit/87c4042cf92bb175cf8c1d46ecd4033537e06078

This is despite building locally on Windows.

[jbms]

For macOS, I believe the issue is that you need to set MACOSX_DEPLOYMENT_TARGET >= 10.14 due to tensorstore's use of sized/aligned operator new/delete:

https://github.com/google/tensorstore/blob/a3f3ee23a8d27a5ba8c1faee944d3a23b00fedf4/setup.py#L117

I am not sure what the problem is with the Ubuntu 20.04 build. Can you confirm which gcc version is being used there?

[jbms]

I think the issue with Ubuntu 20.04 is that, despite our documentation claiming support for GCC 9, we stopped testing with GCC 9 some time ago. Currently we only test with GCC 10 and later, and it appears that TensorStore no longer builds with GCC 9.

It looks like Ubuntu 20.04 does provide GCC 10 as well: https://ahelpme.com/linux/ubuntu/install-and-make-gnu-gcc-10-default-in-ubuntu-20-04-focal/

Is it an option to build with GCC 10 instead?

[dzenanz]

Bumping CI Ubuntu to 22.04 leads to a strange new error:

...
Loading /home/runner/work/ITKIOOMEZarrNGFF/build/_deps/riegeli-src/riegeli/varint/BUILD
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Configuring done
CMake Error at /home/runner/work/ITKIOOMEZarrNGFF/ITK/CMake/ITKModuleMacros.cmake:467 (add_library):
  Feature 'bazel_to_cmake_needed_library', specified through
  generator-expression '$<LINK_LIBRARY>' to link target 'IOOMEZarrNGFF', is
  not supported for the 'CXX' link language.
Call Stack (most recent call first):
  src/CMakeLists.txt:22 (itk_module_add_library)

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

Local build on Ubuntu 22.04 succeeds.

[jbms]

Regarding that error:

TensorStore defines that link feature here: https://github.com/google/tensorstore/blob/b92f40210186e7246c6873bdb3947c30c17cd74a/tools/cmake/bazel_to_cmake.cmake#L70

That file is included from tensorstore's CMakeLists.txt:

https://github.com/google/tensorstore/blob/b92f40210186e7246c6873bdb3947c30c17cd74a/CMakeLists.txt#L75

Therefore I'm not sure what the issue is.

However, that error suggests that you are building TensorStore as a shared library (due to BUILD_SHARED_LIBS=ON). I think in most cases it would be preferable to build it as a static library, even if you are going to link it into a shared library. Otherwise, due to how the build is currently structured, you will have to package a very large number of individual shared libraries. I'd recommend setting BUILD_SHARED_LIBS=OFF before calling FetchContent_MakeAvailable(tensorstore).

[dzenanz]

By always building tensorstore as static library, I am now down to build warnings from AOM, of the style: _deps/aom-src/aom_dsp/grain_table.c:108:5: warning: ignoring return value of 'fscanf' declared with attribute 'warn_unused_result' [-Wunused-result]

[jbms]

This particular warning would be fixed by updating our version of aom.

In general with the large number of dependencies and compilers to support it may be difficult to avoid all warnings, though.

[dzenanz]

This particular warning would be fixed by updating our version of aom.

This would allow me to avoid suppressing this warning, and also close this issue 😄

[dzenanz]

My attempt to update AOM doesn't even build locally:

Loading C:/Misc/tensorstore-vs19/_deps/aom-src/BUILD.bazel

Traceback (most recent call last):

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 185, in get_optional_target_info

    rule_info.impl()

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 285, in <lambda>

    lambda: _cc_library_impl(cast(Package, package), target, **kwargs),

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 311, in _cc_library_impl

    **_handle_cc_common_options(

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 216, in _handle_cc_common_options

    resolved_srcs = _package.context.get_targets_file_paths(

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 304, in get_targets_file_paths

    files.extend(self.get_file_paths(target, custom_target_deps))

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 290, in get_file_paths

    info = self.get_target_info(target)

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 226, in get_target_info

    raise ValueError(f"Target not found: {target}")

ValueError: Target not found: @org_aomedia_aom//:aom_dsp/grain_synthesis.c


The above exception was the direct cause of the following exception:


Traceback (most recent call last):
  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 185, in get_optional_target_info

    rule_info.impl()

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 285, in <lambda>

    lambda: _cc_library_impl(cast(Package, package), target, **kwargs),

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 311, in _cc_library_impl

    **_handle_cc_common_options(

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 220, in _handle_cc_common_options

    cmake_deps = set(_package.context.get_deps(_package.get_label_list(deps)))

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 339, in get_deps

    deps.extend(self.get_dep(target))

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 312, in get_dep

    info = self.get_optional_target_info(target)

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 192, in get_optional_target_info

    raise ValueError(f"Error analyzing {rule_label}") from e

ValueError: Error analyzing @org_aomedia_aom//:dsp

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 185, in get_optional_target_info

    rule_info.impl()

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 285, in <lambda>

    lambda: _cc_library_impl(cast(Package, package), target, **kwargs),

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 311, in _cc_library_impl

    **_handle_cc_common_options(

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\native_rules.py", line 220, in _handle_cc_common_options

    cmake_deps = set(_package.context.get_deps(_package.get_label_list(deps)))

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 339, in get_deps

    deps.extend(self.get_dep(target))

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 312, in get_dep

    info = self.get_optional_target_info(target)

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 192, in get_optional_target_info

    raise ValueError(f"Error analyzing {rule_label}") from e

ValueError: Error analyzing @org_aomedia_aom//:av1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake.py", line 41, in <module>

    sys.exit(bazel_to_cmake.main.main())

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\main.py", line 137, in main

    context.analyze_default_targets()

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 363, in analyze_default_targets

    self.analyze(sorted(self._targets_to_analyze))

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 372, in analyze

    self.get_target_info(target)

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 224, in get_target_info

    info = self.get_optional_target_info(target)

  File "C:\Misc\tensorstore\tools\cmake\bazel_to_cmake\evaluation.py", line 192, in get_optional_target_info

    raise ValueError(f"Error analyzing {rule_label}") from e

ValueError: Error analyzing @org_aomedia_aom//:libaom

CMake Error at C:/Misc/tensorstore-vs19/_deps/aom-src/CMakeLists.txt:26 (execute_process):
  execute_process failed command indexes:

    1: "Child return code: 1"



Configuring incomplete, errors occurred!

The version of this library used by tensorstore is more than a year old now.

[laramiel]

I've updated AOM in commit b8949f2

I've also updated the BUILD file to try and make future updates to AOM a bit easier.

[dzenanz]

Thank you @laramiel. That reduces number of warnings from 6 to 3. It is still the same kind of warnings: _deps/aom-src/aom_dsp/grain_table.c:145:11: warning: ignoring return value of 'fscanf' declared with attribute 'warn_unused_result' [-Wunused-result].

[laramiel]

You could try and contribute upstream fixes for those...

[dzenanz]

I suppressed those warnings in the Linux CI, so now Mac CI finishes. However, it has 50+ warnings of the style: /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /.../ITK-build/libaom_av1.a(optical_flow.c.o) has no symbols. I am trying to suppress this via regex file: .+ has no symbols.

[dzenanz]

That suppression worked, which uncovered further 22 warnings of the kind _deps/curl-src/lib/vtls/sectransp.c:1006:6: warning: address of function 'SecCertificateCopyLongDescription' will always evaluate to 'true' [-Wpointer-bool-conversion]. I will suppress them too.

[dzenanz]

Oh, there is also a bunch of warnings on Windows: D:\a\ITKIOOMEZarrNGFF\build\_deps\riegeli-src\riegeli\base\assert.h(147) : warning C4722: 'riegeli::internal::UnreachableStream::~UnreachableStream': destructor never returns, potential memory leak

[dzenanz]

I solved my noisy warnings CI problem by suppressing them:

set(CTEST_CUSTOM_WARNING_EXCEPTION
  ${CTEST_CUSTOM_WARNING_EXCEPTION}
  "Wunused-result"
  "file: .+ has no symbols"
  "Wpointer-bool-conversion"
  "note: prefix with the address-of operator to silence this warning"
  "warning C4722: 'riegeli::internal::UnreachableStream::~UnreachableStream': destructor never returns, potential memory leak"
  "warning C4722: 'riegeli::internal::CheckFailed::~CheckFailed': destructor never returns, potential memory leak"
  )