Configure directory structure of $EXT_BUILD_DEPS

[dylanrush]

When I pass a few other Bazel dependencies into cmake_external(.. deps = [...]), all of the headers seem to be copied into $EXT_BUILD_DEPS/include and libraries to $EXT_BUILD_DEPS/lib. However, the directory structure is destroyed. If the dependency creates external/org_dependency/include/core/platform/utils/foo.h, then this is just symlinked as $EXT_BUILD_DEPS/include/foo.h. This creates a problem because my source code that I'm trying to compile wants to #include "core/platform/utils/foo.h". And I really can't use $EXT_BUILD_DEPS at all because another dependency decided to create their own time.h which is creating all sorts of linker errors, as you could imagine.

Wondering if there is a way to preserve dependency structure.

Right now my workaround, which isn't quite working, is to use flags like:

        "CMAKE_CXX_FLAGS": "-I $EXT_BUILD_ROOT/bazel-out/k8-opt/bin/external/org_tensorflow/ \
            -I $EXT_BUILD_ROOT/external/org_tensorflow/ \

[edbaunton]

I have run across this issue too. The way that these rules place all the headers in a single flat include dir made it unusable for my use case. It requires that all headers have unique names across your project and dependency hierarchy.

[jsharpe]

I've also run into this with Python and the ctype.h header.

[psigen]

Looking at the source, there are actually two things that get put in $EXT_BUILD_DEPS/include: headers and include_dirs: https://github.com/bazelbuild/rules_foreign_cc/blob/83a0c2a9b1e83b98c3d9546a64518e8bd480ad50/tools/build_defs/framework.bzl#L580

While the headers are passed as a list of files that gets squashed here: https://github.com/bazelbuild/rules_foreign_cc/blob/83a0c2a9b1e83b98c3d9546a64518e8bd480ad50/tools/build_defs/framework.bzl#L606-L607 the same logic applied to include_dirs actually symlinks them directly as entire directory trees.

So I've found that adding cc_library(..., includes=["."]) directives into my bazel targets makes two copies: the flattened pile of files and a directory symlink to the base of the include_dir inside $EXT_BUILD_DEPS/include. I've been able to use this as a workaround to get a full include directory structure inserted into $EXT_BUILD_DEPS/include.

I'm as confused as you are about the utility of the flattened headers in the first place -- not sure what use-case that's expected to cover. :shrug:

[chengji77]

I also run into this issue. glog has a demangle.h that gets linked to $EXT_BUILD_DEPS/include/demangle.h. This conflicts with the demangle.h shipped with gcc 10.

[wudisheng]

+1 to this issue, the handling of (non-foreign, i.e. without a ForeignCcDepsInfo provider) dependencies is no good here. My desired behavior is to symlink such dependencies each into its own sub-directory, just like foreign ones (the /copy_/ => DEPS/), and should take strip_include_prefix and include_prefix into consideration.

I personally tried to make a customized rule to provide a fake ForeignCcDepsInfo with generated contents, that didn't work because once ForeignCcDepsInfo is provided, the build of the cc_library is not triggered.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_foreign_cc!

[psigen]

As far as I can tell, the desired behavior would always be to replicate the directory structure of the source with include_prefix and strip_include_prefix. I can see no use case where a flattened set of files is the expected result.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_foreign_cc!

[psigen]

bump, still affected by this issue

[sulmone]

I ran into this when adding abesil as a dependency - still happening

[soro]

This is also an issue building folly, specifically during the boost discovery step

[alokpr]

Ran into this issue when building opencv with foreign-rules-cmake and adding oneapi tbb built using bazel. It adds a private tbb semaphore.h at EXT_BUILD_DEPS/include which conflicts with system semaphore.h