Failed to build Python package due to missing layer/experimental build

[jonas-eschle]

trafficstars

System information.

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 22.04
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): 2.16
• Python version: 3.11
• Bazel version (if compiling from source): 6.5.0

Describe the problem.

When trying to compile tf-keras (adding it to spack here: https://github.com/spack/spack/pull/43688), it fails when building the python package using create_pip_helper with

PipPackagingError: Pip package missing the file ./tf_keras/layers/experimental/dynamic_lookup.py. If this is expected, add it to PIP_EXCLUDED_FILES in create_pip_helper.py. Otherwise, make sure it is a build dependency of the pip package

Cause

The module layers/experimental (https://github.com/keras-team/tf-keras/tree/master/tf_keras/layers/experimental) doesn't seem to be listed in the BUILD script (https://github.com/keras-team/tf-keras/blob/master/tf_keras/layers/BUILD#L36). Most likely, it should just be added to it?

Helper has an exluding list here https://github.com/keras-team/tf-keras/blob/master/tf_keras/tools/pip_package/create_pip_helper.py#L25 but layer/experimental isn't there either (it should, presumably, be compiled).

Contributing.

• Do you want to contribute a PR? (yes/no): yes
• If yes, please read this page for instructions
• Briefly describe your candidate solution(if contributing):

[mattdangerw]

@jonas-eschle agreed that adding this to the build rule sounds like the right call. Can you try that out? Thanks!

[jonas-eschle]

I tried but actually run into the issue that layers/experimental cannot be used as a build target, as the builds inside experimental require layers -> circular deps. The other folders in layers circumvent this by specifying only the specific build targets.

Instead of changing the logic (i.e. is experimental meant to be dev-built only?), I did what the error advertised and added it to the ignored builds, as it's seemingly not made to be built (see PR #779 )

[tilakrayal]

@jonas-eschle, Have you got the chance to take a look at the PR which was raised and it got merged. https://github.com/keras-team/tf-keras/pull/779. Also when I checked the respective create_pip_helper.py, in both PIP_EXCLUDED_DIRS and EXCLUDED_INIT_FILE_DIRECTORIES contains tf_keras/layers/experimental

PIP_EXCLUDED_DIRS = frozenset(
    [
        "tf_keras/benchmarks",
        "tf_keras/layers/experimental",  # cannot build currently, circular refs
        "tf_keras/tests"

https://github.com/keras-team/tf-keras/blob/master/tf_keras/tools/pip_package/create_pip_helper.py#L37

Thank you!

[jonas-eschle]

Hi @tilakrayal , yes, they include this now, this was my PR and I am glad it was merged, so this issue can be closed