When installing from release zip: "error: package directory 'kubernetes/config' does not exist"

[tholok97]

trafficstars

What happened (please include outputs or screenshots):

Install from release zip fails with message "error: package directory 'kubernetes/config' does not exist". Looking at the exploded zip the kubernetes/config link is broken, and the kubernetes/base directory is empty.

Our issue looks like this one https://github.com/kubernetes-client/python/issues/532, and the solution there was to properly install submodules. Is it possible that the release zip is missing submodules?

What you expected to happen:

I expect installation following the instructions provided for the release to work.

How to reproduce it (as minimally and precisely as possible):

This Dockerfile shows my issue:

FROM ubuntu:16.04

RUN apt-get update && apt-get install -y \
    unzip \
    curl \
    python \
    python-pip \
    && rm -rf /var/lib/apt/lists/*

RUN pip install 'setuptools>=21.0.0' requests

# Installing from zip based on https://github.com/kubernetes-client/python/releases/tag/v11.0.0b2

RUN curl -LO https://github.com/kubernetes-client/python/archive/v11.0.0b2.zip \
    && unzip v11.0.0b2.zip \
    && cd python-11.0.0b2 \
    && python setup.py install

docker build . on this will give the error "error: package directory 'kubernetes/config' does not exist".

The steps above the actual installation in the Dockerfile are just to mimmick our environment.

Anything else we need to know?:

Before we were installing by git clone --recurse-submodules. We want to move away from this and use the zip you provide directly instead, to remove the usage of git.

Note that we are not using pip, this is why we'd like to use the python setup.py install method. The usage of pip in the above Dockerfile is just to quicly mimmick our setup for this issue.

Environment:

• Kubernetes version (kubectl version): N/A
• OS (e.g., MacOS 10.13.6): Ubuntu 16.04 (Demo Dockerfile), Centos 7
• Python version (python --version): Python 2.7.12
• Python client version (pip list | grep kubernetes): N/A

[micw523]

/assign

[micw523]

It sounds like you're in a very restrictive environment. GitHub release does not contain submodules. The source files for download are automatically generated. We did not provide installation instructions for downloaded zip from GitHub.

There are reasons to split our repo into two separate repos though. Without using git, I'd suggest the following that combines the master branch of python-base and releases of python:

1. Download a zip release of the main repo like https://github.com/kubernetes-client/python/archive/v11.0.0b2.zip
2. Download the python-base master branch https://github.com/kubernetes-client/python-base/archive/master.zip
3. Decompress both zip files
4. Move all folders, except hack/ from the python-base folder into the kubernetes/ folder of the main repo release
5. Install using python setup.py install.

However, why not use pip? You're using pip to install setuptools already.

[tholok97]

Thanks for having a look @micw523 :)

We did not provide installation instructions for downloaded zip from GitHub.

In the releases page there are what I read as installation instructions for each release. This is what I followed:

unzip client-python-11.0.0b2.zip
cd client-python-11.0.0b2
python setup.py install

So my impression was that the zip is self-contained and would work when installed like this. If that is not the intention then I have misunderstood. Should the above instructions in the releases page be updated?

If the release is not intended to contain all the source code I'll do as you suggest (thanks!), and modify it slightly to checkout the correct version of python-base for a given release somehow.

However, why not use pip? You're using pip to install setuptools already.

The Dockerfile provided in this issue is not what we use for our infrastructure, it was just a convenient way to show my problem. We install Python libraries including setuptools.py through another method in our real Dockerfiles. Why we are not using pip is a good question, but it has been like this for some time so we'd like to avoid installing pip only for this one package. It was not my decision, I'm sure there's a good reason behind it.

[micw523]

It's never our intention to not include all the files :) The release zip files are automatically generated and I don't think it's possible to get submodules inside. I just saw the instructions on the release page - I had thought they were the same as in the main readme.

I just asked @roycaihw to look into this - we could attach a separate zip file for the base repo when publishing the release, or tag the base repo with appropriate release tags. This way you wouldn't have a version mismatch such as base@master and main@v11beta2.

[micw523]

/lifecycle frozen

[roycaihw]

Currently github doesn't support creating release that contains submodule, so the instruction is wrong and should be updated.

For now you can either use pip or clone the source code recursively using git. Talked with @micw523 on slack and I agree adding tag to python-base will help with the installing-from-zip story.

@micw523 Will you be interested in updating the release instruction and creating a new template for the release page?

[micw523]

This sgtm. I can go create a new release template.

[micw523]

/reopen Accidentally closed this one.

[k8s-ci-robot]

@micw523: Reopened this issue.

In response to this:

/reopen Accidentally closed this one.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.