Kapitan 0.29.x downloads helm chart dependency in the wrong directory, resulting in broken build

[jperville]

Describe the bug/feature

Kapitan 0.29.x downloads helm chart dependency in the wrong directory (outside of KAPITAN_ROOT in my case). The consequence is that the build fails because the Kapitan vendor directory does not contain my downloaded helm chart.

To Reproduce

I run a custom docker image of Kapitan with my code added (can provide access if needed but probably not needed) I inject the inventory target file in my docker container using a bind mount outside of the KAPITAN_ROOT.

The kapitan compile step is done using a script that:

• copies the target file in KAPITAN_ROOT/inventory/targets
• populates the refs
• then runs kapitan compile --search-paths vendor . with an explicit --output-path outside of KAPITAN ROOT.

Note: I include the build script at the end of this issue.

In the project which contains my target file, I run the dockerized build using the following script :

set -ueo pipefail

dir=$(dirname $BASH_SOURCE[0])

builder=${1:?please provide the builder docker image as first argument}
target=${2:?please provide the target file as second argument}

refs_tar_gz_base64=${KAPITAN_REFS_TAR_GZ_BASE64:-''}
if [ -z "${refs_tar_gz_base64}" ]; then
  refs_tar_gz_base64=$(${dir}/serialize-secrets.sh)
fi

#docker pull "${builder}"

docker run --rm -ti \
  -e KAPITAN_REFS_TAR_GZ_BASE64="${refs_tar_gz_base64}" \
  -e http_proxy=${http_proxy} \
  -e https_proxy=${https_proxy} \
  -e no_proxy=${no_proxy} \
  -u `id -u`:`id -g` \
  -v $PWD:/srv -w /srv \
  -v $PWD/refs:/opt/mykapitan/refs \
  ${builder} \
  ${target}

With 0.28, the helm chart dependencies are downloaded as expected in the KAPITAN_ROOT/vendor .

With 0.29.3, the dependencies are downloaded in the docker image's workingdir/vendor instead of the KAPITAN_ROOT/vendor directory ; since the docker image's workingdir/vendor directory in now in the Kapitan search path the build fails.

For now, I reverted to building with Kapitan 0.28 and I will be watching this thread.

Expected behavior

I expect my target (which uses a downloaded helm chart dependency) to build successfully.

Requested environment information:

• python --version: Python 3.7.9
• pip3 --version: pip 20.2.3 from /opt/venv/lib/python3.7/site-packages/pip (python 3.7)
• Are you using pyenv or virtualenv? : virtualenv

Additional context

Build output using Kapitan 0.29.3 (broken) :

$ scripts/build.sh mykapitan:xxx dev-labs.yml
KAPITAN DIR: /opt/mykapitan
TARGET FILE: dev-labs.yml
TARGET NAME: dev-labs
TARGET DIR: /srv
Dependency https://raw.githubusercontent.com/jetstack/cert-manager/v0.11.0/deploy/manifests/00-crds.yaml: saved to /srv/vendor/charts/cert-manager-v0.11.0-crds/cert-manager-crds.yml
# ↓↓↓↓ notice the absolute path /srv/vendor instead of the expected relative path vendor/ ↓↓↓↓
Dependency https://charts.jetstack.io/charts/cert-manager-v0.11.0.tgz: extracted to /srv/vendor/charts 
Compile error: vendor/charts/cert-manager-v0.11.0-crds for target: dev-labs not found in search_paths: ['/opt/mykapitan/vendor', '/opt/mykapitan', '/tmp/tmp9n0z89j2.kapitan']

The same build output using Kapitan 0.28 (successful) :

$ scripts/build.sh mykapitan:yyy dev-labs.yml
KAPITAN DIR: /opt/mykapitan
TARGET FILE: dev-labs.yml
TARGET NAME: dev-labs
TARGET DIR: /srv
Dependency https://charts.jetstack.io/charts/cert-manager-v0.11.0.tgz : fetching now
Dependency https://raw.githubusercontent.com/jetstack/cert-manager/v0.11.0/deploy/manifests/00-crds.yaml : fetching now
Dependency https://charts.jetstack.io/charts/cert-manager-v0.11.0.tgz : successfully fetched
# ↓↓↓↓ notice the relative path where cert-manager chart is downloaded : vendor/ ↓↓↓↓
Dependency https://charts.jetstack.io/charts/cert-manager-v0.11.0.tgz : extracted to vendor/charts
Dependency https://raw.githubusercontent.com/jetstack/cert-manager/v0.11.0/deploy/manifests/00-crds.yaml : successfully fetched
Dependency https://raw.githubusercontent.com/jetstack/cert-manager/v0.11.0/deploy/manifests/00-crds.yaml : saved to vendor/charts/cert-manager-v0.11.0-crds/cert-manager-crds.yml
Compiled dev-labs (58.55s)
** ALL DONE..
** Compiled files locations for target 'dev-labs':
 - Kapitan compiled target: /srv/compiled
 - Pandoc HTML tree: /srv/public

For reference, my build script (inside the Kapitan docker image) looks like this:

# calculate Kapitan root
root=$(readlink -m ${BASH_SOURCE[0]}/../..)

echo "KAPITAN DIR: ${root}"
echo "TARGET FILE: ${target}"
echo "TARGET NAME: ${target_name}"
echo "TARGET DIR: ${target_dir}"

# copy target in the location expected by Kapitan
cp ${target} ${root}/inventory/targets/${target_name}.yml

# chdir to Kapitan root
cd ${root}

# fill the KAPITAN_REFS_TAR_GZ_BASE64 variable containing base64 of the first build's secrets.
# the content of this variable is the output of `tar cfz - -C refs . | base64`
refs_tarball="${KAPITAN_REFS_TAR_GZ_BASE64:?calculate me and declare me as CI variable}"
mkdir -p ${root}/refs
echo -n "${KAPITAN_REFS_TAR_GZ_BASE64}" | base64 -d | tar zxf - -C ${root}/refs

# Kapitan compile
kapitan compile --search-paths vendor . --target ${target_name} --output-path ${target_dir}

# back to ${target_dir} to produce artefacts in the expected format
cd ${target_dir}

[jperville]

I believe that this issue is related to #621 even if my context is slightly different (trying to produce an artefact outside of KAPITAN_ROOT while pwd'ed to KAPITAN_ROOT for me vs trying to build outside of KAPITAN_ROOT for @jamesglennan ).

[ramaro]

@jperville thanks for reporting! Definitely related to #621

[ramaro]

@jperville to confirm, are you setting the inventory's dependencies.type to helm or or http(s)/git?

[jperville]

@ramaro sorry for delay but I think that I am using https.

Here is the inventory class that I use to compile cert-manager files:

# cert-manager/letsencrypt support
parameters:
  cert_manager:
    namespace: cert-manager
    # version must be valid release/tag name from https://github.com/jetstack/cert-manager
    version: v0.11.0

    issuer-email: ?{plain:targets/${target_name}/cert-manager-issuer-email} # must provide
    default-issuer: letsencrypt-prod

    issuers:
      letsencrypt-staging:
        email: ${cert_manager:issuer-email}
        server: https://acme-staging-v02.api.letsencrypt.org/directory
      letsencrypt-prod:
        email: ${cert_manager:issuer-email}
        server: https://acme-v02.api.letsencrypt.org/directory

  kapitan:
    dependencies:
    - type: https
      source: https://charts.jetstack.io/charts/cert-manager-${cert_manager:version}.tgz
      unpack: true
      output_path: vendor/charts
    - type: https
      source: https://raw.githubusercontent.com/jetstack/cert-manager/${cert_manager:version}/deploy/manifests/00-crds.yaml
      unpack: false
      output_path: vendor/charts/cert-manager-${cert_manager:version}-crds/cert-manager-crds.yml
    compile:
      - output_path: pre-deploy/cert-manager
        input_type: jinja2 # hack: we only want to copy this file verbatim
        input_paths:
          - vendor/charts/cert-manager-${cert_manager:version}-crds
      - output_path: manifests
        input_type: helm
        input_paths:
          - vendor/charts/cert-manager
        helm_values:
          http_proxy: ${proxies:http_proxy}
          https_proxy: ${proxies:https_proxy}
          # workaround https://github.com/jetstack/cert-manager/issues/1371
          # by setting adding the cluster service CIDR to ${no_proxy}.
          no_proxy: ${kubernetes:service-cluster-ip-range}
          cainjector:
            enabled: false
          webhook:
            enabled: false
          prometheus:
            enabled: true
            servicemonitor:
              enabled: true
          ingressShim:
            defaultIssuerName: ${cert_manager:default-issuer}
            defaultIssuerKind: ClusterIssuer
        helm_params:
          release_name: cert-manager
          namespace: ${cert_manager:namespace}
      - output_path: manifests/cert-manager/issuers
        input_type: jsonnet
        input_paths:
          - components/features/cert-manager/main.jsonnet
        output_type: yaml