Allow Reuse of Code Across Configs

[adawalli]

Expected behavior

I would expect that blocks of code could propagate or be reused within "required" config modules.

Actual behavior

When trying to break up a large skaffold config file into multiple configs (separate files), some of the reuse opportunities go away. Consider the following "monolith"

apiVersion: skaffold/v2beta24
kind: Config
metadata:
  name: monolith
build:
  tagPolicy:
    gitCommit:
      variant: CommitSha
  artifacts:
    - image: artifact1
      context: artifact1/
    - image: artifact2
      context: artifact2/
  local:
    useDockerCLI: true
    useBuildkit: true

In this case, we can build two artifacts that share a common set of docker settings (useDockerCLI and useBuildKit).

As the complexity (and number of artifacts grows), we decide to get fancy and use configs!

apiVersion: skaffold/v2beta24
kind: Config
metadata:
  name: child1
build:
  tagPolicy:
    gitCommit:
      variant: CommitSha
  artifacts:
    - image: artifact1
      context: artifact1/
---
apiVersion: skaffold/v2beta24
kind: Config
metadata:
  name: child2
build:
  tagPolicy:
    gitCommit:
      variant: CommitSha
  artifacts:
    - image: artifact2
      context: artifact2/
---
apiVersion: skaffold/v2beta24
kind: Config
metadata:
  name: parent
requires:
  - configs: [child1, child2]
build:
  tagPolicy:
    gitCommit:
      variant: CommitSha
  local:
    useDockerCLI: true
    useBuildkit: true

Note: In real life, I would have moved child1 and child2 to separate files, but this is just for the sake of simplicty Now, I go to render the output skaffold diagnose --yaml-only -f skaffold-multi.yaml --module parent

apiVersion: skaffold/v2beta26
kind: Config
metadata:
  name: child1
build:
  artifacts:
  - image: artifact1
    context: /tmp/test/artifact1
    docker:
      dockerfile: Dockerfile
  tagPolicy:
    gitCommit: {}
  local:
    concurrency: 1
deploy:
  logs:
    prefix: container
---
apiVersion: skaffold/v2beta26
kind: Config
metadata:
  name: child2
build:
  artifacts:
  - image: artifact2
    context: /tmp/test/artifact2
    docker:
      dockerfile: Dockerfile
  tagPolicy:
    gitCommit: {}
  local:
    concurrency: 1
deploy:
  logs:
    prefix: container
---
apiVersion: skaffold/v2beta26
kind: Config
metadata:
  name: parent
build:
  tagPolicy:
    gitCommit:
      variant: CommitSha
  local:
    useDockerCLI: true
    useBuildkit: true
deploy:
  logs:
    prefix: container

Uh-oh, I can see that the two children aren't using my local context for Docker (or my tagging strategy)....since they have their own build stanzas.

Things I have tried (unsuccessfully) as work-arounds

• Patching local context into children configs from parent. Unfortunately, this doesn't work since wildcard contexts aren't supported in JSONPATCH
• Using a global YAML Anchore with "local" context - I can't seem to get this to work as the parent's anchors don't seem to be available to the children/config

The only thing I have found that works is to copy the same config options into all of the children configs, which is

• Ugly and reduces readability of the config files
• Prone to Errors (did I remember to set Buildkit back to false everywhere?)

It's very possible that I am missing something obvious as I am fairly new to Skaffold, so any suggestions would be greatly appreciated!

Information

• Skaffold version: v1.35.0
• Operating system: Ubuntu 18.04.6 LTS (Bionic)
• Installed via: Installed from releases page on Github

Steps to reproduce the behavior

please see above

[adawalli]

I think I would even consider using a patch copy command in a pinch, but I don't think you can reference a "required" configs section in that way

[adawalli]

@briandealwis - let me know if I need to provide better clarification. Thanks!

[briandealwis]

This is great @adawalli. We've also heard people wanting to be able to:

• set a tag policy, or default-repo, across configs
• override the build-env across the configs (e.g., use GCB instead of local build)

[tomasko-labuda]

Hello, has this been somehow addressed in recent versions? Is there a way to share configuration between modules? E.g. If you have complex tag policy that you want to use in all modules, do you need to copy it to and maintain it across all the modules? Thank you.