skaffold icon indicating copy to clipboard operation
skaffold copied to clipboard

Issues working with kind and ko

Open Evynglais opened this issue 11 months ago • 4 comments

Expected behavior

When using the ko builder with my kind cluster, I should be able to:

  • use the placeholder ko image name (ko://company.com/very/long/path/to/our/code) so that it's consistent with ko usage
  • have that name replaced with a "nice" name (i.e., what ko would do)
  • have that name be replaced in my Kubernetes deployment files
  • have that image be pushed to kind

Actual behavior

If I use the placeholder ko image name in skaffold.yaml, the image doesn't get loaded into kind. It does get replaced in my knative-service.yaml. It does not get replaced in my knative-containersource.yaml. The generated image name is awful, i.e., if it was ko://company.com/very/long/path/to/our/code, it comes out as kind.local/project/company_com_very_long_path_to_our_code.

If I use a non-placeholder image name, such as "test-image", the image gets loaded into kind. It does get replaced in my knative-service.yaml, but doesn't in my knative-containersource.yaml. This obviously interferes with ko which isn't suitable for my workflow.

Information

  • Skaffold version: v2.13.2 (I think... latest using Cloud Code)

  • Operating system: Ubuntu 24.04.1 LTS using WSL2 on Windows 11 23H2

  • Installed via: Cloud Code

  • Contents of skaffold.yaml:

apiVersion: skaffold/v4beta11
kind: Config
build:
  artifacts:
  - image: ko://company.com/very/long/path/to/our/code
    ko:
      main: ./very/long/path/to/code
      dependencies:
        paths:
        - "**/*.go"
        - go.*
manifests:
  rawYaml:
    - config/backend-container-source.yaml
    - config/frontend-knative-service.yaml
deploy:
  kubectl: {}
  • VS Code Launch Configuration:
        {
            "name": "Kubernetes: Run/Debug",
            "type": "cloudcode.kubernetes",
            "request": "launch",
            "skaffoldConfig": "${workspaceFolder}/skaffold.yaml",
            "watch": true,
            "cleanUp": true,
            "portForward": true,
            "imageRegistry": "kind.local/project",
        }
  • knative-service.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: frontend
spec:
  template:
    spec:
      volumes:
        - name: config-volume
          configMap:
            name: frontend-config
      containers:
        - name: frontend
          image: ko://company.com/very/long/path/to/our/code
          ports:
            - containerPort: 8080
  • knative-containersource.yaml
apiVersion: sources.knative.dev/v1
kind: ContainerSource
metadata:
  name: backend
spec:
  sink:
    ref:
      apiVersion: serving.knative.dev/1
      kind: Service
      name: frontend
  template:
    spec:
      volumes:
        - name: config-volume
          configMap:
            name: backend-config
      containers:
        - name: backend
          image: ko://company.com/very/long/path/to/our/code
          ports:
            - containerPort: 40000

Evynglais avatar Dec 11 '24 16:12 Evynglais

I did some digging into the code, and found the TransformAllowlist in visitor.go which doesn't have the Knative ContainerSource in it, so that explains why I'm unable to set the image automatically there.

Evynglais avatar Dec 11 '24 23:12 Evynglais

Got around that by allowing it with the resourceSelector config:

resourceSelector:
  allow:
  - groupKind: ContainerSource.sources.knative.dev
    image: [ .* ]

Evynglais avatar Dec 11 '24 23:12 Evynglais

So now the only blocker is the ko image not being loaded into kind if I use the automatic name from the ko template... plus the awful naming of said image.

Evynglais avatar Dec 11 '24 23:12 Evynglais

I'm also unable to get debug working. Doesn't appear that Skaffold is calling ko with the debug flag set (otherwise, ko would be wrapping delve into the image, which I can see it isn't).

Edit: realised Skaffold is using an old version of ko that doesn't have the debug flag; could use some guidance on what I need to do to get debugging working with ko on Skaffold. Not sure if the long automatic image name is also a side-effect of the outdated version of ko.

Evynglais avatar Dec 12 '24 00:12 Evynglais