Add support to pull & build OCI kustomize manifests

[FrenchBen]

trafficstars

Fixes https://github.com/kubernetes-sigs/kustomize/issues/5134

I created a new OCI package to support this feature, which caused a few go.mod updates, thus adding a lot of noise.

Core/relevant files changed are:

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        new file:   examples/oci-test/deployment.yaml
        new file:   examples/oci-test/kustomization.yaml
        new file:   api/internal/oci/pull.go
        new file:   api/internal/oci/fluxcd.go

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   api/krusty/localizer/runner.go
        modified:   api/krusty/remoteloader_test.go
        modified:   api/loader/loader.go
        modified:   api/loader/fileloader.go
        modified:   examples/remoteBuild.md
        modified:   kustomize/commands/build/build.go
        modified:   kustomize/commands/localize/localize.go
        modified:   kustomize/commands/localize/localize_test.go
        modified:   kustomize/go.mod
        modified:   kustomize/go.sum
        modified:   go.work
        modified:   go.work.sum

The example in examples/oci-test is to build the OCI manifest, that is used in the golang tests - Ideally this moves under the kubernetes-sigs/kustomize repo package repository, once this PR is merged

[k8s-ci-robot]

This PR has multiple commits, and the default merge method is: merge. You can request commits to be squashed using the label: tide/merge-method-squash

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

Welcome @FrenchBen!

It looks like this is your first PR to kubernetes-sigs/kustomize 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/kustomize has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

[k8s-ci-robot]

Hi @FrenchBen. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[natasha41575]

Hi @FrenchBen, thank you for your PR.

It seems that this PR is specifically targeting adding oci support to kustomize localize, without adding it kustomize build. Since scope of localize is limited to only support the same types of remote URLs that build supports, it doesn't make sense to add it to localize but not build.

IMO we can accept either of these options:

• 2 separate PRs, one that supports oci for kustomize build and one for kustomize localize, but the one that supports oci for kustomize build must go in first.
• A PR that supports oci for both build and localize. This is what I would prefer and is probably easiest since both commands can share the same underlying code.

I left another comment on https://github.com/kubernetes-sigs/kustomize/issues/5134#issuecomment-1520858199 to further clarify the requirements.

[FrenchBen]

@natasha41575 Makes sense, and was on my list of things to add :) I'll get started on the implementation, which will follow the Git model.

[natasha41575]

/hold /ok-to-test

[FrenchBen]

After implementing kustomize build, I had to re-write the history for kustomize localize so that the 2 commands leveraged the same code. Merged in latest changes from master to resolve conflicts

[FrenchBen]

/retest

[FrenchBen]

@natasha41575 @KnVerey Could I get a trigger of the CI on this?

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: FrenchBen Once this PR has been reviewed and has the lgtm label, please assign natasha41575 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[FrenchBen]

With the recent changes from kube-openapi to adopt gnostic-model instead of gnostic, it had lead to a few broken dependencies (Flux PKG being one of them) As a result, I have pulled in the Flux OCI pkg logic into its own file, with the hope of removing it in the future.

ref: https://github.com/kubernetes/kube-openapi/pull/402 + https://github.com/google/gnostic/issues/397

[natasha41575]

@FrenchBen Could you clean up the PR title and commits to ease review?

[FrenchBen]

@natasha41575 which commits do you want to see squashed? Do you want me to shrink them to ~10 commits?

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[natasha41575]

@natasha41575 which commits do you want to see squashed? Do you want me to shrink them to ~10 commits?

I just mean to clean it up so that they are organized in a way that makes sense. E.g. maybe one commit can add your OCI library, another can add the functionality + tests to build, another to localize, and another can update docs.

Each commit should have one clear purpose so that it is easy to understand what each does. The commit messages become the release notes. We can always edit them during release but it is easier if they make sense from the get-go.

[ccureau]

Is there anything that I can do to help get this approved? I'm blocked waiting on this PR to be accepted.

EDIT: Seems this is no longer needed as https://github.com/kubernetes-sigs/kustomize/pull/5167 has been merged.

[blairdrummond]

Is there anything that I can do to help get this approved? I'm blocked waiting on this PR to be accepted.

EDIT: Seems this is no longer needed as #5167 has been merged.

I think #5167 only applies to the helm chart inflator, if I'm not mistaken

[blairdrummond]

It seems like this doesn't allow digest-pinned images eh?

# works
kustomize build 'oci://blairdrummond/cert-manager-manifests2'

# resolves to this sha
# $ crane digest blairdrummond/cert-manager-manifests2
# sha256:30be0d117a210ba4ff54c6f7b61a8ee38e7be0b1280ca90e0ec76f5411db0172

# doesn't work
kustomize build 'oci://blairdrummond/cert-manager-manifests2@sha256:30be0d117a210ba4ff54c6f7b61a8ee38e7be0b1280ca90e0ec76f5411db0172'

[devantler]

I understand the need for this feature. I myself want to set up an OCI registry that houses a collection of services. A similar setup can be seen here:

OCI Registry Example

In my experience configuring Helm charts can be a daunting task, and I often only need to template a handful of the Helm values I change. As a result, I prefer to reuse my flux-gitops HelmReleases, rather than creating and managing multiple versions across different clusters. While Git sources allow for this, the current process can be cumbersome.

In order to reference these services with flux kustomizations, I'm required to create a separate flux kustomization for each service. However, if the feature in question were available, I could simply create one flux kustomization that points to a standard kustomization file. This file would then reference services as an OCI reference including a path.

While I can't speak for all workflows, I believe this proposed feature would be beneficial to many. It provides a flexible approach to deploy from OCI, and enhances reusability where it's currently lacking.

Could we possibly prioritize this PR? 🙏🏻

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[flickerfly]

I don't know that I can, but I'd like to /remove-lifecycle stale

[FrenchBen]

Catching up on this PR. https://github.com/kubernetes-sigs/kustomize/pull/5167 adding support in fetching OCI helm charts.

I believe the changes here are still needed if we want generic resources to live in an OCI repo. Can I get some feedback from the maintainers, so any updates don't go stale as they have in the past?

cc. @knverey @koba1t @varshaprasad96

[flickerfly]

/lint