cyclops-ctrl detects self-hosted GitHub repository as a Helm repository

[halworsen]

Describe the bug When attempting to add a self-hosted GitHub repository with a login redirect as a template store, cyclops-ctrl detects the repository as a Helm repository because the GitHub instance responds to HEAD /index.yaml with a 302 redirect to a login page (which is ultimately a 200 response).

The resulting error message seems to occur when Cyclops attempts to load chart values from the "Helm" repo:

yaml: line 28: mapping values are not allowed in this context

To Reproduce Steps to reproduce the behavior:

1. Store your templates in a repo on a GitHub instance with a login redirect
2. Add a TemplateStore resource that references said repo
3. Validate the store in the Cyclops UI
4. See error in cyclops-ctrl logs

Alternatively, this is how I was reproducing it when trying to figure out why things were breaking:

1. Create a TemplateStore that references an non-Helm repo endpoint which responds to HEAD /index.yaml with 200
2. Portforward to cyclops-ctrl pod and curl "http://localhost:port/templates/initial with the appropriate repo, path and commit params
3. Get {"message":"Error loading template initial values","description":"yaml: line 28: mapping values are not allowed in this context"} or similar back

Expected behavior Cyclops should not make assumptions on repository type based on what URLs look like or responses from remote servers.

Preferably, users should have to explicitly define repository type in the TemplateStore resource through a repoType field or similar.

Screenshots

Desktop (please complete the following information):

• OS: macOS Sonoma 14.6.1
• Browser: Firefox 130.0.1

Additional context Tested and seen on cyclops-ctrl built from 526673a1faf4bacdc42b817053eac17d788ea788

Relevant helm repo check: https://github.com/cyclops-ui/cyclops/blob/526673a1faf4bacdc42b817053eac17d788ea788/cyclops-ctrl/internal/template/helm.go#L131-L150

When replacing the entire isHelmRepo function body with return false, nil, cyclops-ctrl falls through to assuming it's a git repository and everything works as expected.

[petar-cvit]

Hey @halworsen, thanks for the issue and the detailed description!

I agree that users should be able to define the template source, but back then, we decided to try to infer it based on the repo provided for a simpler onboarding.

We can add a dropdown to select the template source, and if nothing is selected, cyclops-ctrl can infer it as it does now.

Will post any updates here!

[petar-cvit]

Hey @halworsen, we made a new release candidate v0.14.0-rc.2 that supports adding template source when adding a new template to Cyclops.

You can install it with the following command:

kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.14.0-rc.2/install/cyclops-install.yaml && kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.14.0-rc.2/install/demo-templates.yaml

As you suggested in the issue, we added a new field sourceType to the TemplateStore and Module CRDs. The command above will install a new version of the CRDs and bump the versions of the controller and the UI.

You can now add your template through the UI and set the appropriate template type.

Let me know if the rc resolves your issue, and we can include it in a new release

[halworsen]

Thank you for the blazing fast fix @petar-cvit!

With the new rc and a fresh install of my template stores/auth rules:

• The curl test (with the new sourceType query param) gives the expected response
• The template fails validation in the UI when clicking the validate button (inspecting my browser's network logs, it looks like the UI may be forgetting a sourceType parameter?), but
• The values are fetched properly when creating a new module from the UI (so it really does work)

I also noticed that the source type selection only appears in the UI when creating new templates, but not while editing them. Our use case would involve deploying the TemplateStores directly, so I don't mind much but just a heads up in case that wasn't intended.

[petar-cvit]

@halworsen, that's awesome! I'll look into fixing the issue with the template validation and adding source type selection on template edit.

We will release v0.14.0 soon with these changes and let you know when it is out

[halworsen]

Error still happens when editing modules as well. Seemingly the same problem as with the validation, no sourceType param in the UI's API call

[petar-cvit]

@halworsen I just pushed a fix for updating templates. If you want, I can build another rc with the latest changes, or you can wait a couple of days until we properly test it out and roll out the v0.14.0. In case you can wait for the v0.14.0, you can update your templates using kubectl in the meantime

[halworsen]

Template stores are fine now, I ran into the error while editing the deployed module from the UI (that is, at /modules/<modulename>/edit).

Also, no need to release a new rc, I've been building my own images with some of the open PRs patched in to run a POC Cyclops deployment, so I'm happy to wait for a proper 0.14.0 release 😃

[petar-cvit]

Oh, sorry, I misread and thought you had issues with editing templates.

Current implementation does not allow you to change the sourceType through the UI while editing modules. If you had an existing Module before you bumped the CRDs and cyclops deployments, it wont populate the sourceType on edit and you could run into issues.

I suggest creating a new Module, and Module edits should work then. In the meantime, we will work on adding template source type when editing a template

[halworsen]

What I mean is if you create a new module using the template with sourceType = git, then edit the new module through the UI, it errors out and won't allow you to edit the module at all from the UI, including chart values:

I don't think we'd ever run into the need for editing sourceType on the modules directly.

[petar-cvit]

Hey @halworsen, you are right. The sourceType was not passed in the edit module screen. We will test the PR and release v0.14.0 by the beginning of next week.

Thanks for pointing this out! 🧡

[petar-cvit]

Hey @halworsen, we have just released v0.14.0, which should resolve all the issues regarding template source. You can install it with the following command:

kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.14.0/install/cyclops-install.yaml && kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.14.0/install/demo-templates.yaml

Let me know if it works for you as expected and we can close the issue

[halworsen]

Can confirm v0.14.0 solves the issues with template sourcing, thanks! 😃

[petar-cvit]

Thanks for the update!