no ingress found for the GitServer

[hugoarroyo]

Describe the bug no ingress found for the GitServer

To Reproduce

What did you do?

1. Install EDP version 3.81

2. I'm able to configure github server configuration following the documentation https://epam.github.io/edp-install/quick-start/integrate-github/

3. creating the Web Applications with Gin Framework

What did you expect to see?

The status of the component to be green

What did you see instead? Under which circumstances?

The status of the component is Status failed. The reason is no ingress found for the Git Server

Kubernetes cluster type:

AWS EKS 1.29

[SergK]

Hello @hugoarroyo

Thank you for contacting us with the details of the issue you're facing with the EPAM Delivery Platform (EDP) version 3.8.1 and the integration of a Git server on an AWS EKS 1.29 Kubernetes cluster.

Based on the information you've provided, it seems the issue revolves around the absence of an ingress for the GitServer, which is leading to the component's status being marked as "failed" due to "no ingress found for the Git Server."

Ingress is indeed a mandatory part of onboarding components within the EDP architecture, especially because it is utilized by WebHooks. The codebase-operator within EDP attempts to retrieve the ingress to set the proper URL for the webhook. That said, there's an alternative approach that you might find useful. This approach involves explicitly defining the URL for webhooks, bypassing the operator's need to search for an ingress object.

You can specify the webhook URL in the values.yaml as shown in the following snippet from the EDP Tekton documentation (If you installed EDP using GitOps approach and created GitServer with helm):

gitServers:
  my-gitlab:
    gitProvider: gitlab
    host: gitlab.com
    webhook:
      url: "http://gitlab.example.com"
      skipWebhookSSLVerification: false
    eventListener:
      enabled: true
      resources:
        requests:
          memory: "64Mi"
          cpu: "50m"
        limits:
          memory: "128Mi"
          cpu: "500m"
      nodeSelector: {}
      tolerations: []
      affinity: {}

This configuration allows the codebase operator to use the specified URL to create webhooks in GitLab without relying on the Ingress value. It's important to note that while ingress is required to receive events from GitHub/GitLab, this workaround allows you to proceed if your current focus is exploratory.

Alternatively, you can update your GitServer configuration as follows:

kind: GitServer
metadata:
  name: my-gitlab
spec:
  gitHost: gitlab.com
  gitProvider: gitlab
  gitUser: git
  httpsPort: 443
  nameSshKeySecret: "ci-gitlab"
  sshPort: 22
  skipWebhookSSLVerification: false
  webhookUrl: "http://gitlab.example.com"

Adopting either of these approaches should resolve the issue with the ingress for the GitServer and ensure the component's status is correctly reflected.

Please don't hesitate to reach out if you have any further questions or require additional assistance.

Best regards, Sergiy

[hugoarroyo]

Thanks @SergK for your prompt response.

I have another question, Can I install EDP using istio instead of ngix ingress?

Regards Hugo

[SergK]

Yes, it is indeed possible to utilize the EPAM Delivery Platform (EDP) without the default NGINX Ingress or even the Istio Ingress controller. During the installation process, you have the flexibility to disable the ingress setup for all EDP components. This approach allows you to use alternative methods for traffic routing according to your specific infrastructure requirements. Here's how you can disable ingress for key components:

EDP Portal: You can disable the ingress for the EDP Portal by modifying the values.yaml file available in the GitHub repository. Set the ingress.enabled parameter to false. Access the configuration here: EDP Headlamp values.yaml.

Tekton Part: Similarly, for the Tekton component, adjust the values.yaml file to disable ingress by setting ingress.enabled to false. The relevant configuration can be found here: EDP Tekton values.yaml.

Once you have disabled the ingress installation for these components, it is crucial to configure the webhookUrl in the GitServer specification. This URL should direct traffic to the solution you've chosen for handling your ingress needs:

kind: GitServer
metadata:
  name: my-gitlab
spec:
  gitHost: gitlab.com
  gitProvider: gitlab
  gitUser: git
  httpsPort: 443
  nameSshKeySecret: "ci-gitlab"
  sshPort: 22
  skipWebhookSSLVerification: false
  webhookUrl: "http://gitlab.example.com" <<<<<<<<<<<<<<<<<< DEFINE THIS VALUE, it's going to be used as webhook URL in you git server

On the Istio side, managing traffic flow can be straightforwardly achieved by defining VirtualService and Gateway configurations. These resources enable you to route incoming traffic to the desired endpoints, such as the Tekton Dashboard, EDP Portal, and Event Listener, based on your requirements.

We do the same approach when configuring EDP with https://kubernetes.io/docs/concepts/services-networking/gateway/

[SergK]

Meanwhile, I'll try to share configs (values.yaml) next week for the cases you've mentioned, including the Ingress"less" installation.

[SergK]

@hugoarroyo, please consult https://github.com/epam/edp-install/blob/master/deploy-templates/values-no-ingress.yaml. The idea is to adjust routing stuff to the Istio objects that are responsible for traffic routing in your case.

[hugoarroyo]

@SergK thank you kindly for the information, I will keep reviewing the documentation, also I wanted to check if EPAM in the near future can support github packages as a container registry

[SergK]

@hugoarroyo we do have support for packages https://epam.github.io/edp-install/operator-guide/package-registry/ but let me check with the team, I think we can add out-of-the-box support for the GitHub container registry

[SergK]

@hugoarroyo here is what you can do. Create Container Registry 'DockerHub'

Then edit kaniko-docker-config and regred in your epd namespace, and align them to format:

          {
            "auths": {
              "ghcr.io": {
                "auth": "$(echo -n "$GIT_USERNAME:$GIT_PASSWORD" | base64 -w0)"
              }
            }
          }

And this should work

[hugoarroyo]

Hi @SergK

I tried what you recommend, but after doing that I can only see

"60b58e48-2ea0-4a6c-ac9b-2de7b73ef290","url":"**https://ghcr.io/v2/**"} {"level":"info","ts":"2024-04-09T00:42:49Z","msg":"Connection failed","controller":"secret","controllerGroup":"","controllerKind":"Secret","Secret":{"name":"regcred","namespace":"edp"},"namespace":"edp","name":"regcred","reconcileID":"60b58e48-2ea0-4a6c-ac9b-2de7b73ef290","error":"**http status code 401 Unauthorized**"}

I know that my github package configuration works because I'm using the same user and token for pulling and pushing containers, this is my config file

{ "auths": { "https://ghcr.io": { "auth": "this is my base64 user password" } } } But I can't pass the 401 unauthorized response

Thanks

[SergK]

ok, I've found a working configuration:

1. Ensure you are using dockerhub as registry
2. Then update kaniko-docker-config and regcred registry endpoint to {"auths":{"https://ghcr.io":
3. It is ok to have 401 issue for now (we will fix it in the next release)
4. Update edp-configmap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: edp-config
  namespace: edp
data:
  container_registry_host: ghcr.io
  ....

6. Run build pipeline and see success build in logs:

To summarize, you should follow github approach and just update values in two places: https://ghcr.io for docker cred and config map container_registry_host

[hugoarroyo]

@SergK

Thanks for the recommendations, but after doing that I'm getting the following error

error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "ghcr.io/verxtex-simulacra/api-dtc-gin-demo:0.1.0-SNAPSHOT.7": creating push check transport for ghcr.io failed: GET https://ghcr.io/token?scope=repository%3Averxtex-simulacra%2Fapi-dtc-gin-demo%3Apush%2Cpull&service=ghcr.io: DENIED: denied

I know is not the token or the permissions.

This how the config map looks like

  container_registry_host: ghcr.io
  container_registry_space: verxtex-simulacra
  container_registry_type: dockerhub
  dns_wildcard: host
  edp_name: edp
  edp_version: 3.8.0
  platform: kubernetes

And I'm using { "auths": { "https://ghcr.io": { "auth": "this is my base64 user password" } } }

[hugoarroyo]

[SergK]

@hugoarroyo does your user has

    read:packages
    Download packages from GitHub Package Registry

Are you using Organization or personal GitHub account?

[hugoarroyo]

@SergK Is an organizational account, also I have write packages grants. With this token I can manually get and publish docker containers in to github packages, I'm missing something?

[SergK]

@hugoarroyo, please check our video, which we've just released, that describes how we resolved the issue: https://www.youtube.com/watch?v=b9dpNSpq1AI&ab_channel=ThePlatformTeam

[hugoarroyo]

@SergK

Thank you so much for the video, looking forward to a new version with GitHub packages as provider for the container registry

Regards Hugo

[SergK]

Feature is already in the master and will be released in a week as part of the new platform version 3.9.0