TektonHub CR not able to clone

[Elyytscha]

Expected Behavior

Creates a working Tektonhub with the default tekton catalog

Actual Behavior

Fails with cloning the default tekton catalog repo

Steps to Reproduce the Problem

kubectl apply -f tektonhub.yml

apiVersion: operator.tekton.dev/v1alpha1
kind: TektonHub
metadata:
  name: hub
spec:
  api:
    catalogRefreshInterval: 30m
    secret: tekton-hub-api
  customLogo: {}
  db: {}
  default: {}
  options:
    disabled: false
  targetNamespace: tekton-pipelines

this will create the following configmap

# Secret: name: tekton-hub-api
kind: ConfigMap
apiVersion: v1
metadata:
  name: tekton-hub-api
  namespace: tekton-pipelines
  labels:
    app: tekton-hub-api
    operator.tekton.dev/operand-name: tektoncd-hub
data:
  CATALOGS: |
    - name: tekton
      org: tektoncd
      type: community
      provider: github
      url: https://github.com/tektoncd/catalog
      revision: main
  CATALOG_REFRESH_INTERVAL: 30m
  CATEGORIES: |
    - Automation
    - Build Tools
    - CLI
    - Cloud
    - Code Quality
    - Continuous Integration
    - Deployment
    - Developer Tools
    - Image Build
    - Integration & Delivery
    - Git
    - Kubernetes
    - Messaging
    - Monitoring
    - Networking
    - Openshift
    - Publishing
    - Security
    - Storage
    - Testing
  DEFAULT: |
    scopes:
      - rating:read
      - rating:write
  SCOPES: ''

which will result in the following error

{"level":"info","ts":1711167219.9637597,"caller":"git/git.go:49","msg":"cloning","service":"git","name":"git","path":"/tmp/catalog/tekton"}
{"level":"error","ts":1711167220.0836065,"caller":"git/git.go:149","msg":"git fetch --recurse-submodules=yes origin main : error exit status 128 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:59\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"error","ts":1711167220.0947776,"caller":"git/git.go:149","msg":"git pull --recurse-submodules=yes origin : error exit status 1 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:62\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"info","ts":1711167220.0948365,"caller":"git/git.go:63","msg":"Failed to pull originerrexit status 1","service":"git","name":"git"}
{"level":"error","ts":1711167220.0962138,"caller":"git/git.go:149","msg":"git checkout main : error exit status 1 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:65\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"error","ts":1711167220.0962665,"caller":"catalog/syncer.go:237","msg":"exit status 1clone failed","service":"syncer","action":"process","stacktrace":"github.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:237\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"info","ts":1711167220.1104074,"caller":"app/gorm_logger.go:76","msg":"UPDATE \"sync_jobs\" SET \"status\"='error',\"updated_at\"='2024-03-23 04:13:40.096' WHERE \"sync_jobs\".\"deleted_at\" IS NULL AND \"id\" = 1","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:214","duration":"[14.027ms]","rows":1}

Additional Info

• Kubernetes version:

  Output of kubectl version:

$ oc version
Client Version: v4.2.0-alpha.0-2008-g286cfa5
Kustomize Version: v5.0.1
Server Version: 4.15.0-0.okd-2024-03-10-010116
Kubernetes Version: v1.28.2-3598+6e2789bbd58938-dirty

• Tekton Pipeline version:

  v0.53.3

• Tekton Operator version:

  v0.69.1

installed via OLM and operatorhub

[piyush-garg]

@PuneetPunamiya please take a look at this

[PuneetPunamiya]

@Elyytscha I tried to reproduce this issue, but was not able to reproduce it. All pods were up and running.

Also seeing your error it seems like it was a network issue error. Can you try it once if you can still reproduce it ?

[Elyytscha]

I can still reproduce it:

apiVersion: operator.tekton.dev/v1alpha1
kind: TektonHub
metadata:
  name: hub
spec:
  api:
    catalogRefreshInterval: 30m
    secret: tekton-hub-api
  customLogo: {}
  db: {}
  default: {}
  options:
    disabled: false
  targetNamespace: tekton-pipelines

{"level":"info","ts":1716835976.1408136,"caller":"catalog/syncer.go:104","msg":"processing the queue","service":"syncer","action":"run"}
{"level":"info","ts":1716835976.142004,"caller":"app/gorm_logger.go:76","msg":"SELECT * FROM \"sync_jobs\" WHERE \"sync_jobs\".\"status\" = 'queued' AND \"sync_jobs\".\"deleted_at\" IS NULL ORDER BY created_at,\"sync_jobs\".\"id\" LIMIT 1","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:217","duration":"[1.089ms]","rows":1}
{"level":"info","ts":1716835976.154482,"caller":"app/gorm_logger.go:76","msg":"UPDATE \"sync_jobs\" SET \"status\"='running',\"updated_at\"='2024-05-27 18:52:56.142' WHERE \"sync_jobs\".\"deleted_at\" IS NULL AND \"id\" = 1","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:214","duration":"[12.366ms]","rows":1}
{"level":"info","ts":1716835976.1553483,"caller":"app/gorm_logger.go:76","msg":"SELECT * FROM \"catalogs\" WHERE \"catalogs\".\"id\" = 1 AND \"catalogs\".\"deleted_at\" IS NULL","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:229","duration":"[0.716ms]","rows":1}
{"level":"info","ts":1716835976.155447,"caller":"git/git.go:49","msg":"cloning","service":"git","name":"git","path":"/tmp/catalog/tekton"}
{"level":"error","ts":1716835976.2589736,"caller":"git/git.go:149","msg":"git fetch --recurse-submodules=yes origin main : error exit status 128 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:59\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"error","ts":1716835976.2688093,"caller":"git/git.go:149","msg":"git pull --recurse-submodules=yes origin : error exit status 1 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:62\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"info","ts":1716835976.2688737,"caller":"git/git.go:63","msg":"Failed to pull originerrexit status 1","service":"git","name":"git"}
{"level":"error","ts":1716835976.2704506,"caller":"git/git.go:149","msg":"git checkout main : error exit status 1 ;output: ","service":"git","name":"git","stacktrace":"github.com/tektoncd/hub/api/pkg/git.git\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:149\ngithub.com/tektoncd/hub/api/pkg/git.(*client).Fetch\n\t/go/src/github.com/tektoncd/hub/api/pkg/git/git.go:65\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:235\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"error","ts":1716835976.2705216,"caller":"catalog/syncer.go:237","msg":"exit status 1clone failed","service":"syncer","action":"process","stacktrace":"github.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Process\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:237\ngithub.com/tektoncd/hub/api/pkg/service/catalog.(*syncer).Run.func1\n\t/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:105"}
{"level":"info","ts":1716835976.2864943,"caller":"app/gorm_logger.go:76","msg":"UPDATE \"sync_jobs\" SET \"status\"='error',\"updated_at\"='2024-05-27 18:52:56.27' WHERE \"sync_jobs\".\"deleted_at\" IS NULL AND \"id\" = 1","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:214","duration":"[15.865ms]","rows":1}
{"level":"info","ts":1716835976.2876918,"caller":"app/gorm_logger.go:76","msg":"SELECT count(*) FROM \"sync_jobs\" WHERE \"sync_jobs\".\"status\" = 'queued' AND \"sync_jobs\".\"deleted_at\" IS NULL","service":"syncer","file":"/go/src/github.com/tektoncd/hub/api/pkg/service/catalog/syncer.go:186","duration":"[1.065ms]","rows":1}
{"level":"info","ts":1716835976.2877243,"caller":"catalog/syncer.go:191","msg":"queued job count: 0","service":"syncer","action":"next"}

[PuneetPunamiya]

Have you upgraded Hub from previous version or did you fresh install Hub ?

[Elyytscha]

i freshly installed the hub, it happens everyime i want to install the hub from scratch

[PuneetPunamiya]

I tried this again by fresh installing hub, i don't see any issue I think you have upgraded from a previous version of operator

[Elyytscha]

I can confirm with Tekton Operator v0.70.0 its now getting past this step, now it syncs the task repo successfully as I interpret the logs. but it still shows:

should I open a new issue regarding this?

i attached the logs for now over here tekton-hub-api-7c8f95779-np5h2-tekton-hub-api.log tekton-hub-ui-5fb499c944-5g7rk-tekton-hub-ui.log

[PuneetPunamiya]

Can you hit this url and check if you can see the resources in the api <API_URL>/v1/resources

[Elyytscha]

yes I can, but I think due to your last comment I found the issue: API_URL is empty

Modified ENV
window.config = {
  API_URL: '',
  API_VERSION: 'v1',
  AUTH_BASE_URL: '',
  REDIRECT_URI: '',
  CUSTOM_LOGO_MEDIA_TYPE: '',
  CUSTOM_LOGO_BASE64_DATA: '',
};

https://github.com/tektoncd/hub/blob/main/docs/DEPLOYMENT.md#update-ui-configmap Could you point me to an example or the docs how its possible to set the API_URL in the TektonHub CR? I think then we can close the issue.

[PuneetPunamiya]

If you are installing Hub on openshift in that case the API_URL, AUTH_BASE_URL and REDIRECT_URL should be set by the operator only, can you see the values in configMap i.e. tekton-hub-ui configMap

And if in case you are using it on other k8s cluster then for that we might have to add a support for that where you can provide the url and UI will be served on that url

[PuneetPunamiya]

As of today you can add these fields only in the Hub CR https://github.com/tektoncd/operator/blob/main/docs/TektonHub.md

[Elyytscha]

If you are installing Hub on openshift in that case the API_URL, AUTH_BASE_URL and REDIRECT_URL should be set by the operator only, can you see the values in configMap i.e. tekton-hub-ui configMap

And if in case you are using it on other k8s cluster then for that we might have to add a support for that where you can provide the url and UI will be served on that url

I use OKD with the original tekton operator, because the tekton operator for openshift is not available on okd.

so i think thats why its not working, i also have to manually create the ingress for UI and API

As of today you can add these fields only in the Hub CR https://github.com/tektoncd/operator/blob/main/docs/TektonHub.md

i can't see where i can configure there the URL, do you mean i can write the API_URL into the db secret?

[PuneetPunamiya]

i can't see where i can configure there the URL, do you mean i can write the API_URL into the db secret?

What I meant was as of today it is not supported to add the API_URL

[Elyytscha]

i can't see where i can configure there the URL, do you mean i can write the API_URL into the db secret?

What I meant was as of today it is not supported to add the API_URL

Should we open a Bug ticket for this? Because as I see this, without this set, the hub cr is basically unuseable.

We could also think about automatically creating the ingress for API and UI so the API_URL could then be set based on the created ingress

[PuneetPunamiya]

Yeah, that makes sense, would you mind to create another issue for that

[error418]

I am also running into this issue when trying to clone a repository using a ssh key. To gain more information about the issue I exec'd into the container and tried to pull the custom catalog from the command line

$ cd /tmp/catalog/tekton
$ git pull
No user exists for uid 65532
fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists

Do not get misdirected by the output claiming to check your access rights. The error seems to be triggered by the missing uid in the image, causing ssh to exit abnormally.

Conclusion

I am suspecting that the missing uid 65532 in the tekton-hub-api container causes the cloning issues.

Solution

One of the two

• set user id 65532 for the user hub in the tekton-hub-api image
• operator needs to set correct uid used by the image when creating the manifest

Edit

I just confirmed this issue by u0-execing into the container, creating a user with the missing uid and successfully pulling the repository in /tmp/catalog/tekton