Unable to install datahub helm chart

[dhananjay-bahal]

Running this command: helm install datahub datahub/datahub returns the following error:

Error: failed pre-install: warning: Hook pre-install datahub/templates/kafka-setup-job.yml failed: Job in version "v1" cannot be handled as a Job: v1.Job.Spec: v1.JobSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Env: []v1.EnvVar: v1.EnvVar.v1.EnvVar.Value: ReadString: expects " or n, but found 3, error found in #10 byte of ...|,"value":3},{"name":|..., bigger context ...|isites-kafka:9092"},{"name":"PARTITIONS","value":3},{"name":"REPLICATION_FACTOR","value":3}],"image"|...

All the prerequisites get installed fine..

[gcaride]

Same issue here when i want to do an upgrade.

My values.yaml is like the default but with an AWS ELB and Google Authentication for React App (OIDC)

[dexter-mh-lee]

Can you try again? We recently fixed the issue!

[alexmach77]

Same issue when installing datahub with default values: helm install datahub datahub/datahub

W0308 09:11:21.473229 71646 warnings.go:70] batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob

W0308 09:11:21.638025 71646 warnings.go:70] batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob

Error: INSTALLATION FAILED: failed pre-install: timed out waiting for the condition

[dexter-mh-lee]

Can you run kubectl get pods ? You should see one of the setup jobs failing.

[alexmach77]

@dexter-mh-lee

[dexter-mh-lee]

Could you post the logs of any one of the elasticsearch-setup-job that errored out?

[dexter-mh-lee]

Ah tho it seems like elasticsearch has not spun up correctly. Could you make sure you gave the cluster enough resources? Also which environment are you running in? Is this minikube or a kubernetes cluster?

[dexter-mh-lee]

All the pods that the prerequisites spun up needs to be in Running state before deploying datahub.

[alexmach77]

Logs from elasticsearch-setup-job. It seems elastics search master is not reaching a running state.

I am using Google Kubernetes Engine (but one of the quick start small clusters). Maybe I need to switch to the normal clusters.

Events inside the elastic search master:

[dexter-mh-lee]

Yeah as mentioned here https://datahubproject.io/docs/deploy/gcp#start-up-a-kubernetes-cluster-on-gke, it requires at least 3 e2-standard nodes, mainly because of Elasticsearch taking up a lot of resources.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

[anshbansal]

Closing this issue due to inactivity. If this is still an issue please open a new issue with details of the version with which the problem can be reproduced.

[iamwaqasahmad]

helm install datahub datahub/datahub -n datahub --debug

install.go:178: [debug] Original chart version: "" install.go:195: [debug] CHART PATH: /Users/waqasahmad/Library/Caches/helm/repository/datahub-0.2.181.tgz

client.go:310: [debug] Starting delete for "datahub-elasticsearch-setup-job" Job client.go:339: [debug] jobs.batch "datahub-elasticsearch-setup-job" not found client.go:128: [debug] creating 1 resource(s) client.go:540: [debug] Watching for changes to Job datahub-elasticsearch-setup-job with timeout of 5m0s client.go:568: [debug] Add/Modify event for datahub-elasticsearch-setup-job: ADDED client.go:607: [debug] datahub-elasticsearch-setup-job: Jobs active: 0, jobs failed: 0, jobs succeeded: 0 client.go:568: [debug] Add/Modify event for datahub-elasticsearch-setup-job: MODIFIED client.go:607: [debug] datahub-elasticsearch-setup-job: Jobs active: 1, jobs failed: 0, jobs succeeded: 0 client.go:568: [debug] Add/Modify event for datahub-elasticsearch-setup-job: MODIFIED client.go:607: [debug] datahub-elasticsearch-setup-job: Jobs active: 1, jobs failed: 0, jobs succeeded: 0 client.go:568: [debug] Add/Modify event for datahub-elasticsearch-setup-job: MODIFIED client.go:607: [debug] datahub-elasticsearch-setup-job: Jobs active: 1, jobs failed: 0, jobs succeeded: 0

the job itself kubectl logs job.batch/datahub-elasticsearch-setup-job 2023/08/14 09:29:42 Waiting for: http://elasticsearch-master:9200 2023/08/14 09:29:42 Problem with request: Get "http://elasticsearch-master:9200": dial tcp 10.66.57.72:9200: connect: connection refused. Sleeping 1s 2023/08/14 09:29:43 Received 200 from http://elasticsearch-master:9200 going to use protocol: http going to use default elastic headers not using any prefix

datahub_analytics_enabled: true

GET _ilm/policy/datahub_usage_event_policy response code is 200 _ilm/policy/datahub_usage_event_policy already exists ✓

GET _index_template/datahub_usage_event_index_template response code is 200 _index_template/datahub_usage_event_index_template already exists ✓

GET _data_stream/datahub_usage_event response code is 200 _data_stream/datahub_usage_event already exists ✓ 2023/08/14 09:29:43 Command finished successfully.

[bijancot]

holly mother, i also got this issue. just happen because i set the ssl to true then turn it of with set it to false. but instead i got this kind of error

`>> GET _ilm/policy/datahub_usage_event_policy response code is 000

failed to GET _ilm/policy/datahub_usage_event_policy ! -> exiting`

anyone got same issue?

[rhysjtevans]

I had it and found my way to this thread. For others that stumble on this I resolve my issue by ensuring the SSL certs were valid. Tested by attempting to curl the elastic endpoint from within the container.

Certs were valid but had different SAN/dns entries on them