wire-server-deploy
wire-server-deploy copied to clipboard
wireapp
wire-server installation failed
Hello,
I got this error, Anyone know what to do ?
bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait
Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition
Can you please help?
If you add --debug to the command, you might get some more information on what's going on.
On Sun, Jun 13, 2021 at 12:20 PM JonnyX11 @.***> wrote:
Hello,
I got this error, Anyone know what to do ?
bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait
Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition
Can you please help?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wireapp/wire-server-deploy/issues/470, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2SFN5QQY6TWCH5IG5JXTTSSA6DANCNFSM46TTVQJA .
--
勇気とユーモア
bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait --debug history.go:56: [debug] getting history for release wire-server upgrade.go:123: [debug] preparing upgrade for wire-server upgrade.go:131: [debug] performing update for wire-server upgrade.go:303: [debug] creating upgraded release for wire-server client.go:282: [debug] Starting delete for "cassandra-migrations" Job client.go:122: [debug] creating 1 resource(s) client.go:491: [debug] Watching for changes to Job cassandra-migrations with timeout of 5m0s client.go:519: [debug] Add/Modify event for cassandra-migrations: ADDED client.go:558: [debug] cassandra-migrations: Jobs active: 0, jobs failed: 0, jobs succeeded: 0 client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED client.go:558: [debug] cassandra-migrations: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
upgrade.go:369: [debug] warning: Upgrade "wire-server" failed: pre-upgrade hooks failed: timed out waiting for the condition Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition helm.go:81: [debug] pre-upgrade hooks failed: timed out waiting for the condition UPGRADE FAILED main.newUpgradeCmd.func2 helm.sh/helm/v3/cmd/helm/upgrade.go:157 github.com/spf13/cobra.(*Command).execute github.com/spf13/[email protected]/command.go:850 github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/[email protected]/command.go:958 github.com/spf13/cobra.(*Command).Execute github.com/spf13/[email protected]/command.go:895 main.main helm.sh/helm/v3/cmd/helm/helm.go:80 runtime.main runtime/proc.go:225 runtime.goexit runtime/asm_amd64.s:1371
bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait --debug history.go:56: [debug] getting history for release wire-server upgrade.go:123: [debug] preparing upgrade for wire-server upgrade.go:131: [debug] performing update for wire-server upgrade.go:303: [debug] creating upgraded release for wire-server client.go:282: [debug] Starting delete for "cassandra-migrations" Job client.go:122: [debug] creating 1 resource(s) client.go:491: [debug] Watching for changes to Job cassandra-migrations with timeout of 5m0s client.go:519: [debug] Add/Modify event for cassandra-migrations: ADDED client.go:558: [debug] cassandra-migrations: Jobs active: 0, jobs failed: 0, jobs succeeded: 0 client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED client.go:558: [debug] cassandra-migrations: Jobs active: 1, jobs failed: 0, jobs succeeded: 0 client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED client.go:282: [debug] Starting delete for "elasticsearch-index-create" Job client.go:122: [debug] creating 1 resource(s) client.go:491: [debug] Watching for changes to Job elasticsearch-index-create with timeout of 5m0s client.go:519: [debug] Add/Modify event for elasticsearch-index-create: ADDED client.go:558: [debug] elasticsearch-index-create: Jobs active: 0, jobs failed: 0, jobs succeeded: 0 client.go:519: [debug] Add/Modify event for elasticsearch-index-create: MODIFIED client.go:558: [debug] elasticsearch-index-create: Jobs active: 1, jobs failed: 0, jobs succeeded: 0 upgrade.go:369: [debug] warning: Upgrade "wire-server" failed: pre-upgrade hooks failed: timed out waiting for the condition Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition helm.go:81: [debug] pre-upgrade hooks failed: timed out waiting for the condition UPGRADE FAILED main.newUpgradeCmd.func2 helm.sh/helm/v3/cmd/helm/upgrade.go:157 github.com/spf13/cobra.(*Command).execute github.com/spf13/[email protected]/command.go:850 github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/[email protected]/command.go:958 github.com/spf13/cobra.(*Command).Execute github.com/spf13/[email protected]/command.go:895 main.main helm.sh/helm/v3/cmd/helm/helm.go:80 runtime.main runtime/proc.go:225 runtime.goexit runtime/asm_amd64.s:1371
Are you installing the demo stuff, or the production stuff? Which pages in the documentation are you following exactly? Can you give us a complete log of the commands you type, and their output, from the beginning?
I don't have an idea right now of what might be going wrong, but providing this information might help somebody else find a solution for you easier than without that information.
Sorry for taking so long to answer, I completely missed the notification for your message in my inbox.
How much RAM and how many CPUs do you have on the server? Can you give us the results of:
cat /proc/cpuinfo | grep processor | wc -l
free -m
As well as the result of «uptime» before and after the helm upgrade command ?
Thanks.
Hello @JonnyX11
we are always happy to help, but we kindly ask people who reach out for help to be cooperative. So, would you mind following the template structure that we put in place for these kind of scenarios? You might notice some significant overlap between the questions lisyted in the template and the ones asked by @arthurwolf
Additionally to what is in the template, please provide us with some more context. For example, is it your first installation or are you trying to upgrade an installation? If the latter, did you check the release notes for the new version you installed - also for wire-server? Please note, you will find the code for each chart in there.
Lastly, please provide us with the output of kubectl get pods -o wide --all-namespaces.
Thank you.
first of all thank you very much for your help!
Basic information
-
On-premises: Yes
-
Cloud-Provider: nope
-
Installation type: production
-
Kubernetes version: bash-4.4# kubectl version Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:15:20Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
-
Helm version: bash-4.4# helm version version.BuildInfo{Version:"v3.5.2", GitCommit:"", GitTreeState:"", GoVersion:"go1.16"}
-
Installed with Kubespray:
-
(Helm) Charts version: bash-4.4# helm show chart wire/wire-server | grep -m 1 version version: 2.108.0
-
List of installed top-level charts:
-
Other related technologies + version:
bash-4.4# ansible --version ansible 2.9.12 config file = /wire-server-deploy/ansible/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /nix/store/407c5v2x920kri79w0sb9gb1g82q28mg-python3.8-ansible-2.9.12/lib/python3.8/site-packages/ansible executable location = /nix/store/407c5v2x920kri79w0sb9gb1g82q28mg-python3.8-ansible-2.9.12/bin/ansible python version = 3.8.8 (default, Feb 19 2021, 11:04:50) [GCC 10.2.0]
How did you install Wire?
according to the productions docs: https://docs.wire.com/how-to/install/version-requirements.html#persistence https://docs.wire.com/how-to/install/helm-prod.html#helm-prod
I used the docker option for helm https://docs.wire.com/how-to/install/dependencies.html#dependencies
I worked on Ubuntu 18.04 for all virtual machines
| Name | Amount | CPU Cores | Memory (GB) | Disk Space (GB) |
|---|---|---|---|---|
| Cassandra | 3 | 2 | 4 | 80 |
| MinIO | 3 | 1 | 2 | 400 |
| ElasticSearch | 3 | 1 | 2 | 60 |
| Kubernetes³ | 3 | 6¹ | 8 | 40 |
| Restund⁴ | 2 | 1 | 2 | 10 |
| Per-Server Totals | — | 11 CPU Cores | 18 GB Memory | 590 GB Disk Space |
| Admin Host² | 1 | 1 | 4 | 40 |
| Asset Host² | 1 | 1 | 4 | 100 |
| Per-Server Totals with Admin and Asset Hosts | — | 13 CPU Cores | 26 GB Memory | 730 GB Disk Space |
Please see the outputs here :
bash-4.4# kubectl version Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:15:20Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
bash-4.4# kubectl get pods -o wide --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default cassandra-migrations-d5g22 0/1 Completed 0 5m19s 10.233.65.23 kubenode02
bash-4.4# cat /proc/cpuinfo | grep processor | wc -l 4 bash-4.4# bash-4.4# free -m total used free shared buff/cache available Mem: 16013 579 12578 1 2855 15135 Swap: 2047 0 2047 bash-4.4#
bash-4.4# helm upgrade --install wire-server wire/wire-server -f my-wire-server/values.yaml -f my-wire-server/secrets.yaml --wait --debug history.go:56: [debug] getting history for release wire-server upgrade.go:123: [debug] preparing upgrade for wire-server upgrade.go:131: [debug] performing update for wire-server upgrade.go:303: [debug] creating upgraded release for wire-server client.go:282: [debug] Starting delete for "cassandra-migrations" Job client.go:122: [debug] creating 1 resource(s) client.go:491: [debug] Watching for changes to Job cassandra-migrations with timeout of 5m0s client.go:519: [debug] Add/Modify event for cassandra-migrations: ADDED client.go:558: [debug] cassandra-migrations: Jobs active: 0, jobs failed: 0, jobs succeeded: 0 client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED client.go:558: [debug] cassandra-migrations: Jobs active: 1, jobs failed: 0, jobs succeeded: 0 client.go:519: [debug] Add/Modify event for cassandra-migrations: MODIFIED client.go:282: [debug] Starting delete for "elasticsearch-index-create" Job client.go:122: [debug] creating 1 resource(s) client.go:491: [debug] Watching for changes to Job elasticsearch-index-create with timeout of 5m0s client.go:519: [debug] Add/Modify event for elasticsearch-index-create: ADDED client.go:558: [debug] elasticsearch-index-create: Jobs active: 0, jobs failed: 0, jobs succeeded: 0 client.go:519: [debug] Add/Modify event for elasticsearch-index-create: MODIFIED client.go:558: [debug] elasticsearch-index-create: Jobs active: 1, jobs failed: 0, jobs succeeded: 0 upgrade.go:369: [debug] warning: Upgrade "wire-server" failed: pre-upgrade hooks failed: timed out waiting for the condition Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition helm.go:81: [debug] pre-upgrade hooks failed: timed out waiting for the condition UPGRADE FAILED main.newUpgradeCmd.func2 helm.sh/helm/v3/cmd/helm/upgrade.go:157 github.com/spf13/cobra.(*Command).execute github.com/spf13/[email protected]/command.go:850 github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/[email protected]/command.go:958 github.com/spf13/cobra.(*Command).Execute github.com/spf13/[email protected]/command.go:895 main.main helm.sh/helm/v3/cmd/helm/helm.go:80 runtime.main runtime/proc.go:225 runtime.goexit runtime/asm_amd64.s:1371
bash-4.4# helm list --all
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
cassandra-external default 2 2021-06-20 11:43:21.448671888 +0000 UTC deployed cassandra-external-2.108.0
databases-ephemeral default 2 2021-06-20 11:43:41.528915369 +0000 UTC deployed databases-ephemeral-2.108.0
elasticsearch-external default 2 2021-06-20 11:43:33.670740997 +0000 UTC deployed elasticsearch-external-2.108.0
fake-aws default 2 2021-06-20 11:49:41.841649782 +0000 UTC deployed fake-aws-2.108.0
minio-external default 4 2021-06-20 11:45:08.514165984 +0000 UTC deployed minio-external-2.108.0
smtp default 2 2021-06-20 11:54:14.683598062 +0000 UTC deployed demo-smtp-2.108.0 1.0
wire-server default 9 2021-06-20 12:50:43.972757468 +0000 UTC failed wire-server-2.108.0
Hello @JonnyX11
we are always happy to help, but we kindly ask people who reach out for help to be cooperative. So, would you mind following the template structure that we put in place for these kind of scenarios? You might notice some significant overlap between the questions lisyted in the template and the ones asked by @arthurwolf
Additionally to what is in the template, please provide us with some more context. For example, is it your first installation or are you trying to upgrade an installation? If the latter, did you check the release notes for the new version you installed - also for wire-server? Please note, you will find the code for each chart in there.
Lastly, please provide us with the output of
kubectl get pods -o wide --all-namespaces.Thank you.
The details was OK on the last post ? it was clear ?