nebari icon indicating copy to clipboard operation
nebari copied to clipboard

Add K8s logging in the CI when QHub deploy fails

Open aktech opened this issue 3 years ago • 7 comments

At the moment when the K8s deploy fails, there is not much information for the user in the CI logs for the user to figure out the root cause of the problem.

A nice approach would be to have a step run on failure, which logs everything from the minikube kubernetes cluster. I have done something similar in https://github.com/dask/dask-gateway

aktech avatar Aug 26 '21 09:08 aktech

@tylerpotts This could possibly affect our debugging of the monitoring PR where the kubernetes tests are failing.

Adam-D-Lewis avatar Sep 02 '21 13:09 Adam-D-Lewis

@aktech I was thinking that we could tie in https://github.com/Quansight/qhub/issues/771 to this issue. The idea would be to run qhub support on failure or something similar to create an artifact on failure that could be used for inspection.

costrouc avatar Sep 02 '21 14:09 costrouc

@costrouc sounds great, would be great if we can print that in the CI step itself instead of an artifact.

aktech avatar Sep 02 '21 14:09 aktech

Sounds great to me. We'll make sure that the support command does that as well.

costrouc avatar Sep 02 '21 14:09 costrouc

This issue has been automatically marked as stale because there was no recent activity in 60 days. Remove the stale label or add a comment, otherwise, this issue will automatically be closed in 7 days if no further activity occurs.

github-actions[bot] avatar Nov 02 '21 01:11 github-actions[bot]

This issue was closed because it has been stalled for 7 days with no activity.

github-actions[bot] avatar Nov 10 '21 01:11 github-actions[bot]

If CI fails, run the nebari support tool and save the output.

iameskild avatar Apr 14 '22 15:04 iameskild

I think for now we can consider this closed. The nebari support grabs a bunch of k8s logs and saves them to the user's local machine in case they want to review them or share them with us over an issue.

iameskild avatar May 29 '23 16:05 iameskild