odo
odo copied to clipboard
redhat-developer
Test failures while running `test-cmd-debug` test suite on ppc64le
I have made changes to the scripts/openshiftci-presubmit-all-tests.sh and scripts/configure-installer-tests-cluster.sh for Power on the lines of the Z changes.
Also done the setup required like creating the secret to pull redhat registry images. Currently I am trying to run the scripts/openshiftci-presubmit-all-tests.s and seeing the below issues for test-cmd-debug test suite:
Summarizing 5 Failures:
[Fail] odo debug command tests odo debug on a nodejs:latest component [It] should expect a ws connection when tried to connect on default debug port locally
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_run.go:34
[Fail] odo debug command tests odo debug on a nodejs:latest component [It] check that machine output debug information works
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_run.go:34
[Fail] odo debug command tests odo debug on a nodejs:latest component [It] should expect a ws connection when tried to connect on different debug port locally and remotely
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_run.go:34
[Fail] odo debug command tests odo debug info should work on a odo component [It] should start a debug session and run debug info on a closed debug session
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_run.go:34
[Fail] odo debug command tests odo debug info should work on a odo component [It] should start a debug session and run debug info on a running debug session
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_run.go:34
Ran 5 of 177 Specs in 66.287 seconds
FAIL! -- 0 Passed | 5 Failed | 0 Pending | 172 Skipped
Ginkgo ran 1 suite in 1m11.122493463s
Test Suite Failed
make: *** [Makefile:284: test-cmd-debug] Error 1
Attaching detailed test run and some post failure investigation steps - odo_test-cmd-debug_failures_ppc64le.txt
@amitkrout: Closing this issue.
In response to this:
Same failures are reported in #3491 and #3497. /close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@amitkrout Facing the following issues in test-cmd-debug test suite:-
Summarizing 4 Failures:
[Fail] odo debug command tests odo debug info should work on a odo component [It] should start a debug session and run debug info on a closed debug session
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_http.go:50
[Fail] odo debug command tests odo debug on a nodejs:latest component [It] should expect a ws connection when tried to connect on default debug port locally
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_http.go:50
[Fail] odo debug command tests odo debug info should work on a odo component [It] should start a debug session and run debug info on a running debug session
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_http.go:50
[Fail] odo debug command tests odo debug on a nodejs:latest component [It] should expect a ws connection when tried to connect on different debug port locally and remotely
/usr/local/go/src/github.com/openshift/odo/tests/helper/helper_http.go:50
Ran 5 of 184 Specs in 365.520 seconds
FAIL! -- 1 Passed | 4 Failed | 0 Pending | 179 Skipped
Ginkgo ran 1 suite in 6m10.374469509s
Test Suite Failed
make: *** [Makefile:332: test-cmd-debug] Error 1
Attaching the detailed spec failure log - odo_test-cmd-debug_failures_ppc64le_1.txt
@sarveshtamba: Reopened this issue.
In response to this:
/reopen
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
Failure here is with port-forward
Running odo with args [odo debug port-forward --local-port 46097 --context /tmp/094321299]
error while requesting: Get http://localhost:46097: dial tcp [::1]:46097: connect: connection refused
[odo] I0820 02:30:04.665636 28967 preference.go:209] The path for preference file is /tmp/094321299/config.yaml
[odo] I0820 02:30:04.665672 28967 util.go:734] HTTPGetRequest: https://raw.githubusercontent.com/openshift/odo/master/build/VERSION
[odo] I0820 02:30:04.665829 28967 util.go:755] Response will be cached in /tmp/odohttpcache for 1h0m0s
[odo] I0820 02:30:04.665979 28967 util.go:768] Cached response used.
[odo] Started port forwarding at ports - 46097:5858
try 1 of 12
error while requesting: Get http://localhost:46097: EOF
[odo] E0820 02:30:09.914754 28967 portforward.go:400] an error occurred forwarding 46097 -> 5858: error forwarding port 5858 to pod 404f5d2b14e482084843e7956674e0a470022426fbf2fcf619b0008433239aaf, uid : exit status 1: 2020/08/20 06:30:09 socat[3081252] E connect(5, AF=2 127.0.0.1:5858, 16): Connection refused
try 2 of 12
[...]
[odo] I0820 02:31:04.471938 28967 odo.go:72] Could not get the latest release information in time. Never mind, exiting gracefully :)
Last output from http://localhost:46097:
Deleting project: ioqtwquywm
Running odo with args [odo project delete ioqtwquywm -f]
[odo] I0820 02:31:05.785982 29021 util.go:734] HTTPGetRequest: https://raw.githubusercontent.com/openshift/odo/master/build/VERSION
[odo] I0820 02:31:05.786156 29021 util.go:755] Response will be cached in /tmp/odohttpcache for 1h0m0s
[odo] I0820 02:31:05.977446 29021 util.go:768] Cached response used.
[odo] ✓ Deleted project : ioqtwquywm
[odo] ⚠ Warning! Projects are deleted from the cluster asynchronously. Odo does its best to delete the project. Due to multi-tenant clusters, the project may still exist on a different node.
[odo] I0820 02:31:06.127310 29021 odo.go:72] Could not get the latest release information in time. Never mind, exiting gracefully :)
Deleting dir: /tmp/094321299
• Failure [139.593 seconds]
odo debug command tests
/usr/local/go/src/github.com/openshift/odo/tests/integration/cmd_debug_test.go:19
odo debug info should work on a odo component
/usr/local/go/src/github.com/openshift/odo/tests/integration/cmd_debug_test.go:107
should start a debug session and run debug info on a closed debug session [It]
/usr/local/go/src/github.com/openshift/odo/tests/integration/cmd_debug_test.go:131
Failed after 12 retries. Content in http://localhost:46097 doesn't include 'WebSockets request was expected'.
@kadel Not sure this is happening due to the wrong port-forward that you mentioned here - https://github.com/openshift/odo/issues/3840 . Can you please look into this.
[odo] Started port forwarding at ports - 46097:5858 try 1 of 12 error while requesting: Get http://localhost:46097: EOF [odo] E0820 02:30:09.914754 28967 portforward.go:400] an error occurred forwarding 46097 -> 5858: error forwarding port 5858 to pod 404f5d2b14e482084843e7956674e0a470022426fbf2fcf619b0008433239aaf, uid : exit status 1: 2020/08/20 06:30:09 socat[3081252] E connect(5, AF=2 127.0.0.1:5858, 16): Connection refused try 2 of 12 [...]
[odo] Started port forwarding at ports - 46097:5858
try 1 of 12
error while requesting: Get http://localhost:46097: EOF
[odo] E0820 02:30:09.914754 28967 portforward.go:400] an error occurred forwarding 46097 -> 5858: error forwarding
port 5858 to pod 404f5d2b14e482084843e7956674e0a470022426fbf2fcf619b0008433239aaf, uid : exit status 1:
2020/08/20 06:30:09 socat[3081252] E connect(5, AF=2 127.0.0.1:5858, 16): Connection refused
try 2 of 12
[...]
@prietyc123 Port forward is happening in the right port but the connection has been refused while requesting for http://localhost:46097. I have never seen this in our prow CI. @sarveshtamba what is the frequency of this issue you are hitting ?
[odo] Started port forwarding at ports - 46097:5858 try 1 of 12 error while requesting: Get http://localhost:46097: EOF [odo] E0820 02:30:09.914754 28967 portforward.go:400] an error occurred forwarding 46097 -> 5858: error forwarding port 5858 to pod 404f5d2b14e482084843e7956674e0a470022426fbf2fcf619b0008433239aaf, uid : exit status 1: 2020/08/20 06:30:09 socat[3081252] E connect(5, AF=2 127.0.0.1:5858, 16): Connection refused try 2 of 12 [...]
[odo] Started port forwarding at ports - 46097:5858 try 1 of 12 error while requesting: Get http://localhost:46097: EOF [odo] E0820 02:30:09.914754 28967 portforward.go:400] an error occurred forwarding 46097 -> 5858: error forwarding port 5858 to pod 404f5d2b14e482084843e7956674e0a470022426fbf2fcf619b0008433239aaf, uid : exit status 1: 2020/08/20 06:30:09 socat[3081252] E connect(5, AF=2 127.0.0.1:5858, 16): Connection refused try 2 of 12 [...]@prietyc123 Port forward is happening in the right port but the connection has been refused while requesting for http://localhost:46097. I have never seen this in our prow CI. @sarveshtamba what is the frequency of this issue you are hitting ?
Seeing this frequently - almost every time I run this test suite.
Issues go stale after 90d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle stale
Stale issues rot after 30d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle rotten /remove-lifecycle stale