cockroach icon indicating copy to clipboard operation
cockroach copied to clipboard

roachtest: ssh_problem failed

Open cockroach-teamcity opened this issue 1 year ago • 27 comments

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test allocbench/nodes=7/cpu=8/kv/r=0/access=skew failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-amd64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-amd64 [email protected]:./cockroach
ssh: connect to host 35.227.97.34 port 22: Connection timed out
lost connection: exit status 1 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=amd64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=8
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=true
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131148 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.12-rc]
  • #130639 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.3-rc]
  • #130478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #130477 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #129312 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc]
  • #127328 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #126478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
  • #125985 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc]
/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-42659

cockroach-teamcity avatar Oct 02 '24 05:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test acceptance/event-log failed: (test_runner.go:808).func4: cluster.PutE: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/_runner-logs/ssh/ssh_055457.690636716_n1_if-hostname-teamcity.log): TRANSIENT_ERROR(ssh_problem): exit status 255 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131148 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.12-rc]
  • #130639 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.3-rc]
  • #130478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #130477 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #129312 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc]
  • #127328 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #126478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]
  • #125985 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 02 '24 05:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test cdc/scan/catchup-cold/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null failed: (cluster.go:2282).Start: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/cdc/scan/catchup-cold/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null/cpu_arch=arm64/run_1/ssh/ssh_064650.795304837_n2_run-start-script.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/cdc/scan/catchup-cold/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=16
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #131148 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.12-rc]
  • #130639 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.3-rc]
  • #130478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #130477 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #129312 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc]
  • #127328 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #126478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 02 '24 06:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test jepsen/bank-multitable/start-kill-2 failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach: signal: killed [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #131148 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.12-rc]
  • #130639 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.3-rc]
  • #130478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #130477 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #129312 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc]
  • #127328 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #126478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 02 '24 09:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test replicagc-changed-peers/restart=true failed: (cluster.go:2282).Start: failed to find services to register: failed to find 1 open ports: TRANSIENT_ERROR(open_ports): output:: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/replicagc-changed-peers/restart=true/cpu_arch=arm64/run_1/ssh/ssh_110938.234327586_n3_find-ports.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/replicagc-changed-peers/restart=true/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #131148 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.12-rc]
  • #130639 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.3-rc]
  • #130478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #130477 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #129312 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc]
  • #127328 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #126478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 02 '24 11:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test tpcc/mixed-headroom/n5cpu16 failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach: signal: killed [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=16
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=zfs
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #131148 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.12-rc]
  • #130639 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.3-rc]
  • #130478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #130477 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #129312 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc]
  • #127328 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #126478 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.7-rc]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 02 '24 11:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test cdc/filtering/session/ignore-filtering=true failed: (cluster.go:2282).Start: failed to find services to register: failed to find 1 open ports: TRANSIENT_ERROR(open_ports): output:: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/cdc/filtering/session/ignore-filtering=true/cpu_arch=arm64/run_1/ssh/ssh_070626.900238196_n2_find-ports.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/cdc/filtering/session/ignore-filtering=true/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 05 '24 07:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test ruby-pg failed: (test_runner.go:808).func4: cluster.PutLibraries: cluster.PutE: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/ruby-pg/cpu_arch=arm64/run_1/ssh/ssh_070235.093425049_n1_if-hostname-teamcity.log): TRANSIENT_ERROR(ssh_problem): exit status 255 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 05 '24 07:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test cdc/workload/kv0/nodes=5/cpu=16/ranges=100k/control failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach
Warning: Permanently added '35.223.198.254' (ECDSA) to the list of known hosts.
: signal: killed [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=16
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 05 '24 07:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test follower-reads/survival=zone/locality=regional/reads=bounded-staleness failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-amd64" failed: error persisted after 2 attempts: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-amd64 [email protected]:./cockroach
Warning: Permanently added '34.139.133.134' (ECDSA) to the list of known hosts.
client_loop: send disconnect: Broken pipe
lost connection: exit status 1 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=amd64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=true
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 05 '24 09:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 63dd3482501dbc49e9e315c02c17c28b117b651e:

test acceptance/version-upgrade failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach
Warning: Permanently added '104.197.120.193' (ECDSA) to the list of known hosts.
[email protected]: Permission denied (publickey).
lost connection: exit status 1 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 07 '24 05:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test copyfrom/crdb-nonatomic/sf=1/nodes=1 failed: (cluster.go:2282).Start: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine [email protected]:certs.tar /tmp/certs.tar1012445722: signal: killed
test artifacts and logs in: /artifacts/copyfrom/crdb-nonatomic/sf=1/nodes=1/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 09 '24 08:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test kv95/enc=false/nodes=3/size=4kb failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach
Warning: Permanently added '34.28.100.136' (ECDSA) to the list of known hosts.
: signal: killed [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=8
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 10 '24 09:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test failover/partial/lease-leader failed: (test_runner.go:808).func4: cluster.PutE: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/_runner-logs/ssh/ssh_085305.232778685_n1_if-hostname-teamcity.log): TRANSIENT_ERROR(ssh_problem): exit status 255 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=2
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 11 '24 08:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test lease-preferences/manual-violating-transfer failed: (cluster.go:2282).Start: failed to find services to register: failed to find 1 open ports: TRANSIENT_ERROR(open_ports): output:: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/lease-preferences/manual-violating-transfer/cpu_arch=arm64/run_1/ssh/ssh_103723.450227569_n4_find-ports.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/lease-preferences/manual-violating-transfer/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 11 '24 10:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test loqrecovery/half-online/workload=tpcc/rangeSize=16mb failed: (cluster.go:2282).Start: failed to find services to register: failed to find 1 open ports: TRANSIENT_ERROR(open_ports): output:: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/loqrecovery/half-online/workload=tpcc/rangeSize=16mb/cpu_arch=arm64/run_1/ssh/ssh_104326.807159852_n5_find-ports.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/loqrecovery/half-online/workload=tpcc/rangeSize=16mb/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 11 '24 10:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test cdc/scan/catchup-cold/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach: signal: killed [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=16
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 13 '24 07:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test acceptance/event-log failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach
ssh: connect to host 34.171.99.93 port 22: Connection timed out
lost connection: exit status 1 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 15 '24 05:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test allocbench/nodes=7/cpu=8/kv/r=95/ops=skew failed: (cluster.go:2282).Start: failed to find services to register: failed to find 1 open ports: TRANSIENT_ERROR(open_ports): output:: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/allocbench/nodes=7/cpu=8/kv/r=95/ops=skew/cpu_arch=arm64/run_1/ssh/ssh_055900.522952246_n4_find-ports.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/allocbench/nodes=7/cpu=8/kv/r=95/ops=skew/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=8
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 15 '24 06:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test transfer-leases/signal failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-amd64" failed: error persisted after 2 attempts: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-amd64 [email protected]:./cockroach
client_loop: send disconnect: Broken pipe
lost connection: exit status 1 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=amd64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 15 '24 07:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test ycsb/B/nodes=3 failed: (test_runner.go:808).func4: cluster.PutE: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/_runner-logs/ssh/ssh_112150.725252847_n1_if-hostname-teamcity.log): TRANSIENT_ERROR(ssh_problem): exit status 255 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=8
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 15 '24 11:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test c2c/shutdown/src/coordinator failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: error persisted after 2 attempts: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach
ssh_dispatch_run_fatal: Connection to 104.198.145.198 port 22: key type does not match
lost connection: exit status 1 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=8
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 16 '24 07:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test failover/non-system/blackhole-recv/lease=expiration failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach: signal: killed [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=2
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 16 '24 08:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test failover/non-system/crash failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach
Warning: Permanently added '34.134.73.70' (ECDSA) to the list of known hosts.
: signal: killed [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=2
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 16 '24 08:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test kv0/enc=false/nodes=3/mt-shared-process failed: (cluster.go:2395).Run: full command output in run_100053.365805833_n4_cockroach-workload-r.log: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/kv0/enc=false/nodes=3/mt-shared-process/cpu_arch=arm64/run_1/ssh/ssh_100053.392360393_n4_cockroach-workload-r.log): TRANSIENT_ERROR(ssh_problem): exit status 255
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/kv0/enc=false/nodes=3/mt-shared-process/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=8
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 16 '24 10:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test typeorm failed: (cluster.go:2282).Start: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine [email protected]:certs.tar /tmp/certs.tar1408164104
client_loop: send disconnect: Broken pipe
: signal: killed
test artifacts and logs in: /artifacts/typeorm/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 17 '24 07:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test copy/bank/rows=100000,nodes=5,txn=true failed: (cluster.go:2282).Start: failed to find services to register: failed to find 1 open ports: TRANSIENT_ERROR(open_ports): output:: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/copy/bank/rows=100000_nodes=5_txn=true/cpu_arch=arm64/run_1/ssh/ssh_071812.304350532_n4_find-ports.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/copy/bank/rows=100000_nodes=5_txn=true/cpu_arch=arm64/run_1

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=4
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 17 '24 07:10 cockroach-teamcity

roachtest.ssh_problem failed with artifacts on release-24.1.6-rc @ 5ca308c616cee5939f4be345069d4f3849617fd3:

test failover/non-system/crash failed: (test_runner.go:808).func4: cluster.PutE: put "/go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64" failed: TRANSIENT_ERROR(ssh_problem): ~ scp -r -C -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roach/.ssh/id_rsa -i /home/roach/.ssh/google_compute_engine /go/src/github.com/cockroachdb/cockroach/bin/cockroach.linux-arm64 [email protected]:./cockroach
ssh: connect to host 34.123.73.164 port 22: Connection timed out
lost connection: exit status 1 [owner=test-eng]

Parameters:

  • ROACHTEST_arch=arm64
  • ROACHTEST_cloud=gce
  • ROACHTEST_coverageBuild=false
  • ROACHTEST_cpu=2
  • ROACHTEST_encrypted=false
  • ROACHTEST_fs=ext4
  • ROACHTEST_localSSD=false
  • ROACHTEST_runtimeAssertionsBuild=false
  • ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

  • #132980 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.3]
  • #131724 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.4-rc]
  • #129915 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1.26-rc]
  • #127118 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2]
  • #122095 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
  • #112310 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
  • #108264 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.1]
  • #90695 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity avatar Oct 20 '24 08:10 cockroach-teamcity

The branch is no longer available

rail avatar Apr 09 '25 13:04 rail