infrastructure icon indicating copy to clipboard operation
infrastructure copied to clipboard

test-packet-ubuntu1604-x64-1 and test-packet-ubuntu2004-x64-2: Cannot delete workspace

Open llxia opened this issue 3 years ago • 11 comments

  • Please put the system name in the title of this issue. https://ci.adoptopenjdk.net/computer/test-packet-ubuntu1604-x64-1/

  • Link to any log file showing the problem: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/4965/console

  • Please describe the issue:

Running on [test-packet-ubuntu1604-x64-1](https://ci.adoptopenjdk.net/computer/test-packet-ubuntu1604-x64-1/) in /home/jenkins/workspace/Grinder
[Pipeline] {
[Pipeline] timeout
Timeout set to expire in 1 hr 0 min
[Pipeline] {
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
ERROR: Cannot delete workspace :Unable to delete '/home/jenkins/workspace/Grinder/jvmtest/perf/framework-benchmarks/results/20220609083424/benchmark.log'. Tried 3 times (of a maximum of 3) waiting 0.1 sec between attempts.
[Pipeline] }
[Pipeline] // timeout
[Pipeline] echo
Exception: hudson.AbortException: Cannot delete workspace: Unable to delete '/home/jenkins/workspace/Grinder/jvmtest/perf/framework-benchmarks/results/20220609083424/benchmark.log'. Tried 3 times (of a maximum of 3) waiting 0.1 sec between attempts.

llxia avatar Jun 09 '22 13:06 llxia

Same issue on https://ci.adoptopenjdk.net/computer/test-packet-ubuntu2004-x64-2/: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/4964/console

Running on [test-packet-ubuntu2004-x64-2](https://ci.adoptopenjdk.net/computer/test-packet-ubuntu2004-x64-2/) in /home/jenkins/workspace/Grinder
[Pipeline] {
[Pipeline] timeout
Timeout set to expire in 1 hr 0 min
[Pipeline] {
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
ERROR: Cannot delete workspace :Unable to delete '/home/jenkins/workspace/Grinder/jvmtest/perf/framework-benchmarks/results/20220609082844/test_metadata.json'. Tried 3 times (of a maximum of 3) waiting 0.1 sec between attempts.
[Pipeline] }
[Pipeline] // timeout
[Pipeline] echo
Exception: hudson.AbortException: Cannot delete workspace: Unable to delete '/home/jenkins/workspace/Grinder/jvmtest/perf/framework-benchmarks/results/20220609082844/test_metadata.json'. Tried 3 times (of a maximum of 3) waiting 0.1 sec between attempts.

llxia avatar Jun 09 '22 13:06 llxia

Could we mark these machines offline for now?

llxia avatar Jun 09 '22 13:06 llxia

I have marked -2 offline with a link to this issue, -1 is offline with this message:

This agent is offline because Jenkins failed to launch the agent process on it.

smlambert avatar Jun 09 '22 13:06 smlambert

@smlambert @llxia The results directory on those machines has been created as root - do you knowhow that has happened? Is it doing something in docker? In which case we need to look at the security of this...

drwxrwxr-x  3 jenkins jenkins 4096 Jun  9 08:31 deployment
drwxrwxr-x 40 jenkins jenkins 4096 Jun  9 08:31 frameworks
drwxr-xr-x  3 root    root    4096 Jun  9 08:34 results
-rwxrwxr-x  1 jenkins jenkins 3007 Jun  9 08:31 tfb
-rw-rw-r--  1 jenkins jenkins    0 Jun  9 08:32 tmpt
drwxrwxr-x 10 jenkins jenkins 4096 Jun  9 08:34 toolset

sxa avatar Jun 13 '22 12:06 sxa

I think this may have occurred as part of @joeyleeeeeee97 testing this PR for the addition of a new test (and yes, its container-based).

smlambert avatar Jun 13 '22 14:06 smlambert

@sxa FrameworkBenchmarks is running test with root in docker.. Maybe I need to reset the file mode to get it removable?

joeyleeeeeee97 avatar Jun 14 '22 02:06 joeyleeeeeee97

Can we mark https://ci.adoptopenjdk.net/computer/test-packet-ubuntu1604-x64-1/ offline? It is back online now and jobs that land on it will fail. https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/5002/

llxia avatar Jun 15 '22 14:06 llxia

Marked -1 offline with a link to this issue.

smlambert avatar Jun 15 '22 15:06 smlambert

build-alibaba-win2012r2-x64-2

two recent builds https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_x86-32_windows_testList_1/51/ https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_x86-32_windows_testList_1/52/

sophia-guo avatar Jul 11 '22 18:07 sophia-guo

build-alibaba-win2012r2-x64-2

@sophia-guo Can you put that in a separate issue please(possibly https://github.com/adoptium/infrastructure/issues/2209 if you want to re-use one) as the cause on Linux vs Windows is unlikely to be the same. Thanks.

sxa avatar Jul 12 '22 08:07 sxa

NOTE: These machines are likely to be replaced soon as part of #2666

sxa avatar Jul 15 '22 10:07 sxa

Closing as they have been decomissioned

sxa avatar Jan 12 '23 10:01 sxa