stratisd icon indicating copy to clipboard operation
stratisd copied to clipboard

make test-loop leaving tmp test directories

Open tasleson opened this issue 7 years ago • 10 comments

After running make test-loop you can find a number of test directories remain in /tmp

e.g.

$ ls -ldh /tmp/stratis_testing.*
drwxr-xr-x. 2 root root 40 Nov 17 16:06 /tmp/stratis_testing.4AYPeJJtD6FY
drwxr-xr-x. 2 root root 40 Nov 17 16:14 /tmp/stratis_testing.58nr4K0n7zHU
drwxr-xr-x. 2 root root 40 Nov 17 16:06 /tmp/stratis_testing.A6W54JdzX9oV
drwxr-xr-x. 2 root root 40 Nov 17 16:14 /tmp/stratis_testing.bsNEmCgB5Yaw
drwxr-xr-x. 2 root root 40 Nov 17 16:06 /tmp/stratis_testing.exZDZoQ8Z86U
drwxr-xr-x. 2 root root 40 Nov 17 16:14 /tmp/stratis_testing.faWsRYpLUqHX
drwxr-xr-x. 2 root root 40 Nov 17 16:06 /tmp/stratis_testing.hkJIfQ7D57Sy
drwxr-xr-x. 2 root root 40 Nov 17 16:13 /tmp/stratis_testing.jtCsWB5UGdk7
drwxr-xr-x. 2 root root 40 Nov 17 16:06 /tmp/stratis_testing.Mq6k02k0bXRa
drwxr-xr-x. 2 root root 40 Nov 17 16:14 /tmp/stratis_testing.QnDRyTR4DiOr
drwxr-xr-x. 2 root root 40 Nov 17 16:14 /tmp/stratis_testing.VoVFTntYps6p
drwxr-xr-x. 2 root root 40 Nov 17 16:06 /tmp/stratis_testing.zMMDKtcYyudo

This didn't happen before in the past AFAIK.

tasleson avatar Nov 17 '17 21:11 tasleson

Reverting 29a30f11d139dd805b0bca0f495b4ca3ce58eb44 the test exhibits the previous behavior of cleaning up the /tmp/stratis_testing.* directories.

tasleson avatar Nov 17 '17 21:11 tasleson

Very strange. It really looks like the test cleanup code ought to get rid of these.

mulkieran avatar Nov 27 '17 15:11 mulkieran

Is this still happening?

mulkieran avatar Nov 12 '18 14:11 mulkieran

Yes, slightly different directory name though

drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testing2wLiE3
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testing9YAfqD
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testingfAzER3
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testingHeVgSV
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testingkNchv0
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testingNKGx5J
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testingoVuv0c
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testingqJa5s3
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testingwI2seM
drwxr-xr-x. 2 root root 40 Nov 14 12:20 stratis_testingXaaVVR

tasleson avatar Nov 14 '18 18:11 tasleson

I believe this has been fixed by https://github.com/stratis-storage/ci/pull/52.

@bgurney-rh If you agree, can you close this?

mulkieran avatar Dec 09 '19 14:12 mulkieran

@mulkieran that change unmounts the directories, but doesn't remove them. Also, the teardown.sh script is only intended to run after a failed test (where the state of things after failure is unknown).

Do we want something where the test removes the "/tmp/stratis_test*" directory it created, but only after the test succeeds? And for failed tests, do we want teardown.sh to remove those directories?

While thinking through most of the possible outcomes, this question arises: in the event of a failed test (or some other suboptimal test run), is there anything of value that would be in these directories that we would want to allow the user to diagnose after a test run?

bgurney-rh avatar Dec 09 '19 16:12 bgurney-rh

It doesn't make sense to call left over test directories a bug...

mulkieran avatar Dec 09 '19 16:12 mulkieran

@mulkieran that change unmounts the directories, but doesn't remove them. Also, the teardown.sh script is only intended to run after a failed test (where the state of things after failure is unknown).

Do we want something where the test removes the "/tmp/stratis_test*" directory it created, but only after the test succeeds? And for failed tests, do we want teardown.sh to remove those directories?

While thinking through most of the possible outcomes, this question arises: in the event of a failed test (or some other suboptimal test run), is there anything of value that would be in these directories that we would want to allow the user to diagnose after a test run?

Ok. Let's hold this a while longer.

mulkieran avatar Dec 09 '19 16:12 mulkieran

It doesn't make sense to call left over test directories a bug...

In most cases, this is true, though there is the potential to have too many directories in /tmp. An ext4 filesystem would have a maximum of 64,000 subdirectories (minus 2, due to "." and ".."); however, /tmp is in /tmpfs, which may have a different limit (I don't know if the limit is lower or higher).

Looking at one of our most heavily used instances that runs the stratisd testing, I saw 199 directories with the "stratis_testing*" prefix, over the past four months or so.

bgurney-rh avatar Dec 09 '19 17:12 bgurney-rh

Could the temp directories be cleaned up by the test code itself?

The CI teardown script is designed to clean up after a failed test, where the state of things is not known; therefore, the teardown.sh script wouldn't be a good place to clean up temp directories, as they would only be cleaned up after failures, instead of after all runs (success or failure).

bgurney-rh avatar Dec 12 '19 19:12 bgurney-rh