Mark requirements

[twied]

Introduces two new functions, "push_requirement" and "pop_requirement", that govern a stack of current requirements. If the requirements are unmet, tests are skipped. This has several positive effects:

• Allows to run as non-root the subset of pjdfstests that work as non-root
• Allows to selectively disable tests for certain file types (closes #45)
• Simplifies test plan calculation ("1..n" lines)

While at it, this series also fixes a bug in rename/09.t and rename/10.t and provides more information in the output (not quite fixing #27, but certainly helping).

[ngie-eign]

I really appreciate all of the efforts you put in.. but I’m going to softly decline your proposal. I have a proposal I’m going to put out to @asomers / @pjd soon to move away from expressing the tests in TAP to pytest.

pytest a much more expressive/pluggable test infrastructure that avoids many of the shortcomings you’re identifying with testcase indexing and the like, and does M x N style testing with fixtures much more gracefully than the adhoc setup and tear down logic in pkdfstest does today.

Moving to pytest for the test infrastructure would allow pjdfstest to focus more on what needs to be tested, instead of how it’s being tested.

[twied]

Rewriting the tests in Python sounds great!

I believe that the MR would actually help in that endeavor. The patches add (or rather: make explicit) the list of requirements for the tests but do not alter the structure of the test files. That should be handy information to have and not get in the way of any work already done.

Any idea how long the rewrite will take? I can see the merge request from March last year, but no activity after the first commit. If the rewrite takes longer, I think you might want to merge this MR anyway, as it is low risk, fixes a bug, and provides a solution to an issue that users would otherwise have to wait for quite long.

[twied]

@ngie-eign how is the Python rewrite coming along? And did you have a chance yet to consider the arguments I made in the last comments? I have rebased the patches onto current master, in case you change your mind. Let me know if there's anything else I can do to convince you that this PR is worthwhile.

[gaul]

Can we merge this PR? I am interested in s3fs using pjdfstest again and having better support for requirements will avoid a lot of hard-coding demonstrated in https://github.com/s3fs-fuse/s3fs-fuse/pull/1882.

[twied]

What an unexpected call from the past 😀

As a favor to @gaul I rebased the merge request and resolved the conflicts. Pretty much straight forward, the only notable change is that I dropped commit 969737476a ("Use different names for variables in outer and inner loop"), as upstream has the very similar 9a90c29d4a ("Fix variable name collision in the rename tests") in place.

[gaul]

To make a point about how painful this is, I configure s3fs to run pjdfstest this way:

    prove -rv \
        ../../pjdfstest/tests/chflags/*.t \
        ../../pjdfstest/tests/chmod/0[4689].t \
        ../../pjdfstest/tests/chmod/10.t \
        ../../pjdfstest/tests/chown/0[4689].t \
        ../../pjdfstest/tests/chown/10.t \
        ../../pjdfstest/tests/ftruncate/0[147-9].t \
        ../../pjdfstest/tests/ftruncate/1[0134].t \
        ../../pjdfstest/tests/granular/*.t \
        ../../pjdfstest/tests/link/*.t \
        ../../pjdfstest/tests/mkdir/0[347-9].t \
        ../../pjdfstest/tests/mkdir/1[12]*.t \
        ../../pjdfstest/tests/mkfifo/0[3478].t \
        ../../pjdfstest/tests/mkfifo/1*.t \
        ../../pjdfstest/tests/mknod/0[479].t \
        ../../pjdfstest/tests/mknod/10.t \
        ../../pjdfstest/tests/open/0[49].t \
        ../../pjdfstest/tests/open/1*.t \
        ../../pjdfstest/tests/open/2[0-134].t \
        ../../pjdfstest/tests/posix_fallocate/*.t \
        ../../pjdfstest/tests/rename/0[2-36-8].t \
        ../../pjdfstest/tests/rename/1[15-9].t \
        ../../pjdfstest/tests/rename/22.t \
        ../../pjdfstest/tests/rmdir/0[13-59].t \
        ../../pjdfstest/tests/rmdir/1[02-5].t \
        ../../pjdfstest/tests/symlink/0[13479].t \
        ../../pjdfstest/tests/symlink/1*.t \
        ../../pjdfstest/tests/truncate/0[147-9].t \
        ../../pjdfstest/tests/truncate/1[0134].t \
        ../../pjdfstest/tests/unlink/0[347-8].t \
        ../../pjdfstest/tests/unlink/1[02-4].t \
        ../../pjdfstest/tests/utimensat/0[1-58-9].t

[gaul]

I guess the rust rewrite never panned out. Does anything block merging this PR?