Feature Request: allowed_os

[bapt]

Please add a allowed_os, so we can skip some tests on a given os

[jmmv]

How do you define the values that allowed_os would take? The plain OS name as returned by uname?

Is that sufficient without version numbers?

Also, how useful is that for Linux, where the distribution name might be more important depending on what you are trying to do?

Why do you need to make test cases specific to a OS?

[bapt]

uname -r should be enough I do have tests in pkg that will only be run on Linux/FreeBSD but not on OSX (because related to elf binary analysis) and some only runnng on OSX and not on the others due to MACH-O analysis for example

[jmmv]

Hmm, but determining those details based on the OS name seems fragile. It's the same issue as configure scripts making decisions based on the OS name: they usually work, until they don't. These dependencies (elf, mach-o, etc.) should be checked for explicitly so that, if the OS ever changes underneath, the test will detect the change regardless of the OS name. Shouldn't they?

(I know some things are "clearly OS X"... but even then, trying to avoid checks based on the OS name, if possible, seems worthwhile.)

[bapt]

So far I have been working around with aft-sh and it worked by simply not defining some test if uname -r replies Darwin. But it is not ideal:

https://github.com/freebsd/pkg/blob/master/tests/frontend/jpeg.sh#L9 https://github.com/freebsd/pkg/blob/master/tests/frontend/001sanity.sh#L4

[jmmv]

I still need to make up my mind regarding this feature request, but one comment on your code: you should use atf_skip when you are on Darwin to clearly mark the test as not applicable to the host instead of just silently exiting. Using atf_skip from within the test is not much different than relying on a metadata property to do the skip for you.