support using a variable (or expression) for conditional skipping, not just a dependency

[jab]

I went through the issue tracker and I think this exact issue has not been discussed yet:

It looks like the current support for conditional skipping is limited to checking for whether a dependency is available.[1]

But some blocks need to be skipped in other situations, e.g. when running with a too-old Python version, or under an alternative Python implementation such as PyPy.

As prior art, sphinx.ext.doctest has a :skipif: option that allows conditionally skipping doctest blocks using an arbitrary Python expression (e.g. :skipif: sys.version_info < (3,). Something like that would be perfect here.

Or if it's easier, perhaps the existing doctest-requires directive could be expanded to accept a variable rather than a dependency name, e.g.

# docs/foo.rst

.. doctest-requires:: cpython

    >>> float('nan') is float('nan')  # True for PyPy
    False

and a conftest.py could define the variable in the doctest_namespace:

# docs/conftest.py

@pytest.fixture(autouse=True)
def add_doctest_globals(doctest_namespace):
    doctest_namespace['cpython'] = sys.implementation.name == 'cpython'

Is there any interest in considering this?

Thanks!

[1] Both based on my testing as well as the README:

Similarly, in Sphinx .rst documentation, whole code example blocks can be conditionally skipped if a dependency is not available.

.. doctest-requires:: asdf

   >>> import asdf
   >>> asdf.open('file.asdf')

[bsipocz]

Having a conditional feature would be nice, I suppose we just need to converge on whether to have it as an option for the currently existing doctest-requires or add a new skip-if (or preferably calling it doctest-skip?).

Do you have any time to work on a PR for this @jab?

[pllim]

Why is doctestplus not using the built-in skipif from doctest? 🤔

[bsipocz]

I don't have access to code atm, but is the sphinx doctest extension the same as the built-in doctest? I would have guessed no.

OTOH, we certainly support a few things from it already so it's worth comparing whether e.g. testsetup works automatically or not.

[jab]

Right, sphinx.ext.doctest is not the same as the standard library's doctest module. It's the former that provides skip-if, not the latter.

Wonderful to hear there's interest in adding support for skip-if here. I believe that would make pytest-doctestplus the only pytest plugin to offer support for this, which would make it the only solution for projects like mine that need this (example). (pytest-sphinx started adding support for this years ago, but the implementation has not been completed yet. See also https://github.com/sphinx-doc/sphinx/issues/1487.)

I took a quick look at https://github.com/astropy/pytest-doctestplus/blob/main/pytest_doctestplus/sphinx/doctestplus.py with an eye toward contributing this, but couldn't immediately see how to do it, and am not sure when I could find more time for it (recently started a new job, and have a 6-month-old at home:). Would this be pretty straightforward to add though for a core maintainer or a more experienced outside contributor to this project?

Thanks for your consideration!

[bsipocz]

I have some offline flight hours later this week to fill with a suitably standalone project, will try to have a look at this then.

[pllim]

Would be nice if we can take advantage of sphinx.ext.doctest somehow without reinventing the wheel but I am also not familiar with the code here. Hopefully @bsipocz can come up with something. Thanks! 🤞 🙇‍♀️

[mhvk]

General :+1: to this idea -- will be helpful for people needing to deal with the changes in how units are represented (thanks @pllim for pointing to this!).