sphinx.ext.doctest: Show only failing tests (hide summary for passing)

[astromatt]

trafficstars

My documentation 1 has thousands of doctests (lines starting with >>>) in hundreds of modules.

1. Can you please provide an option to display only failing tests? Currently my output is flooded by messages like 2:

Document: type/str
------------------
1 items passed all tests:
  81 tests in default
81 tests in 1 items.
81 passed and 0 failed.
Test passed.

Document: type/str-methods
--------------------------
1 items passed all tests:
 220 tests in default
220 tests in 1 items.
220 passed and 0 failed.
Test passed.

If there is an error, you have to scroll through very long output until you find it.

2. Very often failing test line number (in file) are incorrect. If I indent code block, line numbers are not displayed at all (they appears as question marks ?).

3. Is there a chance for this to work with Markdown (recommonmark)? I found, that if you put eval_rst code fence 3, tests are executed.

[tk0miya]

Unfortunately, doctest extension is not maintained. So there is no chance to add a new option. I hope somebody will try this feature request in the future.

[treyhunner]

I had this same issue and ended up forking the extension to fix it.

I've made a branch with the changes I made (there were very few needed, as mostly verbose needed to be disabled for doctest): https://github.com/treyhunner/sphinx/tree/allow-non-verbose-output

I'm not sure how best to write a test for those changes, as I'm unfamiliar with Sphinx's test suite. If anyone wants to take over those changes (no need to credit me) and write some tests, go for it. 👍 I'd love to stop maintaining my own fork of the doctest extension someday.

[AA-Turner]

@treyhunner are you happy to open a PR to add the feature back to the Sphinx core?

A

[warsaw]

Unfortunately, doctest extension is not maintained

I was pointed to this discussion from elsewhere. Long ago I switch to using Sybil for doctests and have been quite happy with it. Here's an example of my conftest.py file setting up Sybil and integrating it with pytest.

[treyhunner]

@treyhunner are you happy to open a PR to add the feature back to the Sphinx core?

I just opened #10938.

Unfortunately, doctest extension is not maintained

I was pointed to this discussion from elsewhere. Long ago I switch to using Sybil for doctests and have been quite happy with it. Here's an example of my conftest.py file setting up Sybil and integrating it with pytest.

This project looks great! 😮 Thanks for pointing out Sybil @warsaw and for sharing that example conftest.py file. I may end up switching to Sybil testing for some of my Sphinx projects.