Testing infrastructure

[agriyakhetarpal]

Also, we should add tests in CI that build docs for a simple site to make sure everything works.

Originally posted by @steppi in https://github.com/jupyterlite/jupyterlite-sphinx/issues/236#issuecomment-2558479327

---

Besides integration testing with a regular site that where the docs build without any uncaught warnings, I would also like to propose:

• visual testing to ensure that the style for the buttons, iframes and their alignment works, such that issues like #232 are reported less often
• a few unit tests for all the Lite directives and the options that can be passed to them.

The test suite could run in the "Tests" job, which at the time just runs black (which could be a pre-commit hook instead).

[steppi]

Walking back my quote from above. In a way, the docs already function as both visual test and integration test. The reason 0.17.0 was shipped broken though was because we had something in the doc requirements that should have been in the regular requirements. I propose adding a very simple test to start. Install the local jupyterlite_sphinx with pip install ., and then check that

from jupyterlite_sphinx.jupyterlite_sphinx import *

doesn't result in an ImportError

Beyond that, maybe the next step should be making it so the docs use all functionality that we'd like to test. One thing that I know is missing now is global_enable_try_examples.

[agriyakhetarpal]

Thanks – yes, I don't intend to implement a big pytest-based test suite, just something that we can keep minimal and also user-facing would be great. One thing we could do besides catching ImportErrors (and ModuleNotFoundErrors) is to turn warnings into errors with Read the Docs, because it already deploys the docs. I can't find a way to enable this within the file, but we can add something like SPHINXOPTS="-W --keep-going -n" to the environment variables to be passed on to.