Add `remove_old` configuration to `sphinx.ext.apidoc`

[echoix]

Makes the option available to be configured when using apidoc as an extension instead of as a command-line tool.

Purpose

The command-line version of apidoc allows to specify the --remove-old option. When passed, the value is true. With the extension version of apidoc, sphinx.ext.apidoc, remove_old defaults to True, but isn't configurable.

https://github.com/sphinx-doc/sphinx/blob/076774224a978e5b7a9e679f30fe84816e50e8f9/sphinx/ext/apidoc/_shared.py#L53-L57

This means that when refactoring an old project to use the apidoc extension (in conf.py) instead of a customized Makefile with multiple handwritten calls to sphinx-apidoc, whilst wanting to keep the same generated output structure is impossible. The extension deletes all other files in the target directory (specified with 'destination''s value of apidoc_modules configuration option). It deletes all other handwritten files, and even deletes the index.rst, which causes other failures in the build process.

I understand that most people would want to generate these files in a subfolder of their source directory, but for legacy reasons, some contents and hand-written overrides are all in the same folder, and the pages name should be kept the same for our use-case.

Having the remove_old configuration value ported to the extension's config, even if not used by our project's makefile with the cli-based version, is the less invasive way for sphinx to allow our use case, and prevent the unexpected deletion of files when building (at least we have git to restore committed files). The unexpected part of the behaviour difference when migrating is caused by the default value that is different. It is False for the cli (opt-in), but True (non-configurable) when used as an extension.

With this PR, the config value added is then properly used at the very end of _run_apidoc_module(), that guards calling _remove_old_files() or not (line 89): https://github.com/sphinx-doc/sphinx/blob/076774224a978e5b7a9e679f30fe84816e50e8f9/sphinx/ext/apidoc/_extension.py#L63-L91

Since I'm new to this code base (and never wrote an extension), I would request to take a special look at the very-end of main() in sphinx/ext/apidoc/_cli.py, to make sure the default value is still the same after this change. I'm still not sure, even after having applied the changes locally. https://github.com/sphinx-doc/sphinx/blob/076774224a978e5b7a9e679f30fe84816e50e8f9/sphinx/ext/apidoc/_cli.py#L255-L279 However, I ran tox on my repo of this fork once I applied the same changes for this PR, and the existing tests passed. There is one test that is found when searching for "remove_old" or "remove-old", but it didn't catch the behaviour change between the cli and extension usage, so... https://github.com/sphinx-doc/sphinx/blob/076774224a978e5b7a9e679f30fe84816e50e8f9/tests/test_extensions/test_ext_apidoc.py#L755-L776

References

• None found

Note: ~~I will need to update the CHANGES.rst entry with the pull-request number once the PR is filed, and the PR number is assigned.~~ Changed