Problems with sections in the "only" directive.

[gmilde]

Describe the bug

The "only" directive's documentation features the warning

This directive is designed to control only content of document. It could not control sections, labels and so on.

Using section markup in "only" may lead to inexpected results -- without warnings in the output or log.

Example 1 (cf. https://github.com/sphinx-gallery/sphinx-gallery/issues/770):

How to Reproduce

Element mix-up
==============
cf. https://github.com/sphinx-gallery/sphinx-gallery/issues/770

.. only:: html or pseudoxml

   Section in "only"
   =================
   Paragraph in "only"

Paragraph after "only".

Subsection after "only"
-----------------------

Section after "only"
====================

Environment Information

Platform:              linux; (Linux-6.1.0-37-amd64-x86_64-with-glibc2.36)
Python version:        3.11.2 (main, Apr 28 2025, 14:11:48) [GCC 12.2.0])
Python implementation: CPython
Sphinx version:        8.2.3
Docutils version:      0.21.2
Jinja2 version:        3.1.2
Pygments version:      2.19.1

Additional context

If the directive's content contains a section with the same or a higher level as the current section level, the "only" directive attaches elements generated by the nested parsing to the section one level up. However, it does not update the "current node" of the state machine.

[gmilde]

A fix should be possible after Docutils commit [r10223], which adds access to the parent state machine to nested state machines. See the tests and analysis in [r10222].

To go forward, we would need to clarify the intended behaviour of the "only" directive:

1. The current implementation uses a fresh "section title style hierarchy" but checks for new title styles and adds the complete content according to its level in the document-wide title style hierarchy. This is unintuitive: try to guess the section levels in the following rST sample:

Section
=======
subsection
----------

.. only:: html or pseudoxml

   in "only"
   ---------
   still in "only"
   ===============

.. only:: html or pseudoxml

   in "only"
   =========
   still in "only"
   ===============

2. The proposal in #12491 would use/enforce the document-wide title style hierarchy for the content (i.e. a change in behaviour for sub-sections). The behaviour would resemble that of content included by the "include" directive: Resulting elements may be attached to different parent nodes (if the content block contains two sections and the second has a lower level). Non-section elements after the directive are attached to the last included section.

3. This behaviour could also be implemented with a new nested_parse() variant that uses a nested state machine to parse the content block into the current state machine's node (instead of a node provided as argument) and updates the "active" node after parsing. See 0001-Support-nested-parsing-with-document-wide-section-ti.patch (apply on top of Docutils commit [r10225]).

Pros and cons:

Variant 1

+1 allows to re-use the cached doctree also with changed tags or builder. -1 unexpected behaviour (section levels, parsing of "only" content when the condition is False, problems with cross-referencing).

Variants 2 and 3

+1 simpler implementation (no transform required, no <only> element required), +1 consistent handling of section titles (solves issues #2306, #2385, #10642, #12899, #13463, and #13484), +1 content is only parsed when the condition is True (solves issues #1991, #2150, #4242, #4403, #4726, #9482, and #11768). -1 changed tags require a re-creation of the doctree-cache.

[gmilde]

Tests with Docutils HEAD work again after Docutils commit [r10226] rST parser: restore backwards compatibility ....

Confirmed with a minor change to a function docstring in https://github.com/sphinx-doc/sphinx/pull/13857/checks