`sphinx.ext.autosummary` - document private objects?

[bimac]

I've been trying to build an API documentation with the recursive variant of sphinx.ext.autosummary that needs to include a few private modules of a package. Unless I missed something, it seems like I can only address public objects (functions, classes, exceptions, methods, attributes, and modules) using Jinja templates.

Is there any way of including private objects? Perhaps, sphinx.ext.autosummary could include all objects for which autodoc-skip-member() returns False, or offer a simliar mechanism?

Thank you!

[electric-coder]

Is there any way of including private objects?

It seems the solution is using a custom template that includes private members, see python-sphinx autosummary for private members.

[bimac]

Thanks for your reply @electric-coder. I found that thread as well, but it doesn't really give a solution. I tried with a custom template, but the available variables (see above) explicitly address public objects. I also tried with the members variable - while the documentation states that it would return a

List containing names of all members of the module or class

it only gives me the dunders ['__builtins__', '__cached__', '__doc__', '__file__', '__name__', '__package__', '__path__']. I'm not familiar with Jinja, perhaps there's a way - but it certainly doesn't seem straightforward.

[bimac]

I managed to include the private modules manually, but that defeats the purpose of autosummary.

For reference, I modified the modules block in a custom-module-template.rst as such:

{%- block modules %}
{%- if modules or name == 'my_module_containing_only_private_submodules' %}
.. rubric:: Modules

.. autosummary::
   :nosignatures:
   :toctree:
   :template: custom-module-template.rst
   :recursive:
{% for item in modules %}
   {{ item }}
{%- endfor %}
{%- if name == 'my_module_containing_only_private_submodules' %}
   _private_submodule_a
   _private_submodule_b
{% endif %}
{% endif %}
{%- endblock %}

(partially based on this StackOverflow entry / 7912)

[electric-coder]

@bimac if you have a Stack Overflow account I'd advise also posting the last answer with the template there to help future users find it. You can write your own question and autoanswer it, or post under this Q since it's the easiest to find.

autosummary is limited and not very flexible. Generally I advise using autodoc+napoleon and generating autodoc directives that include all members because if you've gotten to the point of writing a custom template you're likely to get better results with the same amount of work by just using autodoc.

The note at the top of customizing templates already says this:

If you find yourself spending much time tailoring the stub templates, this may indicate that it’s a better idea to write custom narrative documentation instead.

[bimac]

Thanks for your advice @electric-coder - I added my approach as an answer to the Stack Overflow question you mentioned.

As to your point of using autodoc + napoleon: I do. Its just that the structure of the project in question is quite convoluted and changes regularly. It requires some breaking up into separate pages for modules / submodules / etc. - which is error-prone when done manually. The :recursive: option of sphinx.ext.autosummary seems like a perfect fit for automatic indexing / structuring while still allowing me to adapt the autodoc and napoleon configuration to my liking. Other solutions like sphinx-autoapi seemed less flexible to me. Is there a way to auto-generate a structured (multi-page) API with autodoc alone?

[electric-coder]

the structure of the project (...) changes regularly

I think automodule only works for modules, it doesn't recurse into packages (I'm not sure if this rule changed in the meanwhile). Hence you'll need one automodule per package and then you'll have to declare each module in the rst under its package's declaration, but everything else should be documented automatically if you include :members: and :undoc-members: in the autodoc_default_options. This seems like the most economical choice other than using autosummary.

However much the project is convoluted/changing keeping one declaration per module still seems reasonable (using autosummary is equivalent to not having to maintain any rst at all).

[bimac]

Thanks a lot! I played around with the pure autodoc approach a bit - it would probably be feasible if it were able to recurse into packages (which indeed it isn't). But with the amount of modules I'm facing this is not really an option. Also, by its nature this creates a single, long page ...

Maybe I should also look into structuring my project a bit better ;-)