pdoc
pdoc copied to clipboard
Recurse through submodules in extension modules
Fixes https://github.com/pdoc3/pdoc/issues/319
The original method of finding submodules by recursing through the
directory structure didn't work with extension modules (aka native "C"
extensions) because there is no directory structure to recurse through.
This commit adds code to recurse through submodules by inspecting
obj.__all__
, i.e., using the same mechanism that is already used for
finding variables, functions, and classes.
This is a minimalistic first sketch of a solution. It works on a native
extension that I tried it on but it introduces some duplication since
there are now two strategies to search for submodules. For a native
module, the first strategy (searching the directory structure) will fail
but the second (new) strategy (using obj.__all__
) will work. For
regular modules, both strategies will likely work and so the second
strategy will just overwrite the results from the first strategy with
identical objects. This should technically work but it seems
unnecessary. I just don't feel confident enough to remove the first
strategy without further guidance.
Thanks for the quick feedback! I've moved the new code into the if self.is_package - else
block and added a test.
Studying the existing code for "normal" submodule discovery, I realized I should be more careful to avoid capturing cycles and reexports (not sure if they're possible in native extension modules though). So I made sure the new code only runs if the parent module explicitly exposes public members through the __all__
attribute. This restriction seems to be necessary because the is_from_this_module
function doesn't work on submodules (inspect.getmodule(x)
returns x
rather than x
's parent if x
is a submodule). This restriction doesn't seem to be a problem for the native extension modules I tried, but I'd be happy to relax it if you have an idea how.
Further, if I:
import .foo as bar
module
foo
will now be be documented asfoo
and asbar
... :thinking:
Is that possible with native extension modules? I know too little about Python's module system to tell.
One problem I encountered: the HTML documentation for a native extension module now places all files in the top level directory. The contents of the files shows the correct nested hierarchy of submodules but the paths for the corresponding HTML files doesn't reflect this, so paths may clash if, e.g., a submodule has the same name as its parent module. Any idea how to fix this?