Index of documented modules

[ryangalamb]

Expected Behavior

A way to include the --http mode's "Python module list" index.html as the root index.html of the --html output.

Actual Behavior

By default, there is no index.html at the top level of the HTML output. There doesn't seem to be a way to enable the "Python module list" in the static html output.

Steps to Reproduce

1. pdoc --html [some modules]
2. nothing at ./html/index.html

Additional info

I'm happy to open a PR for this if it seems like a reasonable request. I'm imagining it as a command line flag to run in addition to --html. When present, pdoc would generate an index.html with the "Python module list" the same as the one generated in WebDoc.do_GET() when path is "/"

• pdoc version: pdoc 0.6.3

[kernc]

I think it's safe to just generate the index.html always, why not? Didn't think of that in https://github.com/pdoc3/pdoc/issues/55. :+1:

Besides the index file, why not list all top-level, disjunct packages/modules specified on a single command line also in the sidebar below the members list? Pass the modules as modules= and close https://github.com/pdoc3/pdoc/issues/95. :thinking:

[ryangalamb]

@kernc

I think it's safe to just generate the index.html always, why not?

I'd be hesitant to turn this feature on by default. For my own use case, my docs automation would break if we made this the default. I'm sure others would experience similar problems.

Also, if we make this feature opt-in, we can do nice things like enabling the module breadcrumbs that show up when users are running in --http mode:

That feature wouldn't make sense for users who aren't going to be using the root index.html though.

I'll open a WIP PR to show you what I've got so far and what I'm thinking.

[kernc]

I'm hesitant adding command line flags for little used features.

What do you need the index for and how often? Couldn't a simple shell script, iterating over the same packages specified on the command line, do the job?

packages="package1 package2 package3"
for pkg in $packages; do echo "<a href='$pkg'>$pkg</a><br>" >> index.html; done

What if the index were only created if the user specifies more than one package?

[kernc]

I think, rather in factly, that a proper index is indeed in order, spanning hierarchically all the way to leaf modules.

Created always.

What users specify as --output-dir is pdoc's to work with, and it shouldn't overwrite without force. I'm happy to break some integration for such nice TOC with the next major release, why not? :smiley:

[ryangalamb]

I'm happy to break some integration for such nice TOC with the next major release, why not?

Sounds good!

proper index is indeed in order, spanning hierarchically all the way to leaf modules.

Something sort of like this?

[kernc]

Exactly! Maybe without leading dots and more uniformly indented. Maybe with children also not necessarily exactly in the right grid column but somewhere in the middle.

[ryangalamb]

without leading dots

:+1:

more uniformly indented

That's straightforward enough if we style the tree as a table instead of using flexbox.

With those points, it looks more like this:

children also not necessarily exactly in the right grid column but somewhere in the middle

I'm having trouble picturing how this would look while keeping the uniform indentation. Can you show me an example of something similar to what you're expecting here?

If conserving space is your main concern, instead of showing the docstring glimpse for each submodule like I have above, we could show the top level modules like normal, and then under the description for each, we could show a more compact submodule tree without the docstrings rendered.

[kernc]

I'm having trouble picturing how this would look while keeping the uniform indentation.

No, no, I meant uniform per nesting level, as in your last image already. :+1:

Just the margin is maybe a bit large here. Do subsequent <dt> need to be inside parent <dd>? Maybe we can flatten it out, apply margin per level in CSS?

[ryangalamb]

Do subsequent <dt> need to be inside parent <dd>?

Yes. As I understad, it should be a whole new <dl> inside the <dd>, since it's a different description list, and it's part of that specific description's details.

Let me give out a few options so we have something more concrete to talk about:

A) This is if I reduce the margins a bit, but keep the styling like I had in my last example

B) This is if I only indent the module docstring away from the module name

C) This is with the default dl/dt/dd display

D) This is it as nested ul/lis

Are any of these closer to what you're picturing?

[kernc]

Please excuse my slowness to respond. I get too little productive screen time these days.

I like A, C, and D. I worry D only looks good in this example, with single-line descriptions. Surprisingly, I quite like C too. A or C, your call. :+1:

[kernc]

@rjmill Have you been successfully dissuaded from contributing even what you already have? :sweat_smile:

[pwoolvett]

so... from https://github.com/pdoc3/pdoc/pull/104

I could pick it up during the weekend... though I'm not clear as to which changes should be implemented from there

[kernc]

IIRC, this basically, without a new CLI switch. Of the proposed layouts, I like C and it's in line with dl/dt/dd use elsewhere.

[olivierdalang]

Hi !

Longing for this feature too, it's not clear why https://github.com/pdoc3/pdoc/pull/104 was closed ? From the discussion, it seems it was ready to merge ?

I very quickly tried to fix merge conflicts (see https://github.com/olivierdalang/pdoc/tree/rjmill/implement-static-html-index). It seems to run without errors and to generate an index as expected, albeit only showing the top levels modules (not the submodules as in the examples above).

@pwoolvett If you're still willing to pick that up, you could probably start from there ?

[kernc]

it's not clear why #104 was closed ?

The PR has been discussed in the issue here, then closed for inactivity, I guess.

It would maybe make sense to incorporate @jkbecker's extra file idea from https://github.com/pdoc3/pdoc/issues/301#issuecomment-760256611.

[Kaiser1989]

I opened a ticket, but then found this here (exactly my problem) (#305)

I'm fine with the current behaviour, except there is a missing link for multi project documentations. I would expect a base index.html nearly identical to module page:

• Left side navigation with Index (embedded .md file) and list of packages (instead of sub-modules)
• center page without heading (currently Package my-package) but embedded .md file and below a link list of packages
• All subpages need a backlink to package (instead of super-module)

That's all. I do not care, where this can be configured, maybe as config parameter like we use --config show_source_code=False: --config generate_base_index=True --config base_index_md=introduction.md

This can even make sense for single package documentations, where one wants to add some introduction page before entering the package itself.

[Ricocotam]

Hi, any news on this ? I like @Kaiser1989 proposition. It's consistent with other doc pages + we could include the Readme How hard is it to implement ? I'm super new to pdoc but I'm okay if it takes like 2 hours to dive into the code + do the PR. We can improve later but it's a must have features imho

[kernc]

Maybe something like https://github.com/pdoc3/pdoc/pull/104 (or a new template) could be shaped into use. I agree with @Kaiser1989's points above. I guess let's see a PR.

Note, the current workaround for readme inclusion (for single packages) is via .. include:: (example, example).

[He-Wolf]

Any update on this?

[slayoo]

ping :) just came across this very issue and the proposed solutions above would be a great fix

[slayoo]

FWIW, this is a workaround that we have ended up using in CI:

pdoc_index_workaround.py:

"""
workaround for https://github.com/pdoc3/pdoc/issues/101#issuecomment-526957342
extracted from pdoc's cli.py
"""

import inspect
from pdoc import import_module, _render_template


modules = [
    import_module(module, reload=True)
    for module in ('PKG_A', 'PKG_B')
]

with open('html/index.html', 'w', encoding='utf-8') as index:
    index.write(
        _render_template(
            '/html.mako',
            modules=sorted((module.__name__, inspect.getdoc(module)) for module in modules)
        )
    )

and in the workflow file:

      - run: |
          pip3 install pdoc3
          pip install -e .
          pip install -e PKG_A PKG_B
          python -We -m pdoc --html PKG_A PKG_B
          python -We pdoc_index_workaround.py
      - if: ${{ github.ref == 'refs/heads/main'}}
        uses: JamesIves/[email protected]
        with:
          BRANCH: pdoc
          FOLDER: html
          CLEAN: true

[adarshbattu109]

Stitched the below function to get things going. Reference credit to @slayoo

"""Program to generate `Docs` using `pdoc3` and crate a central `index.html`"""

import inspect
import os
from pathlib import Path

from pdoc import _render_template, import_module


def generate_docs_and_central_index(modules_to_process: list[str]) -> None:
    """Method to generate `docs` folder with central `index.html`

    Args:
        modules_to_process (list[str]): List of modules to process
    """

    modules = [import_module(module, reload=True) for module in modules_to_process]

    # Generate the docs for each module under docs folder
    command = f'pdoc --html --skip-errors --force --output-dir docs {" ".join(modules_to_process)}'
    os.system(command=command)

    # Create a single base `index.html`
    with open(Path("docs", "index.html"), "w", encoding="utf-8") as index:
        index.write(_render_template("/html.mako", modules=sorted((module.__name__, inspect.getdoc(module)) for module in modules)))


if __name__ == "__main__":
    # Update this as per your source
    module_list = ["abc", "def", "ghi", "klm"]
    generate_docs_and_central_index(module_list)