Issue with type resolution

[bryevdv]

Issue

When building the Bokeh docs, things expode:

Traceback (most recent call last):
  File "/Users/bryan/anaconda/envs/dev-311/lib/python3.11/site-packages/sphinx/events.py", line 94, in emit
    results.append(listener.handler(self.app, *args))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bryan/anaconda/envs/dev-311/lib/python3.11/site-packages/sphinx_codeautolink/extension/__init__.py", line 42, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bryan/anaconda/envs/dev-311/lib/python3.11/site-packages/sphinx_codeautolink/extension/__init__.py", line 176, in create_references
    self.filter_and_resolve(transforms, skipped, doc)
  File "/Users/bryan/anaconda/envs/dev-311/lib/python3.11/site-packages/sphinx_codeautolink/extension/__init__.py", line 200, in filter_and_resolve
    key = resolve_location(name, self.inventory)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bryan/anaconda/envs/dev-311/lib/python3.11/site-packages/sphinx_codeautolink/extension/resolve.py", line 36, in resolve_location
    cursor = locate_type(cursor, tuple(comps), inventory)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bryan/anaconda/envs/dev-311/lib/python3.11/site-packages/sphinx_codeautolink/extension/resolve.py", line 92, in locate_type
    for val in previous.value.mro():
               ^^^^^^^^^^^^^^^^^^^^
TypeError: unbound method type.mro() needs an argument

The proximate cause turns out to be that previous.value is builtins.type at this point:

(Pdb) previous
Cursor(location='builtins.type', value=<class 'type'>, instance=False)

Bokeh is a cross-runtime library and uses a fair amount of meta-programming to automate management of that complexity. That might be why this unexpected values ends up here.

Expected behavior

Completed successful build

Steps to reproduce

I am a maintainer for Bokeh and encountered this while building the Bokeh docs. I am afraid I have not been able to isolate a simpler MRE. Dev guide for building Bokeh docs is here:

https://docs.bokeh.org/en/latest/docs/dev_guide/documentation.html#setting-up-and-building-bokeh-s-documentation

environment:

pydata-sphinx-theme       0.9.0                    pypi_0    pypi
sphinx                    5.3.0                    pypi_0    pypi
sphinx-codeautolink       0.12.1                   pypi_0    pypi
sphinx-copybutton         0.5.0                    pypi_0    pypi
sphinx-design             0.3.0                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    2.0.0                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.5                    pypi_0    pypi
sphinxext-opengraph       0.7.2                    pypi_0    pypi

Patch?

Changing the condition in resolve.py to exclude type allows the build to complete "successfully":

        if isclass(previous.value) and previous.value is not type and cursor.location not in inventory:

however then there are hundreds of other warnings, so this does not seem like a complete solution:

.. autolink-examples:: bokeh.plotting.figure.ref :collapse:.
None:4:<autosummary>:1: ERROR: Error in "autolink-examples" directive:
maximum 1 argument(s) allowed, 2 supplied.

.. autolink-examples:: bokeh.plotting.figure.plot :collapse:.
None:4:<autosummary>:1: ERROR: Error in "autolink-examples" directive:
maximum 1 argument(s) allowed, 2 supplied.

.. autolink-examples:: bokeh.plotting.figure.coordinates :collapse:.
/Users/bryan/work/bokeh/src/bokeh/plotting/_figure.py:docstring of bokeh.plotting._figure.figure:1: WARNING: Explicit markup ends without a blank line; unexpected unindent.

Without auto-link extension the doc build is 100% warning clean.

[bryevdv]

Additionally when the "successful" build finishes, some methods are not linked, e.g. the inherited members of figure:

I'm not sure if this is related or not, these methods all have an extensive decorator to derive much of their functionality from a declarative spec. Perhaps that is interfering.

[felix-hilden]

Thanks for submitting the issue and debugging! First, for the easier stuff:

autolink-examples

Please try:

.. autolink-examples:: function
   :collapse:

instead of having it on a single line. We only expect one argument in the directive, and the "collapse" is considered one if on the same line, rather than being an option.

EDIT: I realised that the "hundreds of" warnings might be generated by our autodoc integration rather than you adding example blocks manually. So to silence them for now you can just remove the configuration for autodoc_inject. The docstring injection should be clean but we can investigate it in another issue.

methods with no links

We do support linking inherited methods (test def, test src). It's most likely a matter of either not finding the method in the inventory (i.e. no ref in autodoc) or the function not being accessible at import.

type error

I'll look into this in more detail later 👍

[bryevdv]

Hi @felix-hilden thanks for the reply, a couple of comments:

It's most likely a matter of either not finding the method in the inventory (i.e. no ref in autodoc)

How can I verify this? In the built docs, the (inherited) methods do show up in autoclasstoc

And in particular scatter is in this list further down. The base class (GlyphAPI in this case) is not user-facing, so it is not itself explicitly included in the docs anywhere on its own, could that be the issue?

I will try the autodoc_inject suggestion and report back.

[felix-hilden]

That's good to know, thanks 👍 Just to set some expectations, I'm unfortunately personally quite busy for a few weeks. I hope to get to this before Christmas though. Surely 😅

The inventory is generated by Sphinx, and available in the build folder. For example Bokeh's inventory is here. Sphinx reads it here. It's generated and passed around during the build in codeautolink here. So either that, or our name resolver not being up to par!

[bryevdv]

@felix-hilden thanks for the details, I will try to dig more into the inventory. Also FYI I we are not yet using sphinx-codeautolink, it's something I am investigating on a branch and would very much like to add, but I'd say it's a 2023Q1 thing at the earliest, so there's no huge time pressure on my account! (There's also a feature that would take auto code linking from nice to really, really nice for Bokeh specifically, that I could potentially work on with some guidance, but I'll open separate issue for that later)

[felix-hilden]

Oh, sounds good! 👍

[felix-hilden]

Hi! Sorry for dropping this for such a long time. Are you still interested in adding this in?

I believe __mro__ should be a safer way of getting the class hierarchy, because type seems to have a special implementation of .mro() different from other objects. But we'll have to go deeper into Bokeh to know what exactly was causing it.

[bryevdv]

@felix-hilden I would definitely still be interested in having auto-code linking work from our example code, I am just not sure what it would entail. I don't have a ton of time at present to dig into this, but I can either: help with any setup/building questions or issues if you wanted to take a quick look yourself, or, if you have specific information or data points that would be useful to collect, try to collect those with some guidance on what you need.

[felix-hilden]

Do you still have the branch that you tried? I can try to build the docs with that and see what's up. Otherwise I think it'd be best to start from scratch and try to introduce things gradually and see what we get!

[bryevdv]

I'm sure I don't still have the branch, and we have also since removed autoclasstoc for now as well, since it was exploding our build times. Probably best to start from scratch with the current state.

[felix-hilden]

Okay, I did not succeed in building the documentation. I hit multiple weird bumps, the latest of which was a Flask server starting and blocking in the middle of the documentation build. I think I'd rather guide you through it if that's alright. (I really gotta get off Windows soon too, this was just unnecessarily painful...)

Given a working docs build, if you simply add the library into requirements and as an extension, are some examples linked at least? (maybe you'll have to install from main with git to get the type fix)