Cannot resolve forward reference in type annotations of "p1.my_class.Subclass": name 'MyEnum' is not defined

[nbro10]

Describe the bug

I have 2 packages: p1 and p2.

In p1 I define an enum MyEnum and a dataclass MyClass, which has 2 fields, one of which is a value of MyEnum, so it's annotated like this

# p1.my_class.py

class MyEnum(Enum):
    A = "a"
    B = "b"

@dataclass
class MyClass:
    my_attr: MyEnum = MyEnum.A

Now, in the other package p2, whose documentation is able to reference p1 in general (and its conf.py file contains the intersphinx mapping "p1": ("p1_url", None)), I created a subclass of MyClass

# p2.my_subclass.py

from p1 import MyClass

@dataclass
class MySubclass(MyClass):
    other_attr = 2
    # other functions that don't directly reference MyEnum

In MySubclass, I don't reference MyEnum directly and in its module I don't import MyEnum. However, I import MyClass from p1.

Now, this situation leads to the error described in the title.

The only way to solve it seems to import MyEnum in p2.my_subclass.py, but this would be a dummy import that would only solve this sphinx issue, and I really would like to avoid this.

So, in conclusion, it seems that sphinx does not know that an object imported in another package exists, unless we also import it in our package, even though we have the intersphinx mapping and we're importing code from the other package that references that non-imported code.

This error also seems to disappear if I remove the @dataclass decorator from MySubclass, so this is most likely an bug with how sphinx interacts with dataclasses.

How to Reproduce

Please, see above

Environment Information

sphinx                        6.2.1      Python documentation generator
sphinx-autodoc-typehints      1.23.0     Type hints (PEP 484) support for the Sphinx autodoc extension
sphinx-rtd-theme              1.2.2      Read the Docs theme for Sphinx
sphinxcontrib-applehelp       1.0.4      sphinxcontrib-applehelp is a Sphinx extension which outputs Apple help books
sphinxcontrib-devhelp         1.0.2      sphinxcontrib-devhelp is a sphinx extension which outputs Devhelp document.
sphinxcontrib-htmlhelp        2.0.1      sphinxcontrib-htmlhelp is a sphinx extension which renders HTML help files
sphinxcontrib-jquery          4.1        Extension to include jQuery on newer Sphinx releases
sphinxcontrib-jsmath          1.0.1      A sphinx extension which renders display math in HTML via JavaScript
sphinxcontrib-qthelp          1.0.3      sphinxcontrib-qthelp is a sphinx extension which outputs QtHelp document.
sphinxcontrib-serializinghtml 1.1.5      sphinxcontrib-serializinghtml is a sphinx extension which outputs "serialized" HTML files (json and pickle).

Sphinx extensions

extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.napoleon",
    "sphinx.ext.autosectionlabel",
    "sphinx_autodoc_typehints",
    "sphinx.ext.intersphinx",
]

Additional context

No response

[picnixz]

1. What is the version of your Python interpreter ?
2. What happens if you remove sphinx_autodoc_typehints (I don't know whether this changes anything but since it's a dataclass and you are annotating dataclasses, maybe it's related) ?
3. What is the error that you are getting: is it a warning ? is it a real error ? is a Python error ?
4. Can we get the full terminal output (with Sphinx options -n and -E) ?

[nbro10]

@picnixz

1. Python 3.8.13
2. I get the same error/warning, plus 2 more similar warnings, i.e. it cannot reference other objects in p1
3. It's a warning but my CI/CD will fail with warnings: WARNING: Cannot resolve forward reference in type annotations of "p2.my_subclass.MySubclass": name 'MyEnum' is not defined
4. See below (I've only replaced the real paths to my classes and packages with some fake ones) - as you can see I am using poetry, but that should not be the problem

cd docs && poetry run make clean && poetry run make html
Removing everything under 'build'...
Running Sphinx v6.2.1
making output directory... done
loading intersphinx inventory from https://docs.python.org/3.8/objects.inv...
loading intersphinx inventory from https://www.crummy.com/software/BeautifulSoup/bs4/doc/objects.inv...
loading intersphinx inventory from https://url_to/p1/docs/0.3.0/objects.inv...
building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 5 source files that are out of date
updating environment: [new config] 5 added, 0 changed, 0 removed
reading sources... [100%] index                                                                                                                                                                                                        
WARNING: Cannot resolve forward reference in type annotations of "p2.my_subclass.MySubclass": name 'MyEnum' is not defined
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index                                                                                                                                                                                                         
generating indices... genindex py-modindex done
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build finished with problems, 1 warning.
make[1]: *** [html] Error 1
make: *** [docs] Error 2

[picnixz]

Ah, I see what happens. Actually, autodoc tries its best in general but there are situations where it is really hard to resolve annotations.

Unfortunately, with dataclasses in addition, it may even be more tricky because dataclasses are like namedtuples, namely they are dynamically created and are not real classes. I don't know if you have a from __future__ import annotations thing at the top of your file also, because the latter may also be a reason why it's really difficult to recover the annotations properly.

AFAIK, there are a lot of issues where autodoc fails to find what was imported, from where it was imported. I cannot give you a good solution, unless not to use autodoc for such cases.

I suspect that the warning "Cannot resolve forward reference" occurs when enabling sphinx-autodoc-typehints. Here, for Sphinx itself, I cannot come up with an easy patch (IMHO, the whole autodoc logic should be reworked from scratch because it's been years that bugs have been accumulating). Instead, you could also report to sphinx-autodoc-typehints.

[nbro10]

@picnixz Ok, thanks for this info. Maybe I should also open this issue there too. Do you mean the future import in p2? I'll add it and check if that solves the issue. But shouldn't the annotations just be used when you want to reference e.g. the own class where the method is defined?

[picnixz]

Using from __future__ import annotations ensures that annotations are forward references, meaning that they are not evaluated at runtime, allowing you to only import what needs to be imported in a TYPE_CHECKING block.

Since there are forward references that could not be resolved according to the error, I thought you had something like that.

[benbovy]

FWIW, I had the same warning issued from a file with from __future__ import annotations. Upgrading Python from 3.10 to 3.11 in my environment (with same versions of sphinx and sphinx-autodoc-typehints) made the warning disappear. Not sure exactly why...

EDIT: this might also be because I installed a missing dependency imported in the TYPE_CHECKING bock in that file.