Portray and Django

[zak10]

Hey - cool project! I'm trying to run portray in some of my django projects but am running into issues. They appear to be django specific, so apologies if you're not familiar.

Minimal Steps to Reproduce

pip install django
pip install portray
django-admin startproject testproj
cd testproj
touch pyproject.toml
portray in_browser

Trace

WARNING: Unable to identify `repo_name` and `repo_url` automatically
/Users/zach/sayapi/lib/python3.6/site-packages/pdoc/__init__.py:223: UserWarning: Couldn't get/parse source of '<Module 'testproject'>'
  warn("Couldn't get/parse source of '{!r}'".format(doc_obj))
Traceback (most recent call last):
  File "/Users/zach/sayapi/lib/python3.6/site-packages/pdoc/__init__.py", line 183, in import_module
    module = importlib.import_module(module_path)
  File "/Users/zach/sayapi/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/zach/go/src/github.com/zak10/testproject/testproject/urls.py", line 20, in <module>
    path('admin/', admin.site.urls),
  File "/Users/zach/sayapi/lib/python3.6/site-packages/django/utils/functional.py", line 256, in inner
    self._setup()
  File "/Users/zach/sayapi/lib/python3.6/site-packages/django/contrib/admin/sites.py", line 529, in _setup
    AdminSiteClass = import_string(apps.get_app_config('admin').default_site)
  File "/Users/zach/sayapi/lib/python3.6/site-packages/django/apps/registry.py", line 153, in get_app_config
    self.check_apps_ready()
  File "/Users/zach/sayapi/lib/python3.6/site-packages/django/apps/registry.py", line 134, in check_apps_ready
    settings.INSTALLED_APPS
  File "/Users/zach/sayapi/lib/python3.6/site-packages/django/conf/__init__.py", line 79, in __getattr__
    self._setup(name)
  File "/Users/zach/sayapi/lib/python3.6/site-packages/django/conf/__init__.py", line 64, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/zach/sayapi/bin/portray", line 10, in <module>
    sys.exit(__hug__.cli())
  File "/Users/zach/sayapi/lib/python3.6/site-packages/hug/api.py", line 441, in __call__
    result = self.commands.get(command)()
  File "/Users/zach/sayapi/lib/python3.6/site-packages/hug/interface.py", line 633, in __call__
    raise exception
  File "/Users/zach/sayapi/lib/python3.6/site-packages/hug/interface.py", line 629, in __call__
    result = self.output(self.interface(**pass_to_function), context)
  File "/Users/zach/sayapi/lib/python3.6/site-packages/hug/interface.py", line 123, in __call__
    return __hug_internal_self._function(*args, **kwargs)
  File "/Users/zach/sayapi/lib/python3.6/site-packages/portray/api.py", line 53, in in_browser
    server(directory=directory, config_file=config_file, open_browser=True)
  File "/Users/zach/sayapi/lib/python3.6/site-packages/portray/api.py", line 76, in server
    with render.documentation_in_temp_folder(project_config) as doc_folder:
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/Users/zach/sayapi/lib/python3.6/site-packages/portray/render.py", line 84, in documentation_in_temp_folder
    pdoc3(config["pdoc3"])
  File "/Users/zach/sayapi/lib/python3.6/site-packages/portray/render.py", line 52, in pdoc3
    pdoc.cli.main(Namespace(**config))
  File "/Users/zach/sayapi/lib/python3.6/site-packages/pdoc/cli.py", line 442, in main
    for module in args.modules]
  File "/Users/zach/sayapi/lib/python3.6/site-packages/pdoc/cli.py", line 442, in <listcomp>
    for module in args.modules]
  File "/Users/zach/sayapi/lib/python3.6/site-packages/pdoc/__init__.py", line 603, in __init__
    self.doc[root] = m = Module(import_module(fullname),
  File "/Users/zach/sayapi/lib/python3.6/site-packages/pdoc/__init__.py", line 185, in import_module
    raise ImportError('Error importing {!r}: {}'.format(module, e))
ImportError: Error importing 'testproject.urls': Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Appears to be an issue with attempting to load some modules of the project which IIRC is a no-no in django before the settings.py module has been loaded and configured.

Note sure if anyone has an elegant solution here or if there's a hook I can provide in pyproject.toml perhaps

[timothycrosley]

Hi @zak10,

I have used django in the past, and I think it's a big enough of a cornerstone of the Python community that supporting it is important! I'll be thinking through the best way to do so, in the meantime if anyone else has ideas feel free to comment here as well.

Thanks for alerting me to this!

~Timothy

[nodarai]

Hey @zak10 , the bug doesn't seem to reproduce for me with the steps above.

• Python 3.6.8
• Django==2.2.4
• portray==1.1.0

[zak10]

@nodarai good to know. I have my project structured with a src directory that contains all of my django apps. I wonder if that has something to do with it, perhaps if the directories are traversed in a certain order?

[giovanism]

I tried to reproduce the bug today. This bug is related to django.setup() call that's required for standalone django usage. This can be accidentally triggered by loading asgi.py or wsgi.py. On Django version below 3.0.0 however, asgi.py isn't available and the ROOT_URLCONF which is usually urls.py comes before wsgi.py alphabetically.

I think the only correct method is to provide some kind of hook to execute django.setup(). Relying on asgi.py or wsgi.py side effect also makes it harder to load specific settings as they load settings meant to be run under their own environments.

[giovanism]

@timothycrosley what do you think about this? Or should this go to https://github.com/timothycrosley/pdocs ?

[Northo]

Are there any updates on this issue?