django_uwsgi and cli execution

[dmchmk]

Hi, guys! First of all, thanks a lot for your great project! It really helps me heavily:)

And while using it, I've faced an interesting issue. Don't really know, whether it a bug or a feature, just wanted to draw your attention to it and discuss it a little bit.

When I'm trying to use @spool decorator somewhere on the way of global imports waterfall (@spool -> utils.py -> views.py -> urls.py), everything works fine until I'm trying to use manage.py command from cli. Than I'm getting:

$ ./manage.py migrate
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/management/base.py", line 327, in execute
    self.check()
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/management/base.py", line 359, in check
    include_deployment_checks=include_deployment_checks,
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/management/commands/migrate.py", line 62, in _run_checks
    issues.extend(super(Command, self)._run_checks(**kwargs))
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/management/base.py", line 346, in _run_checks
    return checks.run_checks(**kwargs)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/checks/registry.py", line 81, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/checks/urls.py", line 16, in check_url_config
    return check_resolver(resolver)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/core/checks/urls.py", line 26, in check_resolver
    return check_method()
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/urls/resolvers.py", line 254, in check
    for pattern in self.url_patterns:
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/urls/resolvers.py", line 405, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/var/venv/project_name/lib64/python3.6/site-packages/django/urls/resolvers.py", line 398, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/www/project_name/project_main_app_name/urls.py", line 5, in <module>
    import project_main_app_name.views as main_views
  File "/var/www/project_name/project_main_app_name/views.py", line 15, in <module>
    from project_main_app_name import utils
  File "/var/www/project_name/project_main_app_name/utils.py", line 8, in <module>
    from django_uwsgi.decorators import spool
  File "/var/venv/project_name/lib64/python3.6/site-packages/django_uwsgi/decorators.py", line 17, in <module>
    raise ImportError("uWSGI is not available")
ImportError: uWSGI is not available

which, really looks quite logical after some thinking. No uWSGI in scope - no honey.

But should it be like this? The first workaround I thought of, was something like this:

try:
    from django_uwsgi.decorators import spool
except:
    # workaround for cli calls of manage.py
    from functools import wraps

    def spool(f):
        @wraps(f)
        def wrapped(*args, **kwargs):
            return None
        return wrapped

and for now it works like a charm, buuuut... I understand, that it may bring possible controversial situations to the future usage. So, after meditating on the issue a little bit more, I found out one more way out:

To move all the needed background spooler-driven tasks to the separate module and import it not globally at the beginning of the views.py, but inside exact views and functions, so it shouldn't be affected by cli calls of manage.py

So, here's the question - how do you think, does django_uwsgi need any tricky workaround like dummy decorators for such cases or not?

Thanks a lot!

[vstoykov]

Probably a stub uwsgi.py file can be created which can have only needed functionality to not throw import errors but at the end it will do nothing. If the application is running inside uWSGI this file will not be imported because the uWSGI binnary will load it's internal module. If you are executing some of the manage.py commands (or are running the application with some other WSGI server) it will load uwsgi.py file.

I created a simple stub to open /admin/uwsgi/ when runnig with manage.py runserver without exception raised.

from time import time as __time

started_on = __time()
opt = {}
loop = None
buffer_size = None
numproc = None
cores = None
cache_exists = False
has_threads = True


def workers():
    return ()


def masterpid():
    return


def total_requests():
    return


def logsize():
    return


def signal(sig):
    return


def log(message):
    print(message)

Probably you can put similar thing in your project which has the needed functionality to execute your management commands without problems.

Probably such kind of file can be distributed separately which will mimic most of the uWSGI Python API and people who need it can use it.

In future if it is stable enough it can be put as dependency of django-uwsgi and/or uwsgidecorators packages. There should be a way to distinguish if currently the code is running under uwsgi or not but without failing with import error.