pythonmonkey.SpiderMonkeyError: InternalError: too much recursion

[almahdi404]

Issue type

Bug

How did you install PythonMonkey?

Installed from pip

OS platform and distribution

Linux Mint 20.3 x86_64

Python version (python --version)

3.9.17

PythonMonkey version (pip show pythonmonkey)

0.2.1

Bug Description

doing a require inside a Django application results a recursion error

from pythonmonkey import require

def view(request)
    require('./module.js')
    ...

Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/.../project/env/lib/python3.9/site-packages/django/utils/autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "/.../project/env/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 133, in inner_run
    self.check(display_num_errors=True)
  File "/.../project/env/lib/python3.9/site-packages/django/core/management/base.py", line 485, in check
    all_issues = checks.run_checks(
  File "/.../project/env/lib/python3.9/site-packages/django/core/checks/registry.py", line 88, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/.../project/env/lib/python3.9/site-packages/django/core/checks/urls.py", line 14, in check_url_config
    return check_resolver(resolver)
  File "/.../project/env/lib/python3.9/site-packages/django/core/checks/urls.py", line 24, in check_resolver
    return check_method()
  File "/.../project/env/lib/python3.9/site-packages/django/urls/resolvers.py", line 494, in check
    for pattern in self.url_patterns:
  File "/.../project/env/lib/python3.9/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/.../project/env/lib/python3.9/site-packages/django/urls/resolvers.py", line 715, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/.../project/env/lib/python3.9/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/.../project/env/lib/python3.9/site-packages/django/urls/resolvers.py", line 708, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/.../project/example/project/urls.py", line 24, in <module>
    path("", include("core.urls")),
  File "/.../project/env/lib/python3.9/site-packages/django/urls/conf.py", line 38, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/.../project/example/apps/core/urls.py", line 1, in <module>
    from core import views
  File "/.../project/example/apps/core/views.py", line 6, in <module>
    from djust.views import api, render_jsx
  File "/.../project/src/djust/views.py", line 25, in <module>
    response = require(
  File "/.../project/env/lib/python3.9/site-packages/pythonmonkey/require.py", line 353, in require
    return createRequire(filename)(moduleIdentifier)
pythonmonkey.SpiderMonkeyError: InternalError: too much recursion

Standalone code to reproduce the issue

No response

Relevant log output or backtrace

No response

Additional info if applicable

No response

What branch of PythonMonkey were you developing on? (If applicable)

No response

[wesgarland]

This is likely due the 2nd require not finding the exports memo created by the first require in the require in ctx-module. That implies that ctx-module is not getting the correct canonical module id.

Possibly related to how the python automatic require hangs together.

Possible workaround - in python, require=pythonmonkey.createRequire(file)

Thanks for the bug report, we’ll get this fixed.

[wesgarland]

“File” above should be underscore underscore file underscore underscore

[almahdi404]

Your suggested code resulted the same error.

Doing a pythonmonkey.eval also results an InternalError: too much recursion

The reason might be threading

[almahdi404]

Confirmed. The cause of this error is threading

here is the code to reproduce the error

# monkey.py
import threading
from pythonmonkey import require

def func():
    require("./module.js").hello()

t = threading.Thread(target=func)
t.start()

// module.js
function hello() {
  console.log("hello world");
}

module.exports = { hello };

[wesgarland]

@Xmader when you are back, it would be worth having a look at this. If we are entering spidermonkey from two threads at the same time, it will never end in happiness. We shouldn't need to hold the Python GIL to run JS code, but we at least need to have some kind of mutex and formalized JS cx hand-off from one thread to another. Hopefully spidermonkey is no longer using TLS memory, but we should check the docs on that as well. Historical versions of spidermonkey have previously associated OS threads with contexts and used TLS for important details.

[Hansimov]

Use python subprocess as a workaround. Just take care when parsing all kinds of quotes.

An example:

import subprocess

def eval_js_codes(js_codes):
    js_codes_to_eval = js_codes.replace('"', '\\"')
    js_codes_to_eval_in_python = f"from pythonmonkey import eval; eval('{js_codes_to_eval}')"
    pm_cmd = f'python -c "{js_codes_to_eval_in_python}"'
    output = subprocess.getoutput(pm_cmd)
    return output

js_codes = 'console.log("Hello World!")'
eval_js_codes(js_codes)

# python -c "from pythonmonkey import eval; eval('console.log(\"Hello World!")')"

Output:

Hello World!

[almahdi404]

Well i had to switch to PyMiniRacer, because i want to run multiple code snippets in a separate single context.

Unlike PyMiniRacer, PythonMonkey doesn't have a context api. And it also doesn't work with threading.

[philippedistributive]

no threading support in near future

[baseplate-admin]

Hi, @philippedistributive, do you have any plans on having a way for PythonMonkey with a web framework like django?