basilisp icon indicating copy to clipboard operation
basilisp copied to clipboard

Published 20 hours ago • verified publisher icon basilisp-lang

Can't require child namespaces

Open mitch-kyle opened this issue 6 months ago • 3 comments

Can't require child namespace

The code from this report is available in the Reference Project

  • If namespace a depends on a.b then it is not possible run namespace a.
  • If namespace c depends on a.b then a is loaded erroneously.

Demonstration

Setup

this only needs to be done once.

export PYTHONPATH=src
poetry install

First failure case

namespace a requires a.b. This is expected to load a.b then a.

poetry run basilisp run -n a

Instead we get an error:

Loaded a.b
Loaded a
Traceback (most recent call last):
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/compiler/__init__.py", line 194, in compile_and_exec_form
    return getattr(ns.module, final_wrapped_name)()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'a' has no attribute '__lisp_expr___69'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/compiler/__init__.py", line 194, in compile_and_exec_form
    return getattr(ns.module, final_wrapped_name)()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "a", line 1, in __lisp_expr___65
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 4485, in load
    (defn load
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 2459, in dorun
    (defn dorun
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 2468, in dorun__arity1
    (when (seq ptr)
          ^^^^^^^^^
  File "/usr/lib/python3.12/functools.py", line 909, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py", line 274, in _to_seq_lazyseq
    return o.seq()
           ^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py", line 174, in seq
    self._compute_seq()
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py", line 169, in _compute_seq
    self._obj = gen()
                ^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 4506, in ____load__for_3672_3679__lisp_fn_3681
    (doseq [path (seq paths)]
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 4459, in load_file
    (with-open [f (python/open path ** :mode "r")]
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 4460, in load_file
    (load-reader f)))
    ^^^^^^^^^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 4424, in load_reader
    (defn load-reader
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 2459, in dorun
    (defn dorun
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 2468, in dorun__arity1
    (when (seq ptr)
          ^^^^^^^^^
  File "/usr/lib/python3.12/functools.py", line 909, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py", line 274, in _to_seq_lazyseq
    return o.seq()
           ^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py", line 174, in seq
    self._compute_seq()
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py", line 169, in _compute_seq
    self._obj = gen()
                ^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 4444, in ____load_reader__for_3615_3634__lisp_fn_3636
    (basilisp.lang.compiler/compile-and-exec-form form
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/compiler/__init__.py", line 192, in compile_and_exec_form
    exec(bytecode, ns.module.__dict__)  # pylint: disable=exec-used
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mitch/Sources/bug_reports/src/a.lpy", line 1, in <module>
    (ns a (:require [a.b]))
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 4919, in require
    (doseq [libspec (map require-libspec args)]
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 2459, in dorun
    (defn dorun
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy", line 2468, in dorun__arity1
    (when (seq ptr)
          ^^^^^^^^^
  File "/usr/lib/python3.12/functools.py", line 909, in wrapper return
    dispatch(args[0].__class__)(*args, **kw)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File
    "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py",
    line 274, in _to_seq_lazyseq return o.seq() ^^^^^^^ File
    "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py",
    line 174, in seq self._compute_seq() File
    "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/seq.py",
    line 169, in _compute_seq self._obj = gen() ^^^^^ File
    "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy",
    line 4925, in ____require__for_4008_4021__lisp_fn_4023
    (require-lib current-ns libspec) File
    "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy",
    line 4846, in require_lib (eval (list 'require* File
    "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy",
    line 4411, in eval_ (defn eval File
    "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/core.lpy",
    line 4420, in eval___arity2
    (basilisp.lang.compiler/compile-and-exec-form form
    ^^^^^^^^^^^^^^^^^^^^^^^^^^ File
    "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/compiler/__init__.py",
    line 196, in compile_and_exec_form del
    ns.module.__dict__[final_wrapped_name]
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ KeyError:
    '__lisp_expr___69'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/bin/basilisp", line 8, in <module>
    sys.exit(invoke_cli())
             ^^^^^^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/cli.py", line 669, in invoke_cli
    parsed_args.handler(parser, parsed_args)
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/cli.py", line 543, in run
    eval_namespace(target, ctx, ns)
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/cli.py", line 68, in eval_namespace
    return compiler.load(path, ctx, ns)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/compiler/__init__.py", line 302, in load
    return compile_and_exec_form(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/compiler/__init__.py", line 196, in compile_and_exec_form
    del ns.module.__dict__[final_wrapped_name]
        ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
KeyError: '__lisp_expr___65'

Second Failure Case

namespace c requires a.b, it is expected to load a.b and only a.b as namespaces are hierarchically independent of one another.

poetry run basilisp run -n c

Looks like a is being loaded after a.b for some reason.

Loaded a.b
Loaded a
Loaded c

Why is this happening?

It looks like a.b causes a to be loaded. in the second case a loads successfully (though it shouldn't have loaded at all) because a.b is loaded first. When we try to load a.b first it also triggers another run of the code generator for a which overwrites the expected function in the compiler.

  File "/home/mitch/.cache/pypoetry/virtualenvs/bug-reports-adCv5oh1-py3.12/lib/python3.12/site-packages/basilisp/lang/compiler/__init__.py", line 194, in compile_and_exec_form
    return getattr(ns.module, final_wrapped_name)()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'a' has no attribute '__lisp_expr___69'

'__lisp_expr___69' isn't there because it was clobbered by the second load.

It looks like this is being triggered by importlib.import_module in the require logic.

Other Information

Python modules by default do not load their parent module. So this is specifically a problem with basilisp and/or importlib

d imports d.e works as expected.

poetry run python run src/d.py

Output:

loaded: e
loaded: d

f imports d.e without loading d

poetry run python run src/f.py

Output:

loaded: e
loaded: f

mitch-kyle avatar Aug 10 '24 17:08 mitch-kyle