Transcrypt Import Error: Failing to import file for classes that I've defined.

Details

OS: Windows 10 Python Version: 3.9.6 Transcrypt version: 3.9.0

Summary

I'm trying to get my pure-Python project with no dependencies to be Transcrypt compatible, but ran into a problem very early with Import Errors for things that I don't understand. Specifically, names of functions or classes I'm importing are being reported as missing module files (examples below).

Steps to Reproduce

git clone https://gitlab.com/iffyb/bezoar.git
transcrypt -v -n bezoar_sample.__main__

Expected Results

It would import the function get_version and classes Entity and Player the way CPython does?

Actual Results

org.transcrypt.utils.Error: Error while compiling (offending file last):
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar_sample/__main__.py', line 5, at import of:
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar_sample/model.py', line 10, at import of:
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar/model/__init__.py', line 29, at import of:
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar/model/version_declaration.py', line 3, at import of:
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar/model/player_declaration.py', line 6, at import of:
        File 'bezoar.model.relationship_declaration', line 0, namely:

        Import error, can't find any of:
                C:/cygwin/home/iffy/ws/bezoar/bezoar/model/Entity.py
                C:/cygwin/home/iffy/ws/bezoar/bezoar/model/Entity.js
                c:/python/python39/lib/site-packages/transcrypt/modules/bezoar/model/Entity.py
                c:/python/python39/lib/site-packages/transcrypt/modules/bezoar/model/Entity.js
                C:/Python/Python39/Scripts/transcrypt.exe/bezoar/model/Entity.py
                C:/Python/Python39/Scripts/transcrypt.exe/bezoar/model/Entity.js
                C:/Users/iffy/AppData/Roaming/Python/Python39/site-packages/bezoar/model/Entity.py
                C:/Users/iffy/AppData/Roaming/Python/Python39/site-packages/bezoar/model/Entity.js
                c:/python/python39/lib/site-packages/bezoar/model/Entity.py
                c:/python/python39/lib/site-packages/bezoar/model/Entity.js

bezoar.model is a package, and Entity is a class defined in the bezoar.model.entity_declaration module, which has a file in C:/cygwin/home/iffy/ws/bezoar/bezoar/model/entity_declaration.py.

relationship_declaration imports bezoar.model.entity_declaration as entity and refers to entity.Entity, among several other symbols in the bezoar.model.entity_declaration module that it doesn't complain about.

Full transcript here: transcrypt_transcript.txt

Other Notes

It spits out many different Import Errors, some of them seemingly the same.

There's another internal except in the log, not sure how relevant it is:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python/python39/lib/site-packages/transcrypt/modules\org\transcrypt\compiler.py", line 2028, in visit_ClassDef
    self.visit (statement.value)
  File "c:\python/python39/lib/site-packages/transcrypt/modules\org\transcrypt\compiler.py", line 985, in visit
    super () .visit (node)
  File "c:\python/python39/lib\ast.py", line 407, in visit
    return visitor(node)
  File "c:\python/python39/lib\ast.py", line 411, in generic_visit
    for field, value in iter_fields(node):
  File "c:\python/python39/lib\ast.py", line 249, in iter_fields
    for field in node._fields:
AttributeError: 'NoneType' object has no attribute '_fields'

The line number reported is "line 0" which seems odd.
There's also a similar-looking import error for from collections.abc import MutableMapping in bezoar.base.overlay_mapping

Aug 04 '21 06:08 iffyb

Thanks @iffyb, unfortunately not all standard library modules are supported by Transcrypt.

Aug 04 '21 11:08 fzzylogic

Thanks, @fzzylogic, that explains MutableMapping, which I can find some way past, but Player, Entity, and get_version are all my functions in my source code?

Aug 04 '21 16:08 iffyb

You're right @iffyb, I noticed also that you're using a globally installed instance, which a few have had problems with in the past. To use a virtual env, the simple approach i often use is to make one right in the project folder (instead of in a central place) - in a normal windows cmd.exe shell:

> py -3.9 -m venv wenv
> wenv\Scripts\activate
> pip install transcrypt

That will get rid of the local import issues. Look forward to knowing how this develops, it sounds like a fun project!

Aug 05 '21 00:08 fzzylogic

Thanks, @fzzylogic!

I'm not sure if I did this right. I'm still barely Python-literate, so many, many apologies.

I uninstalled transcrypt from my global environment, got a venv running, activated it, and installed transcrypt under the venv.

I re-ran transcrypt, and it's still basically giving me these errors. I suspect it's not actually an import path error, but maybe something funny I'm doing with my imports between my modules. There are circular dependencies, which I've managed to get mypy and python happy with, but perhaps transcrypt still sees them as distasteful?

player_declaration.py, relationship_declaration.py, and entity_declaration.py all refer to each other's types. And then I have bezoar/model/__init__.py importing from all of them as a sort of façade.

Here's a log snippet. It is still asking about Entity.py, but now it is indeed searching the virtual environment... But Entity is a class defined in bezoar/model/entity_declaration.py, and only transitively imported through bezoar/model/__init__.py.

org.transcrypt.utils.Error: Error while compiling (offending file last):
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar_sample/__main__.py', line 5, at import of:
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar_sample/model.py', line 10, at import of:
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar/model/__init__.py', line 29, at import of:
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar/model/version_declaration.py', line 3, at import of:
        File 'C:/cygwin/home/iffy/ws/bezoar/bezoar/model/player_declaration.py', line 6, at import of:
        File 'bezoar.model.relationship_declaration', line 0, namely:

        Import error, can't find any of:
                C:/cygwin/home/iffy/ws/bezoar/bezoar/model/Entity.py
                C:/cygwin/home/iffy/ws/bezoar/bezoar/model/Entity.js
                C:/cygwin/home/iffy/ws/bezoar/venv/lib/site-packages/transcrypt/modules/bezoar/model/Entity.py
                C:/cygwin/home/iffy/ws/bezoar/venv/lib/site-packages/transcrypt/modules/bezoar/model/Entity.js
                C:/cygwin/home/iffy/ws/bezoar/venv/Scripts/transcrypt.exe/bezoar/model/Entity.py
                C:/cygwin/home/iffy/ws/bezoar/venv/Scripts/transcrypt.exe/bezoar/model/Entity.js
                C:/cygwin/home/iffy/ws/bezoar/venv/bezoar/model/Entity.py
                C:/cygwin/home/iffy/ws/bezoar/venv/bezoar/model/Entity.js
                C:/cygwin/home/iffy/ws/bezoar/venv/lib/site-packages/bezoar/model/Entity.py
                C:/cygwin/home/iffy/ws/bezoar/venv/lib/site-packages/bezoar/model/Entity.js
                C:/Users/iffy/AppData/Roaming/Python/Python39/site-packages/bezoar/model/Entity.py
                C:/Users/iffy/AppData/Roaming/Python/Python39/site-packages/bezoar/model/Entity.js
                C:/Python/Python39/lib/site-packages/bezoar/model/Entity.py
                C:/Python/Python39/lib/site-packages/bezoar/model/Entity.js

Aug 05 '21 17:08 iffyb

Another thing to note, I worked past the abc.MutableMapping, and then I had a similar import error with itertools.count. I had chalked it up to another thing that wasn't supported by Transcrypt, but I later noticed that there was an itertools.js file in the __target__ output, and it had count in there!

Aug 06 '21 06:08 iffyb

Thanks @iffyb, good points. I haven't look into how Transcrypt handles imports, but for some reason I was getting further into the process on my side, using:

> transcrypt -v -n bezoar_sample.__main__  # seemed promising, as it was only choking on textwrap

So i did a git pull, and it fast-forwarded from my clone of the other night from commit e1abba3 to 74b7a4c, after which i got the same results as on your side. So it must have been failing first on textwrap, preventing me from seeing what would inevitably have failed next.

You're right too about the façade import not working, although i noticed that it does work when not using enum. Also, regarding async / await, it is supported, but since js is event driven by default, asyncio is not needed.

It might be somewhat easier to start from a minimal compiling example and work up from there. I also found it helpful to play around with Transcrypt a bit, for example seeing how to use it to interact with the DOM. Hope this helps.

Aug 06 '21 12:08 fzzylogic

Thanks again, @fzzylogic! Yeah, I didn't discover Transcrypt until I had already made a fair amount of progress. It would have been much easier to build it up incrementally with a Transcrypt step in my continuous integration. I'll have to see what I can do to minimize what I'm trying to work with at first.

Aug 06 '21 22:08 iffyb

I believe I figured out what was causing the issue. These exceptions were causing spurious errors:

Traceback (most recent call last):
  File "c:\python/python39/lib/site-packages/transcrypt/modules\org\transcrypt\compiler.py", line 2028, in visit_ClassDef
    self.visit (statement.value)
  File "c:\python/python39/lib/site-packages/transcrypt/modules\org\transcrypt\compiler.py", line 985, in visit
    super () .visit (node)
  File "c:\python/python39/lib\ast.py", line 407, in visit
    return visitor(node)
  File "c:\python/python39/lib\ast.py", line 411, in generic_visit
    for field, value in iter_fields(node):
  File "c:\python/python39/lib\ast.py", line 249, in iter_fields
    for field in node._fields:
AttributeError: 'NoneType' object has no attribute '_fields'

In compiler.py, at line 2028 (in the 3.9.0 release), in visit_ClassDef, I inserted an:

    if statement.value:

before

    self.visit (statement.value)

And it removed all of the spurious Import Errors, though it did still generate broken code.

The code in question was this, and some similar cases:

class RelationshipHandlerProtocol(Protocol):
    can_relate: Optional[CanRelateHandler]
    can_unrelate: Optional[CanUnrelateHandler]
    on_relate: Optional[RelateHandler]
    on_unrelate: Optional[UnrelateHandler]

Here I'm defining a Protocol, and I'm just declaring the properties of the Protocol. But a declaration without an initialization seems to cause these statement.value being None cases.

A similar case (I was trying to work around not having enum):

class Arity(TypedNumber):
    def __init__(self, value: int) -> None:
        super().__init__(value)

    ONE:'Arity'
    MANY:'Arity'

Arity.ONE = Arity(1)
Arity.MANY = Arity(2)

Both MyPy and CPython seem to be fine with this, but Transcrypt not so much. MyPy in fact seems to require these declarations or it will complain of Arity not having a ONE or MANY property to assign to.

I was able to work around both cases once I knew what the problem was.

Aug 07 '21 07:08 iffyb

Thanks for this valuable feedback @iffyb, glad you're making headway and look forward to seeing a browser based bezoar in future.

Aug 07 '21 14:08 fzzylogic