pyscript Execution stops without error

Checklist

[X] #1369
[X] I searched for other issues and couldn't find a solution or duplication
[X] I already searched in Google and didn't find any good information or help

What happened?

Execution hangs without error when accessing a function in the pydicom library that has not been imported. I would expect to see a NameError as we see in normal Python.

Python 3.11.2 (main, Feb 20 2023, 09:41:11) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydicom
>>> from pydicom.data import get_testdata_file
>>> path = get_testdata_file("CT_small.dcm")
>>> ds = pydicom.dcmread(path)   
>>> print(f"Pixel dtype: {pixel_dtype(ds)}")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'pixel_dtype' is not defined
>>> from pydicom.pixel_data_handlers.util import pixel_dtype
>>> print(f"Pixel dtype: {pixel_dtype(ds)}")
Pixel dtype: int16

This code reproduces the issue:

<html>
  <head>
    <link rel="stylesheet" href="https://pyscript.net/latest/pyscript.css" />
    <script defer src="https://pyscript.net/latest/pyscript.js"></script>
  </head>
  <body>
    <py-config>
      packages = [
        "pydicom",
        "numpy",
      ]
    </py-config>
    <py-script>
        import js; js.document.getElementById('python-status').innerHTML = 'Python is now ready. You may proceed.'
    </py-script>  
    <div id="python-status">
      Python is currently starting. Please wait...
    </div>
    <py-script>
      import micropip
      import asyncio
      import pydicom
      import numpy as np
      from pydicom.data import get_testdata_file
      #from pydicom.pixel_data_handlers.util import pixel_dtype
      
      async def main():
          print("Reading pydicom package test file")
          path = get_testdata_file("CT_small.dcm")
          print(f"Loading {path}")
          ds = pydicom.dcmread(path)   
          print("Loaded ds")       
          
          print(f"{ds.pixel_array.shape}")
          print(f"TransferSyntaxUID: {ds.file_meta.TransferSyntaxUID}")
          print(f"BitsAllocated: {ds.get('BitsAllocated', 'NONE')}")
          print(f"SamplesPerPixel: {ds.get('SamplesPerPixel', 'NONE')}")
          print(f"Pixel dtype: {pixel_dtype(ds)}")

      asyncio.ensure_future(main())
    </py-script>
  </body>
</html>

Produces the following output

Reading pydicom package test file
Loading /lib/python3.10/site-packages/pydicom/data/test_files/CT_small.dcm
Loaded ds
(128, 128)
TransferSyntaxUID: 1.2.840.10008.1.2.1
BitsAllocated: 16
SamplesPerPixel: 1

There is no additional info in the console logs:

I know that I can solve this issue by adding from pydicom.pixel_data_handlers.util import pixel_dtype to the imports. But I think this issue is masking other problems that I am experiencing when trying load other DICOMS that rely on additional packages that pydicom imports itself e.g JPEG2000 encoding DICOMS for which I have build a Pyodide wheel.

What browsers are you seeing the problem on? (if applicable)

Firefox, Chrome

Console info

Firefox 111.0.1 (64-bit) on MacOS

Chrome 111.0.5563.146 (Official Build) (arm64)

Additional Context

No response

Apr 06 '23 08:04 dazzag24

You're right to be wary - there is an ongoing issue #1137 with Exceptions in coroutines not being propagated to the screen.

Apr 06 '23 12:04 JeffersGlass

@JeffersGlass Thanks for the heads up. I attempted to add your exception handler to my example. It didn't result in any extra info in the output or in the console.

Apr 06 '23 15:04 dazzag24

This is definitely a pyscript problem. If you do the following in Pyodide:

await pyodide.loadPackage(["micropip", "numpy"]);
let micropip = pyodide.pyimport("micropip");
await micropip.install(["pydicom"])
pyodide.runPython(`
      import micropip
      import asyncio
      import pydicom
      import numpy as np
      from pydicom.data import get_testdata_file
      #from pydicom.pixel_data_handlers.util import pixel_dtype
      
      async def main():
          print("Reading pydicom package test file")
          path = get_testdata_file("CT_small.dcm")
          print(f"Loading {path}")
          ds = pydicom.dcmread(path)   
          print("Loaded ds")       
          
          print(f"{ds.pixel_array.shape}")
          print(f"TransferSyntaxUID: {ds.file_meta.TransferSyntaxUID}")
          print(f"BitsAllocated: {ds.get('BitsAllocated', 'NONE')}")
          print(f"SamplesPerPixel: {ds.get('SamplesPerPixel', 'NONE')}")
          print(f"Pixel dtype: {pixel_dtype(ds)}")

      asyncio.ensure_future(main())
`);

It says:

PythonError: Traceback (most recent call last):
  File "<exec>", line 20, in main
NameError: name 'pixel_dtype' is not defined

So PyScript is somehow catching the error and throwing it away? I'm confused about how exactly.

Apr 06 '23 17:04 hoodmane

Minimal reproduction is:

<html>
  <head>
    <link rel="stylesheet" href="https://pyscript.net/latest/pyscript.css" />
    <script defer src="https://pyscript.net/latest/pyscript.js"></script>
  </head>
  <body>
    <py-script>
      import asyncio
        
      async def main():
          raise Exception("hi")

      asyncio.ensure_future(main())
    </py-script>
  </body>
</html>

There isn't even an error in the console...

Apr 06 '23 17:04 hoodmane

This is definitely a pyscript problem. If you do the following in Pyodide:

await pyodide.loadPackage(["micropip", "numpy"]);
let micropip = pyodide.pyimport("micropip");
await micropip.install(["pydicom"])
pyodide.runPython(`
      import micropip
      import asyncio
      import pydicom
      import numpy as np
      from pydicom.data import get_testdata_file
      #from pydicom.pixel_data_handlers.util import pixel_dtype
      
      async def main():
          print("Reading pydicom package test file")
          path = get_testdata_file("CT_small.dcm")
          print(f"Loading {path}")
          ds = pydicom.dcmread(path)   
          print("Loaded ds")       
          
          print(f"{ds.pixel_array.shape}")
          print(f"TransferSyntaxUID: {ds.file_meta.TransferSyntaxUID}")
          print(f"BitsAllocated: {ds.get('BitsAllocated', 'NONE')}")
          print(f"SamplesPerPixel: {ds.get('SamplesPerPixel', 'NONE')}")
          print(f"Pixel dtype: {pixel_dtype(ds)}")

      asyncio.ensure_future(main())
`);

It says:

PythonError: Traceback (most recent call last):
  File "<exec>", line 20, in main
NameError: name 'pixel_dtype' is not defined

So PyScript is somehow catching the error and throwing it away? I'm confused about how exactly.

Interesting how did you test in Pyiodide?

Apr 06 '23 18:04 dazzag24

Interesting how did you test in Pyodide?

Go to https://pyodide.org/en/0.22.1/console.html and wait for it to finish loading
Open the browser console (ctrl+shift+J in Chrome)
Paste the code I wrote in my comment into the console

Apr 06 '23 18:04 hoodmane

Oh wait, never mind I seem to get the same problem in Pyodide. So it is an upstream problem!

Eventually it says:

Task exception was never retrieved
future: <Task finished name='Task-1' coro=<main() done, defined at <exec>:4> exception=Exception('hi')>

Apr 06 '23 18:04 hoodmane

You can do:

import asyncio
import traceback

async def main():
    raise Exception("hi")

def maybe_format_err(fut):
   exc = fut.exception()
   if exc:
      traceback.print_exception(exc)   

asyncio.ensure_future(main()).add_done_callback(maybe_format_err)

I thought you would be able to do:

asyncio.ensure_future(main()).catch(lambda e: traceback.format_exception(e))

but that doesn't work. I will make a PR to fix it...

Apr 07 '23 17:04 hoodmane

Interestingly there appears to be a bug in Pyodide v0.22.1 which was fixed (accidentally??) in v0.23.0. Probably would be good to figure out what's going on there and add a test.

Apr 07 '23 18:04 hoodmane

No follow up in here, closing as Pyodide is way beyond 0.22.1 or 0.23

Dec 06 '23 17:12 WebReflection

pyscript pyscript copied to clipboard

Execution stops without error

Checklist

What happened?

What browsers are you seeing the problem on? (if applicable)

Console info

Additional Context

pyscript
pyscript copied to clipboard