Multiprocessing gets stuck upon Exception handling

[mathisrichter]

Objective of issue: When an exception is thrown, the program hangs before the program can terminate and report the exception.

Lava version:

• [ ] 0.4.0 (feature release)
• [ ] 0.3.1 (bug fixes)
• [x] 0.3.0 (current version)
• [ ] 0.2.0
• [ ] 0.1.2

I'm submitting a ...

• [x] bug report
• [ ] feature request
• [ ] documentation request

Current behavior:

• When an exception occurs (possibly in the builder-phase), the program gets stucks and runs indefinitely. What actually happens is that the exception is thrown but the program gets stuck before the exception can bubble up all the way to the surface. This does not happen for all exceptions but for those where it happens, the behavior seems to be deterministic.

In order to debug any exceptions that arise like this and get at the traceback information, one can set a breakpoint in lava/magma/runtime/message_infrastructure/multiprocessing.py (line 43) and inspect the tb variable. Also, make sure that this exception is configured to stop all threads. In PyCharm that can be set in Run->View breakpoints->(select a breakpoint)->Suspend: 'All' (not 'Thread').

Expected behavior:

• When an exception is thrown, it bubbles up all the way to the surface (if it is not handled somewhere) and the program terminates, printing the traceback information.

Steps to reproduce:

Not all exceptions get stuck. Here is a change that creates an exception that gets stuck. In lava/magma/compiler/builders/builder.py, add name = "foo"

    def _get_lava_type(self, name: str) -> LavaPyType:
        name = "foo"
        return getattr(self.proc_model, name)

and then execute a unit test that builds a Port, for instance the unit tests in magma/core/process/ports/test_virtual_ports_in_process.py. This internally throws the following exception AttributeError: type object 'PyRefPortWriteProcessModelFloat' has no attribute 'foo' but the program gets stuck before it can terminate and print the traceback information.

Related code:

Other information: