incubator-livy icon indicating copy to clipboard operation
incubator-livy copied to clipboard

[LIVY-764] Give code block a filename and cache it in linecache

Open mzhang-code opened this issue 4 years ago • 1 comments

[LIVY-764] Give code block a filename and cache it in linecache

What changes were proposed in this pull request?

Today python shell (fake_shell.py) uses 'stdin' as file name to compile code blocks and never caches them in linecache. This breaks the user code that depends on inspect.findsource, like torch.jit.script

To enable code block caching at the backend, we can use the way how ipython does: https://github.com/ipython/ipython/blob/44bd47d5ac27805a297798d39ad7e31f63181468/IPython/core/compilerop.py#L54

How was this patch tested?

E2E test with local livy server

mzhang-code avatar Apr 24 '20 23:04 mzhang-code

This change enables inspect.findsource and torch.jit.script

image

image

mzhang-code avatar Apr 24 '20 23:04 mzhang-code