drepl icon indicating copy to clipboard operation
drepl copied to clipboard

Published 20 hours ago • verified publisher icon astoff

can't run drepl-ipython

Open pank opened this issue 1 year ago • 7 comments
trafficstars

Hi,

I have installed v0.3 from ELPA on windows using Anaconda python v3.11 or v3.12.

I wanted to try this project as I miss comint when using emacs-jupyter.

However, the python process stops before drepl is started:

Python 3.11.5 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:26:23) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.15.0 -- An enhanced Interactive Python. Type '?' for help.

Do you really want to exit ([y]/n)? 

Process dnatplot/*IPython* finished

I don't fully understand the initialization process, but it seems drepl--init calls python -c "import sys; exec(''.join(sys.stdin)); Drepl.instance().mainloop()" via the ipython drepl--command. drepl-ipython.py is then sent as stdin (I'm curious why Drepl.instance().mainloop() couldn't just be run if __name__ == '__main__'?).

The log isn't very revealing either:

[11:42:21] starting dnatplot/*IPython*
[11:42:23] read {"op": "getoptions"}
[11:42:23] send msg {"id":1,"op":"setoptions","prompts":["In [{}]: ","...: ","\u001B[31mOut[{}]:\u001B[0m ","\n",""]}
[11:42:23] read {"op": "status", "status": "ready"}
[11:42:23] read {"op": "status", "status": "rawio"}

It seems that the EOFError exception in the Drepl.mainloop method is triggered. when I change the default to 'n' in the exit question, the error get triggered continuously.

I'm not quite sure how to debug this further. Any ideas?

pank avatar Jul 30 '24 10:07 pank

Okay, I'm pretty sure this is related to the fact you're using Windows. Certainly it's necessary that the IPython subprocess runs in a PTY as opposed to via pipes. I guess you can check that by running M-: (process-tty-name (get-buffer-process (current-buffer))) RET in the dREPL buffer.

You could modify the way initialization works, e.g. by just calling python path/to/drepl-ipython.py. The disadvantage (and reason for the more contrived initalization) is that this will not work over Tramp, since drepl-ipython.py is not present in the remote machine.

If we are able to confirm that the contrived initalization cannot work on Windows, then I would add a simpler, Tramp-incompatible workaround for Windons only.

I'm curious why Drepl.instance().mainloop() couldn't just be run if __name__ == '__main__'?

I guess you're right ;-).

astoff avatar Aug 08 '24 07:08 astoff

I can run python -i drepl-ipython.py and it starts. Or in emacs I can eval (setq python-shell-interpreter "python" python-shell-interpreter-args "-i c:/users/me/.emacs.d/elpa/drepl-0.3/drepl-ipython.py") and then use run-python. I don't think I associate it with drepl after. In any case (process-tty-name (get-buffer-process (current-buffer))) returns nil in this buffer.

When I modify the drepl-startup:

(cl-defmethod drepl--command ((_ drepl-ipython))
  `(,python-interpreter "-i"
                        "c:/users/me/.emacs.d/elpa/drepl-0.3/drepl-ipython.py"))

I get funny errors about indentation errors. I don't see any. I tried to change the file encoding and the line endings to the silly ones (windows ones), but that didn't fix the issue.

>>> 'IPython interface for dREPL.'
>>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> ... ... ... ... >>> >>> ... ... ... ... ... ... >>> >>> >>> ... ... >>> >>> ... ... ... ... ... ... ... ... ... ... >>> >>> ... ... >>> >>> ... ... ... ... ... ... >>>   File "<stdin>", line 1
    def write_format_data(self, format_dict, md_dict=None) -> None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    for mime, handler in self.shell.mime_renderers.items():
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if mime in format_dict:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    handler(format_dict[mime], None)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    return
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    super().write_format_data(format_dict, md_dict)
IndentationError: unexpected indent
>>> >>> >>> ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... >>>   File "<stdin>", line 1
    system = InteractiveShell.system_raw
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    displayhook_class = DreplDisplayHook
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def make_mime_renderer(self, type, encoder):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    def renderer(data, meta=None):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if encoder:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    data = encoder(data)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    header = json.dumps({**(meta or {}), "type": type})
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if len(data) > self.mime_size_limit:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    fdesc, fname = mkstemp()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    with open(fdesc, "wb") as f:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    f.write(data)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    payload = "tmp" + Path(fname).as_uri()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    else:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    payload = base64.encodebytes(data).decode()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    print(f"\033]5151;{header}\n{payload}\033\\")
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    return renderer
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def enable_mime_rendering(self, mime_types=None):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    """Enable rendering of the given mime types; if None, enable all."""
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if mime_types is None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    mime_types = MIME_TYPES
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    for t in mime_types:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if t in MIME_TYPES:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.display_formatter.formatters[t].enabled = True
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def ask_exit(self):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.keep_running = False
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def enable_gui(self, gui=None):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if gui != "inline":
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    print("Can't enable this GUI: {}".format(gui))
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def mainloop(self):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    while self.keep_running:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    try:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.run_once()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    except EOFError:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(op="status", status="rawio")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if (not self.confirm_exit) or self.ask_yes_no(
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    "Do you really want to exit ([y]/n)?", "y", "n"
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    ):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.ask_exit()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    except (DreplError, KeyboardInterrupt) as e:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    print(str(e) or e.__class__.__name__)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def run_once(self):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    "Print prompt, run REPL until a new prompt is needed."
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if self.current_ps1 is None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(op="getoptions")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.current_ps1, separate_in = "", ""
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    else:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    separate_in = self.separate_in if self.current_ps1 else ""
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.current_ps1 = sys.ps1.format(self.execution_count)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    print(separate_in + self.current_ps1, end="")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    while True:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    data = readmsg()
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    op = data.pop("op")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    fun = getattr(self, "drepl_{}".format(op), None)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if fun is None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    raise DreplError("Invalid op: {}".format(op))
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    fun(**data)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if op == "eval":
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    self.execution_count += 1
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    break
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if op == "setoptions":
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    break
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_eval(self, id, code):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(op="status", status="rawio")
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    r = self.run_cell(code)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_complete(self, id, code, pos):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    with provisionalcompleter():
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    completions = rectify_completions(code, self.Completer.completions(code, pos))
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    first = next(completions, None)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if first is None:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    return
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    prefix = code[first.start: pos]
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    completions = chain([first], completions)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    candidates = [{"text": c.text, "annot": c.signature} for c in completions]
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id, prefix=prefix, candidates=candidates)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_checkinput(self, id, code):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    status, indent = self.check_complete(code)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    prompt = sys.ps2.format(self.execution_count).rjust(len(self.current_ps1))
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id, status=status, indent=indent, prompt=prompt)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_describe(self, id, code, pos):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    name = token_at_cursor(code, pos)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    try:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    info = self.object_inspect(name)
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    defn = info["definition"]
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    id=id,
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    name=info["name"],
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    type=" ".join(defn.split()) if defn else info["type_name"],
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    file=info["file"],
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    text=self.object_inspect_text(name),
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    )
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    except Exception:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id)
IndentationError: unexpected indent
>>> >>>   File "<stdin>", line 1
    def drepl_setoptions(self, id, prompts=None):
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    if prompts:
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sys.ps1, sys.ps2, sys.ps3, self.separate_in, self.separate_out = prompts
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    sendmsg(id=id)
IndentationError: unexpected indent
>>>

pank avatar Aug 09 '24 13:08 pank

Okay, python -i is not the right way to do it. What you would need is:

  1. Add the if __name__ == "__main__" trick to drepl-ipython.py (which is a change I just pushed to Github)
  2. Change the following 2 methods:
(cl-defmethod drepl--command ((_ drepl-ipython))
  `(,python-interpreter ,drepl-ipython--start-file))

(cl-defmethod drepl--init ((repl drepl-ipython))
  (cl-call-next-method repl)
  (drepl--adapt-comint-to-mode ".py")
  (push '("5151" . comint-mime-osc-handler) ansi-osc-handlers))

Again, this is what one would need to do to run the process over pipes (as opposed to a PTY). But ideally I would like to find out why the PTY solution is not working on Windows.

astoff avatar Aug 10 '24 07:08 astoff

I will try this on Monday when I turn on my windows pc again.

pank avatar Aug 10 '24 07:08 pank

With the new versions of drepl-ipython.{py, el} and the two changes above, I can run drepl-ipython. Completions and plots work as expected (based on very brief testing only).

pank avatar Aug 12 '24 09:08 pank

Okay, thanks for bringing to my attention that Windows doesn't have PTYs. I still need to think how to best work around this limitation.

astoff avatar Aug 12 '24 10:08 astoff

@pank If you would like to, feel free to test this branch: https://github.com/astoff/drepl/pull/5

astoff avatar Aug 14 '24 13:08 astoff