Use SublimeText Build Systems for dubugging

[YaraslauZhylko]

I have an alternative build system set-up for my Python 3 project:

"build_systems": [
    {
        "file_regex": "^[ ]*File \"(...*?)\", line ([0-9]*)",
        "name": "Test Project Python Builder",
        "selector": "source.python",
        "shell_cmd": "~/.virtualenvs/test/bin/python3.5 -u \"$file\""
    }
]

As you can see this one uses python3.5 interpreter from a virtual environment with all specific dependencies installed.

But Sublime Debugger plug-in seems to use the default python3 interpreter.

Is there a way to choose a specific "debugging" interpreter for each project?

Or add a functionality to utilize corresponding Build Systems' paths?

[matiasmorant]

Not yet. It's the second time it's requested, so I will implement it when I can.

[matiasmorant]

As a work around, you can substitute the string "python2" by "python3.5" (or full path) in the DBPython2 file, and from then on use the Python 2 debugger in sublime (which will use your interpreter). This is not guaranteed to work

[YaraslauZhylko]

Thanks for offering the workaround.

Unfortunately, it does not work for me.

That's what I get when I try to enter debug:

Started Debugging
Exception in thread Thread-12:
Traceback (most recent call last):
  File "./python3.3/threading.py", line 901, in _bootstrap_inner
  File "./python3.3/threading.py", line 1142, in run
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 48, in <lambda>
    threading.Timer(.2, lambda: DB.runscript(filename)).start()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython2.py", line 19, in runscript
    self.peer.D_set_breakpoints(self.breakpoints)
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/comm_utils.py", line 57, in f
    self.client_conn.send((m+"$@#"+json.dumps(args)+"$@#.").encode("UTF-8"))
BrokenPipeError: [Errno 32] Broken pipe

And that's when I try to add a breakpoint:

Traceback (most recent call last):
  File "/opt/sublime_text/sublime_plugin.py", line 797, in run_
    return self.run()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 123, in run
    toggle_breakGUI(filename, line)
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 158, in toggle_breakGUI
    bps.pop(line) if line in bps else bps.update({line: {}})
  File "./python3.3/contextlib.py", line 55, in __exit__
    
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 136, in bp_manager
    V.add_regions("bp", [get_line(V, l - 1) for l in bps], *style)
AttributeError: 'NoneType' object has no attribute 'add_regions'

Am I missing anything?

Do I have to install some package into the virtual environment?

[matiasmorant]

If after changing the string, you closed and re started sublime (which you should do), then I don't know. The traceback is not very informative. I'll have to analyze and try the problem myself

[YaraslauZhylko]

I've restarted it several times. To no avail.

Today I've tried to start out-of-the-box Python2 debugger on another machine. And it failed with the following error:

Started Debugging
Exception in thread Thread-11:
Traceback (most recent call last):
  File "./python3.3/threading.py", line 901, in _bootstrap_inner
  File "./python3.3/threading.py", line 1142, in run
  File "mydebugger in /home/yaraslau/.config/sublime-text-3/Installed Packages/Python Debugger.sublime-package", line 48, in <lambda>
  File "backends.dbPython2 in /home/yaraslau/.config/sublime-text-3/Installed Packages/Python Debugger.sublime-package", line 19, in runscript
  File "backends.comm_utils in /home/yaraslau/.config/sublime-text-3/Installed Packages/Python Debugger.sublime-package", line 57, in f
BrokenPipeError: [Errno 32] Broken pipe

Python 3 debugger (with default python3) is working OK though.

[YaraslauZhylko]

I've also got this in the console:

...
reloading plugin Python Debugger.mydebugger
...
plugins loaded
...
lang_service_enabled: True
...
2017-12-01 09:37:07,527: 140247586776896: ERROR: can not send request; node process not running
ref view not found
...
2017-12-01 09:40:27,109: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,110: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,110: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,111: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,111: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,112: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,112: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,113: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,113: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,113: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,114: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
2017-12-01 09:40:27,114: 140247586776896: ERROR: can not send request; node process not running
ERROR:TS:can not send request; node process not running
...
doesn't want to connect 0 5004
doesn't want to connect 1 5004
doesn't want to connect 2 5004
doesn't want to connect 3 5004
doesn't want to connect 4 5004
doesn't want to connect 5 5004
doesn't want to connect 6 5004
doesn't want to connect 7 5004
doesn't want to connect 8 5004
doesn't want to connect 9 5004
doesn't want to connect 0 5005
doesn't want to connect 1 5005
doesn't want to connect 2 5005
doesn't want to connect 3 5005
doesn't want to connect 4 5005
doesn't want to connect 5 5005
doesn't want to connect 6 5005
doesn't want to connect 7 5005
doesn't want to connect 8 5005
doesn't want to connect 9 5005
fully connected
language: Python2
Traceback (most recent call last):
  File "backends.comm_utils in /home/yaraslau/.config/sublime-text-3/Installed Packages/Python Debugger.sublime-package", line 80, in loop
  File "backends.comm_utils in /home/yaraslau/.config/sublime-text-3/Installed Packages/Python Debugger.sublime-package", line 75, in __call__
  File "backends.comm_utils in /home/yaraslau/.config/sublime-text-3/Installed Packages/Python Debugger.sublime-package", line 47, in recv_message
OSError: [Errno 107] Transport endpoint is not connected
connection down [Errno 107] Transport endpoint is not connected

Not sure if all of that is related to Python Debugger...

[YaraslauZhylko]

And yes, python2 is installed and available via PATH:

yaraslau@dell:~ $ echo $PATH
/home/yaraslau/bin:/home/yaraslau/.local/bin:/home/yaraslau/bin:/home/yaraslau/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
yaraslau@dell:~ $ which python2
/usr/bin/python2
yaraslau@dell:~ $ python2
Python 2.7.13 (default, Nov 23 2017, 15:37:09)
[GCC 6.3.0 20170406] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

As well as python3:

yaraslau@dell:~ $ which python3
/usr/bin/python3
yaraslau@dell:~ $ which python3.5
/usr/bin/python3.5
yaraslau@dell:~ $ python3
Python 3.5.3 (default, Nov 23 2017, 11:34:05)
[GCC 6.3.0 20170406] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

[matiasmorant]

Yes, it is related with Python debugger. I think what is happening is that dbPython2 isn't successfully launching the process running the debugger. Which in your case would be running python3.5, if you changed the string.

If the string is "python3.5", my guess is that the python2 debugger backend code is incompatible with python3.

If the string is "python2", I don't know what is happening.

The following is something I do when debugging the plug in, it takes some work.

The backend doesn't print error messages in sublime because it runs in a different process. We want to see them. So, instead of letting the plug in launch the process, we'll launch it ourselves in a terminal.

In dbPython2 comment the line which launches the process (the one containing popen and your interpreter string) and all the code which depends on that process being created.

Then, before starting sublime, launch the debugger process yourself in a terminal (find out the command by reading the arguments passed to popen that you just commented out)

Open sublime, and at some point, we will get our desired error in the terminal.

[YaraslauZhylko]

I started python2 dbpy2_server.py manually without errors.

When I started Sublime Text and switched to Python 2 I got the following output into the ST console:

fully connected
language: Python2

dbpy2_server.py output was:

mydb2.py started
debugger ready
('connecting', 5004)
('connected', ('127.0.0.1', 54826))
('connecting', 5005)
('connected', ('127.0.0.1', 34426))
parent ready

Then I tried adding a breakpoint.

Sublime Text:

Traceback (most recent call last):
  File "/opt/sublime_text/sublime_plugin.py", line 797, in run_
    return self.run()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 123, in run
    toggle_breakGUI(filename, line)
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 158, in toggle_breakGUI
    bps.pop(line) if line in bps else bps.update({line: {}})
  File "./python3.3/contextlib.py", line 55, in __exit__
    
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 136, in bp_manager
    V.add_regions("bp", [get_line(V, l - 1) for l in bps], *style)
AttributeError: 'NoneType' object has no attribute 'add_regions'

dbpy2_server.py output did not change.

Then I tried entering debug.

Sublime Text started working as expected stepping through code. With the following output:

Started Debugging
{'bytearray': <type 'bytearray'>, 'IndexError': <type 'exceptions.IndexError'>, 'all': <built-in function all>, 'help': Type help() for interactive help, or help(object) for help about object., 'vars': <built-in function vars>, 'SyntaxError': <type 'exceptions.SyntaxError'>, 'unicode': <type 'unicode'>, 'UnicodeDecodeError': <type 'exceptions.UnicodeDecodeError'>, 'memoryview': <type 'memoryview'>, 'isinstance': <built-in function isinstance>, 'copyright': Copyright (c) 2001-2017 Python Software Foundation.
All Rights Reserved.
...

dbpy2_server.py also started producing sane output:

--line--
('break at', '/home/yaraslau/test_project/loop.py', 1, 'in', '<module>')
--line--
('break at', '/home/yaraslau/test_project/loop.py', 3, 'in', '<module>')
--line--
('break at', '/home/yaraslau/test_project/loop.py', 4, 'in', '<module>')
--line--
('break at', '/home/yaraslau/test_project/loop.py', 7, 'in', '<module>')
--line--
('break at', '/home/yaraslau/test_project/loop.py', 11, 'in', '<module>')
--line--
('break at', '/home/yaraslau/test_project/loop.py', 22, 'in', '<module>')
--line--
('break at', '/home/yaraslau/test_project/loop.py', 37, 'in', '<module>')
1000
--line--
('break at', '/home/yaraslau/test_project/loop.py', 38, 'in', '<module>')

But, as you can get, there were no breakpoints 'cause they initially failed to be set.

Then I stopped debugging with q command. Both outputs did not change.

Upon closing Sublime Text I've got the following error from dbpy2_server.py:

Traceback (most recent call last):                                                                   
  File "/home/yaraslau/Dropbox/sys/dot.config/sublime-text-3/Packages/Python Debugger/backends/comm_u
tils.py", line 80, in loop                                                                           
    while True: self()                                                                               
  File "/home/yaraslau/Dropbox/sys/dot.config/sublime-text-3/Packages/Python Debugger/backends/comm_u
tils.py", line 76, in __call__                                                                       
    ans = self[msg]
  File "/home/yaraslau/Dropbox/sys/dot.config/sublime-text-3/Packages/Python Debugger/backends/comm_u
tils.py", line 66, in __getitem__
    instruction, parameters , _ = m.split('$@#')
ValueError: need more than 1 value to unpack
('connection down', ValueError('need more than 1 value to unpack',))

I've decided to repeat the experiment and just switch back to Python 3 in Sublime Text instead of quitting it.

There were no errors:

language: Python3

But when I tried switching to Python 2 again I got the following error in Sublime Text console:

Traceback (most recent call last):
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/comm_utils.py", line 80, in loop
    while True: self()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/comm_utils.py", line 75, in __call__
    msg = recv_message(self.client_conn)
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/comm_utils.py", line 47, in recv_message
    message = conn.recv(BUFFER_SIZE).decode("UTF-8")
OSError: [Errno 107] Transport endpoint is not connected
connection down [Errno 107] Transport endpoint is not connected

dbpy2_server.py produced nothing.

Closing Sublime Text produced the same dbpy2_server.py error described above.

[YaraslauZhylko]

Next, I tried the same scenario but with my virtualenv's python3.5.

Expectedly got the following 2to3 related error:

  File "dbpy2_server.py", line 7
    print "mydb2.py started"
                           ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("mydb2.py started")?

Modified all print statements and got it to launch:

mydb2.py started
debugger ready
connecting 5004

N.B.: "mydb2.py started" message was produced by print("mydb2.py started") statement. Actual command was python3.5 from virtualenv.

I managed to connect to it by switching to Python 2 in Sublime Text.

Tried setting breakpoints. But they silently failed without any output to either Sublime Text or dbpy2_server.py console. It just did nothing.

Then I tried entering debug mode. It did work. And I managed to step through the code importing virtualenv specific libraries without errors.

Closing Sublime Text resulted in the same dbpy2_server.py "quitting" error that we saw for python2 in the message above:

Traceback (most recent call last):
  File "/home/yaraslau/Dropbox/sys/dot.config/sublime-text-3/Packages/Python Debugger/backends/comm_utils.py", line 80, in loop
    while True: self()
  File "/home/yaraslau/Dropbox/sys/dot.config/sublime-text-3/Packages/Python Debugger/backends/comm_utils.py", line 76, in __call__
    ans = self[msg]
  File "/home/yaraslau/Dropbox/sys/dot.config/sublime-text-3/Packages/Python Debugger/backends/comm_utils.py", line 66, in __getitem__
    instruction, parameters , _ = m.split('$@#')
ValueError: not enough values to unpack (expected 3, got 1)
connection down not enough values to unpack (expected 3, got 1)

[YaraslauZhylko]

So there are definitely some errors:

1. Debugger does not start correctly via Popen(). Maybe there are some problems with PATH within Sublime Text?
2. Breakpoints are not set.
3. Dropping connection results in error. That's probably not a problem since debugger code terminates the process anyway.

[matiasmorant]

See if the latest commit solves the problem.

There is a settings file now, where you can set your interpreter

[YaraslauZhylko]

Python 3 (Sublime built-in)

1. Loads without errors.

2. Can set breakpoints without errors.

3. Raises the following exception when starting debugging without opening Variables/Expression/Breakpoints windows:

Started Debugging
--line--
break at /home/yaraslau/test_project/json-logic.py 1 in <module>
Traceback (most recent call last):
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3S.py", line 176, in runscript
    self.run(statement)
  File "./python3.3/bdb.py", line 405, in run
  File "<string>", line 1, in <module>
  File "/home/yaraslau/test_project/json-logic.py", line 1, in <module>
    from json_logic import jsonLogic
  File "/home/yaraslau/test_project/json-logic.py", line 1, in <module>
    from json_logic import jsonLogic
  File "./python3.3/bdb.py", line 47, in trace_dispatch
  File "./python3.3/bdb.py", line 65, in dispatch_line
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3S.py", line 50, in user_line
    self.wait_cmd(frame) # continue to next breakpoint
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3S.py", line 72, in wait_cmd
    cmd = self.parent.get_cmd(line(frame),ls,gs, filename(frame))
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 69, in get_cmd
    refresh_expressions()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 297, in refresh_expressions
    expressions = get_keys(get_view_content("Expression"))
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 303, in get_keys
    keys = [l.split(' ┃ ')[0].strip() for l in txt.split('\n')]
AttributeError: 'NoneType' object has no attribute 'split'
Uncaught exception. Entering post mortem debugging
Exception in thread Thread-10:
Traceback (most recent call last):
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3S.py", line 176, in runscript
    self.run(statement)
  File "./python3.3/bdb.py", line 405, in run
  File "<string>", line 1, in <module>
  File "/home/yaraslau/test_project/json-logic.py", line 1, in <module>
    from json_logic import jsonLogic
  File "/home/yaraslau/test_project/json-logic.py", line 1, in <module>
    from json_logic import jsonLogic
  File "./python3.3/bdb.py", line 47, in trace_dispatch
  File "./python3.3/bdb.py", line 65, in dispatch_line
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3S.py", line 50, in user_line
    self.wait_cmd(frame) # continue to next breakpoint
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3S.py", line 72, in wait_cmd
    cmd = self.parent.get_cmd(line(frame),ls,gs, filename(frame))
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 69, in get_cmd
    refresh_expressions()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 297, in refresh_expressions
    expressions = get_keys(get_view_content("Expression"))
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 303, in get_keys
    keys = [l.split(' ┃ ')[0].strip() for l in txt.split('\n')]
AttributeError: 'NoneType' object has no attribute 'split'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./python3.3/threading.py", line 901, in _bootstrap_inner
  File "./python3.3/threading.py", line 1142, in run
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 48, in <lambda>
    threading.Timer(.2, lambda: DB.runscript(filename)).start()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3S.py", line 187, in runscript
    self.wait_cmd(self.stack[self.curidx][0])
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3S.py", line 72, in wait_cmd
    cmd = self.parent.get_cmd(line(frame),ls,gs, filename(frame))
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 69, in get_cmd
    refresh_expressions()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 297, in refresh_expressions
    expressions = get_keys(get_view_content("Expression"))
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 303, in get_keys
    keys = [l.split(' ┃ ')[0].strip() for l in txt.split('\n')]
AttributeError: 'NoneType' object has no attribute 'split'

4. Starts as expected with Variables/Expression/Breakpoints windows open. But fails to load virtualenv specific modules (as expected):

Started Debugging
--line--
break at /home/yaraslau/test_project/json-logic.py 1 in <module>
--exception--
exception in <module> (<class 'ImportError'>, ImportError("No module named 'json_logic'",), <traceback object at 0x7fe2a11b1098>)
--return--
return from <module> None
--exception--
exception in <module> (<class 'ImportError'>, ImportError("No module named 'json_logic'",), <traceback object at 0x7fe2a11b29e0>)

Those errors are only visible in Sublime Text console. While previously import errors where in the status bar.

5. Making several attempts to step to the next line (n command) after the import error results in Sublime Text opening an empty window called <string>. With the following error in ST console:

Unable to open /opt/sublime_text/<string>

This error did appear before the fix.

6. If I comment out virtualenv specific code I'm able to reach all the breakpoints with c command. Even the nested ones:

@ for a in range(10):
      do_something()
@     print('Breakpoint here')
      do_something_else()

See custom interpreter data below for details.

[YaraslauZhylko]

Python 3 [with custom virtualenv interpreter set via configuration file]

1. Loads without errors.

2. Can set breakpoints without errors.

3. Raises the following exception when starting debugging without opening Variables/Expression/Breakpoints windows (this exception is repeated multiple times, about 100 at least):

Traceback (most recent call last):
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/comm_utils.py", line 69, in __getitem__
    ret = eval("self."+instruction)(*json.loads(parameters))
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/backends/dbPython3.py", line 40, in E_get_cmd
    def E_get_cmd       (self, line,locals,globals,filename): return self.parent.get_cmd (line,locals,globals,filename)
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 69, in get_cmd
    refresh_expressions()
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 297, in refresh_expressions
    expressions = get_keys(get_view_content("Expression"))
  File "/home/yaraslau/.config/sublime-text-3/Packages/Python Debugger/mydebugger.py", line 303, in get_keys
    keys = [l.split(' ┃ ')[0].strip() for l in txt.split('\n')]
AttributeError: 'NoneType' object has no attribute 'split'

4. Starts as expected with Variables/Expression/Breakpoints windows open. And manages to load virtualenv specific modules without errors.

5. Empty <string> window with the same error does pop-up when I try to enter internal loop code with n command for 2+ times. (see below)

6. When trying to use the c command debugger sometimes fail to reach the nested breakpoints (those within the loop code):

@ for a in range(10):
      do_something()
@     print('Breakpoint here')
      do_something_else()

Most of the times it does manage to stop at them during first loop iteration. But then fails to do so during the following ones. The same is true for n command: internal loop code is only accessible during first iteration.

[matiasmorant]

Latest commit solves 3. Confirm

[YaraslauZhylko]

Yes, number 3 seems to be gone.

2, 5 and 6 are still there though.

[YaraslauZhylko]

And I also think it would be better to actually get interpreter command from currently active Sublime Build System as compared to using a config file.

Otherwise people will have to edit the config every time they want to debug another project they are working on. Or do some more hacking to add each of their projects' interpreters to the Set Language menu.

And current Build System's interpreter (the one that is run by hitting ctrl+b) is a better candidate for default interpreter than Sublime's internal one (that does not have required dependencies for any of the projects). So people will have to switch language every time they want to debug something instead of just hitting ctrl+alt+d.

[metakot]

Config editing is not available, the file "SublimeDebugger.sublime-settings" gots rewrited every time then choosing Python version from "Tools > Debugger > Set Language", rendering it as useless. Is there a workaround for that kind of situation?