sublime_text
sublime_text copied to clipboard
sublimehq
Sublime Text 4 freezes on open quote, parenthesis, brackets, braces, etc
Description
A description of what the bug is. It happens more often in a big files, but it also happens to a lesser extent in small files.
I have run strace and this happens when it freezes.
11:30:58 futex(0x7ff9fc158000, FUTEX_WAKE, 1) = 1
11:30:58 futex(0x7ff9fc138000, FUTEX_WAIT_BITSET|FUTEX_CLOCK_REALTIME, 0, NULL, FUTEX_BITSET_MATCH_ANY) = -1 EAGAIN (Recurso no disponible temporalmente)
11:30:58 futex(0x7ff9fc158000, FUTEX_WAKE, 1) = 1
11:30:58 futex(0x7ff9fc158000, FUTEX_WAKE, 1) = 1
11:30:58 mmap(NULL, 54521856, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9b0c01000
11:31:02 munmap(0x7ff9b0c01000, 54521856) = 0
11:31:02 futex(0x7ff9e40197d8, FUTEX_WAKE_PRIVATE, 1) = 1
11:31:02 recvmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\241 w9\7\0`\3\316\1\0\0006\3\0\0\0\0\0\0\234\321\217\336\0\0\0\0\0\0\0\0"..., iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 4096
Steps to reproduce
For example having something like this in a big file (100KB), these examples cause a freeze.
From:
$test = 'Test text';
class tv_broadcasts
{
}
To:
$test = 'Test text
class tv_broadcasts
{
}
Or from:
$test = 'Test ' . ( . 'text';
class tv_broadcasts
{
}
To:
$test = 'Test ' . () . 'text';
class tv_broadcasts
{
}
I understand that this happens because this changes causes a structure change throughout the document.
Expected behavior
Sublime Text 4 should not freeze, previously with Sublime Text 3 I did not have this problem
Actual behavior
Sublime Text freezes.
Environment
- Build: 4113
- Operating system and version: Ubuntu 20.04.2 LTS
- [Linux] Desktop Environment and/or Window Manager: Gnome 3.36.8, Xorg
In safe mode it keeps happening.
I have tried to reduce the size of the file little by little and the smaller it is, the less time the freeze lasts, although it continues to happen.
This happens to me a lot when editing SQL queries, as I am constantly adding and removing quotes/parentheses.
Is it PHP code?
SQL highlighting is achieved via push with_prototype atm.
I wonder if things change with
- PHP package from https://github.com/sublimehq/Packages/pull/2789
or even
- PHP package from https://github.com/deathaxe/sublime-packages/tree/wip/php
Both add changes to avoid with_prototype and its implications. By using embed they should also isolate different syntaxes more strictly, so changing/breaking SQL code shouldn't effect the whole document.
Yes is PHP code, but not only happens with SQL queries, I have made a recording so that you can see how it freezes, I have tested it with the functions.php file of phpBB, I have enlarged it by duplicating the code several times so that it can be seen in the recording, since otherwise it lasted only about a second.
I will test the package you comment.
https://github.com/phpbb/phpbb/blob/8b597f59ce833fc134fe259ea106b9b05bf2440a/phpBB/includes/functions.php
https://user-images.githubusercontent.com/18549191/129475864-41b3be0e-e20e-4dc8-aecf-918831f1ef73.mp4
I have tried https://github.com/deathaxe/sublime-packages/tree/wip/php ans https://github.com/sublimehq/Packages/pull/2789 it and it keeps happening in safe mode, these are the commands I have used
git clone --single-branch --branch wip/php https://github.com/deathaxe/sublime-packages.git
sudo ln -sf `pwd`/Packages/PHP /opt/sublime_text/Packages/
and
git clone --single-branch --branch pr/php/change-base-scope https://github.com/FichteForks/Packages.git
sudo ln -sf `pwd`/Packages/PHP /opt/sublime_text/Packages/
One thing I have noticed is that with https://github.com/deathaxe/sublime-packages/tree/wip/php is that it freezes only when adding the closing parenthesis (As in the video), but with https://github.com/sublimehq/Packages/pull/2789 it freezes when adding and removing it.
This is the PHP file in which I am testing it, I have increased the size so that it is appreciated as much as possible.
Just noticed, SQL syntax is still pushed with_prototype in both suggested branches. I'll turn them into embeds, to see whether it improves things.
I can confirm ST freezing in this situation on Windows 10 as well.
It seems not related with embedded syntaxes such as SQL in your case though.
It just appears PHP is missing some bailouts from some kinds of incomplete expressions, which would/should otherwise limit the amount of document highlighting changes.
Removing closing parenthesis of groups causes the whole document to be reparsed as the whole syntax gets broken.
ST no longer feezes when adding the following bailout in contexts which represent parenthesized expressions.
- match: (?=;|\s*\?>)
pop: 1
But I guess there's more to consider.
ST no longer feezes when adding the following bailout in contexts which represent parenthesized expressions.
- match: (?=;|\s*\?>) pop: 1
Having no idea how this fixes my attachment (as the rule should never be visited) but yes, it does.
Tested with some different languages (C++, JS, Java, Python). It looks like only PHP has this issue.
The issue only happens when editing, it kinda takes forever long to re-highlight the document. But the document is opened quickly.
I've added a commit with several bailouts to my "wip/php" branch. It keeps existing syntax tests intact. Can't say anything about real world code though.
ST no longer freezes on incomplete expressions.
A syntax definition just needs to make sure to limit the amount of code being modified while editing. Otherwise ST can't perform well, if it needs to re-lex a whole document of 500k lines with each key stroke.
I have noticed that in my /opt/sublime_text/Packages/ instead of directories I have compressed files, like PHP.sublime-package.
The tests that I did above I did not do them correctly, what do I have to do to test correctly the changes? Thanks.
I've renamed the shipped PHP.sublime-package to disable it. That's required due to several structural changes with the mensioned branches/PRs.
Otherwise you'd just want to place the symlink to ~/.config/sublime-text/Packages, which is the place unpacked user packages are orgenized in.
Thanks, now it loads me from the PHP folder, but I have this error when opening Sublime Text Error loading syntax file "Packages/PHP/PHP.sublime-syntax": missing variable script_content_end at line 114 column 15 I have also tried PHP folder from https://github.com/sublimehq/Packages/pull/2789 and this error not appear, but it keeps freezing.
You may need to checkout HTML package from that branch as well.
This WIP branch uses several HTML related PRs, which have not yet been merged or released with ST.
Yes, with the HTML folder opens correctly.
I have tried it and it no longer freezes with parentheses, but it continues to do so with the quotes in some conditions.
For example from this:
$test = 'Test ' . () . 'text';
class tv_broadcasts
{
}
To this:
$test = 'Test ' . () . 'text';
class tv_broadcasts
{
'';
}
This also produces.
From this:
$test = 'Test ' . () . 'text';
function test(){
class tv_broadcasts
{
}
To this:
$test = 'Test ' . () . 'text';
function test(){}
class tv_broadcasts
{
}
This will need some work to refactor strings. Basic solution is to add a bailout to pop strings off stack at eol, but it needs some further work due to current implementation details.
Ugh. eol-bailout seems not applicatable as PHP supports multi-line strings. Not sure how to fix it than, as quoted strings may contain everything.
Pushed another commit to wip/php which refactors string contexts. It can't avoid the lag totally, but reduces it from about 16-19s to about 100-300ms.
It also causes the file to be loaded much faster.
FWIW, ST took those 16 seconds to be displayed when returning from another app, if the large file was modified on disk in the meanwhile, which has improved a lot now, too.
The syntax changes are certainly good, but ST shouldn't be re-lexing the whole file every time a change is made so this seems to also be a core issue.
I think I'm seeing similar "freezing" when editing Python or Bash scripts too and dealing with larger files / brackets. Could it be the same issue?
I think I'm seeing similar "freezing" when editing Python or Bash scripts too and dealing with larger files / brackets. Could it be the same issue?
Provide a reproducible file would be helpful.
Out of curiosity: What is expected to happen if someone adds a single quote at the beginning of a file? By doing so everything up to the next quotation mark in a file (if any) becomes a quoted string. Because the next quotation mark is now consumed as closing quote the following string is consumed as ordinary code up to the next quotation mark. ... Such buffer mutations effect the whole file. Not lexing them would mean symbols to probably get outdated once such a kind of huge change was intented or was made to fix a former broken syntax.
@deathaxe It only updates the visible tokens, lazily. I think it's applied to the whole file when you use Goto-symbol or similar.
seeing similar "freezing" when editing Python or Bash scripts too
Bash always pops most "statement" contexts at the end of a line if no line-continuation operator is present. So most incomplete changes wouldn't propagate to the whole document.
A syntax definition can't prevent that happening for statement like code blocks though. If closing brace is missing or added, this changes everything following as there is often no higher-level construct which can be used as bailout.
Also Bash doesn't (yet) make any use of branch_points which could trigger any back-tracking.
It uses embed to correctly drive backtick quoted command expansions. Those may have some impact if used often, but shouldn't be noticable in real world code.
Python code hasn't changed too much since ST3, so syntax definition can't/shoudn't be the culprit for any slowdowns.
Pushed another commit to wip/php which refactors string contexts. It can't avoid the lag totally, but reduces it from about 16-19s to about 100-300ms.
It also causes the file to be loaded much faster.
FWIW, ST took those 16 seconds to be displayed when returning from another app, if the large file was modified on disk in the meanwhile, which has improved a lot now, too.
I have tested the new changes and confirm that now it works much better, now the freezing when writing PHP code is acceptable and does not bother me when programming.
I think it's applied to the whole file when you use Goto-symbol or similar.
May it be caused by word completions?
I recently had a feeling ST freezes after closing a quoted string only if it is about to display completion panel.
I have an equivalent issue to what's described here but not when typing, but instead when tabbing back to Sublime from other app - it's frozen for a while, ~1-2 seconds before it actually shows the ST4 window. Could this be related?
Same when running in --safe-mode, with and without Vintage enabled and auto_complete set to false?
