notepadqq
notepadqq copied to clipboard
notepadqq
Preview of OpenTextEdit Integration
It's not done but I want to post it anyways. It's in a good enough shape that I started using it as my default editor.
A short overview:
- CodeMirror is gone, and with it QWebEngine and most of the async code we've had to build.
- I'm now using KSyntaxHighlighting instead of my own solution. As a result there are now modes for nearly 300 languages available.
Most of the usual editor functionality works. Apart from a good deal of bugs the biggest issues are:
- ~No code folding~ (fixed, but I expect many bugs)
- No multiple cursor selection
- ~No themes other than a default light and dark theme.~ (added two more themes)
- ~Opening larger documents (couple thousands loc+) is pretty slow since the highlighter still blocks the main thread.~ (fixed)
I listed bugs I found in the TODO.txt
The diffstat is pretty crazy
1692 files changed, 163923 insertions(+), 143376 deletions(-)
https://github.com/JuBan1/notepadqq/tree/ote
@Teklad You can give this a whirl now if you want.
The amount of outstanding issues I know of is fairly manageable by now:
- No multiple cursor selection.
- Some issues with Find & Replace (e.g. doesn't match \n yet).
- In Preferences, Line Height can't be set anymore (I don't know if this is feasible anymore).
But there must be dozens of bugs lurking around.
There were a few build errors when I first went to try this but it was mostly syntax/naming issues. There was also an undefined function "setTabWidth". After getting all this sorted I gave this a test run.
From the looks of it most of the base features we have typically used seem to be there. You even somehow managed to fix the slow syntax highlighting (or maybe it was fixed in a recent upstream release of ksyntaxhighlighting).
The lack of themes is a bit of an issue but from what I've seen implementing new themes doesn't seem like it would be too hard of a task. The language support is definitely there no matter how you look at it.
As it stands this is almost more usable than the current QWebEngine branch. At minimum the performance seems to be leaps and bounds better. The logistical issues regarding that branch are quite a hassle.
I'd personally prefer to move forward with focus being on this component in the long run. It would take a little extra time to get it up to speed but I believe the gains could definitely outweigh the loss.
Yes, sorry about the build issues! I've been working on it from two different computers, one of which doesn't have Qt installed. So I was hacking away at it in Notepad and used the git branch to upload my changes to test them at home. I should use a private branch for that..
I pushed a few new commits to the public branch. This includes fixing the syntax errors as well as patching a few other holes I found.
The syntax highlighting is still "slow" (~3000 lines/s on my machine), but I changed it so the Highlighter gives back control to Qt's event loop after highlighting each line. That way the GUI doesn't freeze while the document is gradually being highlighted.
Note: I rebased the ote branch so if you want to test, you'll probably have to do a force pull
Making good progress. The OTE branch now compiles fine into a snap in the default environment. So Qt 5.5 is a sufficient minimum requirement.
Nearly all features are now re-enabled (indentation detection and printing being the most recent additions) and countless bugs lie slain by the wayside.
So.. to address the elephant in the room: we're in a bit of a pickle right now. Most of our users are stuck on the older, buggy WebKit version because we can't bring WebEngine to them easily. Since this doesn't seem to change anytime soon (and WebEngine has its own fair share of issues) I suggest we transition to using OTE quickly once it's reached the necessary level of quality (which, I hope, will be done in a few days, plus necessary testing).
The one big ticket to reaching feature parity with the current version is multiple cursor/text selection. Because of its complexity I suggest we leave it to after the integration. I want to get this feature done, but moreover I want to get Nqq back to a stable, unified base.
What you say, @danieleds @Teklad ?
@JuBan1 I like how it's shaping.
About replacing it now, I don't want to rush: some features I particularly care about are indeed multiple cursors and Math mode, and I don't think removing them would be good, even if for just a few releases. Every big change inevitably comes with bugs that have to be fixed with time, so I'd like to let our codebase settle a little bit for some time... We've got quite a lot of features but we never stopped to concentrate on bugfixing (either WebEngine bugs or different kind of bugs).
Replacing the core editor with OTE now feels like starting over again... better wait some more time ;)
In my opinion the hasty swap to QWebEngine as a way to fix outstanding QWebKit bugs was pushed in a fairly hasty manner. Now we've got a version in master that will barely compile/run for most users. Our issue tracker inflated quite a bit without even getting to release state.
QWebEngine itself has a mountain of bugs as it is (1759 reported, and its still fairly new). I'm keen on losing the math feature for a couple of releases if it means fixing stability. Multiple cursor selection I can see, but the math feature doesn't strike me as vital (and honestly wouldn't hurt to be made as a plugin instead later on).
The master branch as it is already has no ETA whatsoever as to when we can even release an update that's even remotely up to quality due to QWebEngine having quite a few oddities. If we ever make it to release with it I foresee a new flood of bug reports due to a lot of upstream bugs between different Qt versions.
At the end of the day I would prefer stability over a temporary loss of a feature or two. The fact of the matter is most of the features will already be there. I don't really see it as starting over. The editor component itself isn't from scratch and most of the features you'd want in an editor are already covered for us, while the rest aren't particularly difficult to implement, though possibly time consuming.
some features I particularly care about are indeed multiple cursors and Math mode, and I don't think removing them would be good, even if for just a few releases.
I can understand that. Neither are features I use much, so I don't miss them. I hope it won't take long to implement them; mcs is quite a piece of work, although I know it's possible since Qt Creators has it too.
we never stopped to concentrate on bugfixing (either WebEngine bugs or different kind of bugs).
I'd like to mention that OTE by itself already fixes a ton of outstanding issues (#365, #415, #466, #497, #416, #576, #323, #658, #464, #653, #542, #690, #701, #405), and makes it easy to implement plenty of requests that would otherwise require us to mess with CM's code (#148, #306, #140, #324, #441, #444, #234, let's just assume I listed them all :).
Leaving that aside, plenty of users can't enjoy our bugfixing if we can't get snaps/Ubuntu 16.04 to work. We've got more people on 1.4.2 than on any other version put together.
Math mode
Any idea on how to graciously save it? I've only given it a quick glance and it appears to me as if it functions within a browser.js environment. That would mean it can't just run in a a Qml/JS container but needs Webkit/Webengine.
Best would be if it could render to an arbitrary surface. From there on I can blit the contents into OTE.
If push comes to shove I might be able to render it in an offscreen webengine window. But then it would have to be some kind of plugin or we'd lose the advantage of being able to ship without WebEngine again..
@JuBan1
About Math mode, someone suggests this, but I have no idea about its dependencies.
In general, it would be awesome being able to render it off-screen and then use some "generic OTE api" to embed it within the correct line/columns inside the editor. We could use this for potentially a lot of things (off the top of my head, git information and stuff like that)
I found a handy library for Qt that can render to pixmaps without external dependencies.
I tried to replicate the current behavior, but it turns out manipulating the height of individual lines is really difficult to do. So I settled for looking for free space beneath the formula line to insert the image. It's not as nice as before but it's the best trade-off between usability and ease-of-development.
I think there's a function which gets the current text height and such... Should be really useful in this scenario. I can't remember the exact name of the function though.
Getting line or block height isn't the issue, but setting it is. You can see in the original Mathjax issue thread how the line height grows (a little) to accomodate the larger math formula.
At first I tried replicating that but QPlainTextEdit doesn't really support setting the height of individual lines/blocks. Infuriatingly, changing that wouldn't be much of an issue but that piece of code is buried deep inside Qt5's source so I'd have to copy/paste at least half a dozen Qt classes into OTE and maintain them myself. And that's not gonna happen.
Then I played around with inserting artificial line-breaks (like word-wrapping does) but ultimately that led to no solution I was satisfied with.
So I ended up doing what you saw in that webm.
I'm not dead yet! And OTE isn't dead yet either. I've been putting more work into an API to project arbitrary labels into the editor. That can be used for Math labels and other shenanigans.
That hex color clickable text is just downright fancy. I don't think I've seen that feature before on a default editor (and I've used a few). :p
Okay, the labels stuff is in a pretty good spot now and I've pushed a rebase+update to https://github.com/JuBan1/notepadqq/tree/ote. For testing purposes LaTex- and Colorlabels are always enabled at the moment.
@Teklad @danieleds Would you mind testing this again? Perhaps even installing it instead of your current Nqq install for a while? It'd be great to get some input on whether your day-to-day activities are negatively impacted by bugs/changes in functionality/etc.
That is, unless you can't live with multiple-cursor selection. That's the one big ticket still missing.
@JuBan1 how's the status on this? Sorry it's been a long time since I've checked in I got consumed by some other projects. I'm excited to see this finally implemented... our bug list is piling up due to the stagnant state of the project.
My last comment tells most of the story. I've got some rudimentary code for multi-cursor selection now but nothing presentable yet. It's a fairly tricky feature because of its tight coupling with existing code for cursor and event handling.
If you want to lend a hand, my nqq-ote branch is ready and waiting :)
Good news! I'll finally have some free time this weekend to help with this. Feature requests and bug reports are piling up insanely high. We need to put a lot of focus on this before the entire application explodes, lol.
Multi-cursor selection is now in https://github.com/JuBan1/notepadqq/tree/ote branch, though rough edges remain.
It doesn't work exactly like what we currently have since I basically made up the rules as I went along. That leaves us to decide how exactly mcs ought to work. Here's a quick rundown of how it works in master and how it works in ote:
Master:
- While holding down CTRL you can click to add new cursors. By click-dragging you can add new selections too.
- There's no way (that I can see) to add new cursors via keyboard.
- Using Alt+Shift you can "block-select" text.
OTE:
- While holding down Alt+Shift you can click to add new cursors. Click-dragging doesn't do anything.
- While holding down Alt+Shift you can use Up/Down keys to add new cursors (see the gif above)
- There's no block-selection using mouse, but it's easily done via keyboard.
Other things are the same, such as:
- Using arrows keys moves all cursors at the same time.
- Escape exits out of mcs.
- Shift+arrow keys expands selection for all cursors at the same time.
- Copy/paste tries to paste one line from the clipboard into each cursor (provided the number of lines equals the number of cursors)
Care to give me a quick rundown on what's left to implement? I'll try to put in some work for this throughout the next week or so. Feel free to include any bugs that might need fixing as well.
The biggest help would be to just try the branch and find bugs. I try to fix them as I find them but Nqq just has such much functionality that could bug out by some change or another.
Non-bugs left to consider:
- Few themes available and I think some of them have badly chosen colors (solarized).
- Can't turn on/off math rendering or the hex-color labels yet.
- A few functions inside MainWindow are still stubs (e.g.
on_actionSpace_to_TAB_All_triggered) - Probably more, but that's what I can think of right now.
The list of bugs changes all the time but here's what I've got on my todo-list at the moment:
== Miscellaneous ==
- Left-facing selections cause constant redraws
== Multi-Cursor Selection ==
- Selections of multiple cursors shouldn't be able to overlap.
== Find & Replace ==
- Find might behave strangely for special chars (\t, \n, ..).
== Labels ==
- LatexLabels should only show up in comments
- Labels sometimes don't get painted when the line it's anchored to is moved downwards.
@Teklad If you'd like you can give Extensions a try too. I don't know if the changes break anything with what support we have for extensions and I don't want to break @qcerasmus 's stuff :) But I can't really run extensions on my system. Just doesn't want to work.
I have yet to get extensions to work myself. The process of getting it going is rather convoluted, which is why we don't ship any extensions with NQQ in the first place, lol.
Work's progressing nicely and there are only a handful of items left on the agenda (sans whatever bug testing drags out of the woods). There are a few more organizational issues I wanted to ask you input on, @danieleds , if you've got the time.
- First of all, I hope you're actually amiable towards merging this in the not too distant future. Nqq has had three months to settle since the WebEngine change, and OTE is now practically feature-matched with it. The size of the changes will make proofreading very impractical unfortunately but OTE's in a very good place after hammering away at it for the past week or two.
- Right now, OTE is included as a sub-project (with its own .pri file) and its own namespace. We can keep it like that or fully pull it into Nqq. I'm for keeping it that way but it somewhat fractures the whole project a little.
- OTE is using source code from two other libraries. One LGPL and one MIT licensed. As I understand it they don't conflict with our GPL license at all. I'm not entirely sure on the specifics of what we have to do to adhere to any rules. Right now I only mention their use in Nqq's About-dialog.
Also two more less organizational things:
- We ought to do a version bump. Off we go to v2.0?
- The LaTeX "Math Mode" as well as bracket matching and finding hex color labels are implemented using a little Plugin-system, and we can bring more in the future (like a spell checker). So instead of having a menu action to turn them on/off (like currenly with Math Mode) I thought we could add a new page to the Settings dialog that lists all Plugins and gives users the ability to turn them on or off. Initially that would just be a bunch of checkboxes to make things quick. Later we can do more awesome stuff. However, that would remove the ability to turn them off/on for individual editors. (At least for now.)
I'm looking forward to bring this to users. The benefits are great.
@JuBan1 WebEngine has settled but not in the hands of the users... yet :)
The main issue I see for merging OTE is actually still the same I expressed months ago: my concern is that the editor component should be quickly "hackable".
What I mean with that is that, for example, it took me something like an afternoon to implement math mode. Doing the same here obviously required a bit more time. If I wanted to add the "bookmarks" feature (just as an example), I could go in the codemirror directory, import some js, and then write a couple interfaces in c++. Here, I'd have to code it myself from scratch.
That's of course natural... CM is a project with 600+ contributors, and OTE is brand new. We couldn't expect anything different. But this also means that CM has an alive and independent community that improves the editor in parallel to what we're doing. And they're doing quite a lot of work for us. I don't want to lose that... I see a lot of value in that, something that we couldn't possibly match.
That said, yes, I agree, having to deal with web engine sucks a lot for a lot of reasons. However:
- It is the path that most are following (e.g. vscode, atom). I'm not saying that we should do it because they're doing it (actually, I don't like their approach, as I think a native UI is way superior). What I'm saying is that there are quite good reasons to do it.
- It's not that it completely sucks. I mean, it worked fine for all of our increasing user base since we left Scintilla.
So there are pros and cons to the switch. I talked about the cons, but the pros are evident: I like the fact that it is way more lightweight, it doesn't require a whole web browser to work, and it is architecturally simple. But I have to evaluate the balance between the two and the risk I fear is stagnation down the road caused by the fact that we won't have the resources to do what CM is doing in parallel. This could change if somehow we grow and get a lot more regular contributors, as OTE proved to be definitely valid.
TL;DR: OTE is way cooler than CodeMirror but I don't want to risk project stagnation because we won't have the resources to do what CM is doing in parallel for us.
Bummer. I've been developing OTE with the primary prospect of ultimately having it merged into Nqq. (Though I don't complain being able to use it privately).
Thinking back, was there really all that much that we gained from having a hackable editor? I'm looking at the commit history of the CM folder and in the past two years there's been "math mode", "Font Selection" and "Show tabs/spaces/line symbols". Other than that it's just bug fixes or workarounds for iffy behavior, and apart from math mode those features were easy to code in C++ too. Sure, having the potential to add practically anything made in JS is great, but we've made little use of it in the past. (That bookmarks feature was requested two years ago. So "easy to implement" doesn't translate into "will be implementeed" either.)
(Edit: Of course there's the benefit of bugfixes/etc from CM's continued development that we get by rebasing our fork. But in that respect OTE isn't entirely alone either. It's based on KSyntaxHighlighting which is developed by KDE.)
What I'm worried about is that the stagnation is already creeping up on us. The list of open issues becomes longer, nearly all PRs are authored by us (except for l10n), and if our stats are to be trusted Nqq is on a slow downwards trend (although I'm not sure if the stats are at all reliable for that). IMO the reason is that we have had v1.4.8 for more than four months now, yet most of our users are still on 1.4.2 because we can't even deliver it to them properly. We racked up near 200 commits after v1.4.8 that are just sitting there being useful no nobody. I'm sure it's as frustrating to you as to me.
Can you think of anything to do that would make OTE more appealing? Obviously I can keep adding stuff to it (like bookmarks) but that doesn't do much to address the root of your concerns. And while I don't intend on jumping ship anytime soon I can't guarantee anything of course.
We racked up near 200 commits after v1.4.8 that are just sitting there being useful no nobody. I'm sure it's as frustrating to you as to me.
This long delay was actually planned after the big switch to WebEngine. Actually, v2.0 should be out as soon as I clear this list: https://github.com/notepadqq/notepadqq/milestone/3 (#405 is an upstream bug and won't be fixed by us, it's there just as a reminder). The plan is to release it asap and after that we're probably going to merge the cmake work from @Teklad
Can you think of anything to do that would make OTE more appealing? Obviously I can keep adding stuff to it (like bookmarks) but that doesn't do much to address the root of your concerns.
Appealing in the short term? Probably not... I know that it's still young, but it's not mature enough yet. I'm actually using it, but a couple of bugs* showed up in just the first few minutes... they can be fixed, of course, but this already gives us an approximation of the expected stability. Don't take this bad: the work itself is remarkable (and man if it's smooth!)
Appealing in the long term? Well, that would be maturity. Maturity not just evaluated on features and bugs fixed, but rather in the sense of a piece of software that is regularly maintained and proves itself valid also outside notepadqq. That would be it.
* I'll write them in the next reply
Some of the bugs I encountered:
- I'm not able to move text around with the mouse (e.g. select something, click on it and then drag somewhere else within the editor)
- if I doubleclick on the blue thing in the screenshot, nqq crashes
- whenever I'm on a mac I can't input curly braces (Italian keyboard)