idle-reimagined
idle-reimagined copied to clipboard
asweigart
Using Pygments for Syntax Highlighting
Syntax highlighting using Pygments http://pygments.org/ (which is used in docs created from Sphinx) can provide a fresh look and it is something that we could consider for Idle-Reimagined.
We might not be able to include Pygments bundled with Python, though it will be a good place to look at for ideas. I'm still reviewing the current IDLE code base (since IDLE already has syntax highlighting). Are there any features that Pygments has that you'd like to see added to IDLE's current syntax highlighting?
One of the compelling reasons I suggested Pygments as a choice is because of the consistency observed in the syntax highlighting. The official Python Documentation, Requests, Flask and many more packages make use of Sphinx for their documentation and Sphinx makes use of Pygments, as if it was a standard for highlighting python code. Hence the aforementioned uniformity in code highlighting which might work wonders for visual learners. This made me realize, If the official Python Documentation and many other packages and frameworks makes use of a particular highlighting scheme, why not have the same scheme for the interpreter that beginner Python programmers would be using or switching between Python Docs and IDLE-Reimagined. Since color schemes are pretty subjective in nature, this brings to another feature of Pygments that you don't need to stick to a particular scheme. You can very well change it. Although IDLE is capable of having tweaks in the highlighting, it is more on the lines of changing the color of Python keywords, stdout etc. Having styles for highlighting could extend IDLE-Reimagined in the aesthetics department. Just like here. http://pygments.org/demo/1038115/ Is there a technical constraint that prevents us from including Pygments in the bundle? If yes, then we could make change in the existing codebase and probably mimic the highlighting style observed in Python Docs / Requests / Flask etc. Thanks for responding. IDLE-Reimagined is a much needed Initiative and I am happy to help in all the ways I can. :)
(tl;dr: adding Pygments as a required module may be too daunting a task; having it as an optionally imported module via a plugin might be more easily achievable.)
kevinaloys asked "Is there a technical constraint that prevents us from including Pygments in the bundle?". As asweigart made a reference too in perhaps a too indirect way, it would be very difficult. The idea behind idle-reimagined is to improved upon the existing IDLE included with the Python standard library and to replace it. While Pygments is extremely well known in the Python community and used to create the official Python Documentation as kevinaloys pointed out, Pygments is nonetheless NOT included in the standard library. For better of for worse, there are a number of conditions that have to be met before a new package is included in the standard library. One of these constraints (but not the only one) is that the maintainer of a given package (or an existing Python core developer) has to agree to actively maintain the package for the foreseeable future and ensure that it be updated in sync with new Python releases. Simply tackling the issues of 1) making idle-reimagined be better than the existing IDLE, 2) ensuring that all demos/modules/unit tests in the Python standard library are adapted to use idle-reimagined instead of IDLE, 3) convincing the Python core developers that idle-reimagined (with proposed changes) is a desired improvement is going to constitute a huge task. Attempting to include Pygments in the standard library at the same time would only increase the difficulty of completing the project.
There is however another possibility: perhaps idle-reimagined could be made extensible so that one could write a plugin that could make use of Pygments if it is available. This would not require having Pygments included in the Python standard library. It would be especially nice to have Pygments usable, especially if we can generate "help" documentation that would use the same style as the official Python documentation.
I agree that including the whole Pygments module in IR or as part of the standard library may be too much to ask. But I like kevinaloys' idea of having a consistent highlighting scheme between the editor and code that a user would find at sites like readthedocs, stackoverflow, or github. It may be worthwhile to rework IDLE's default highlighter to be more in line with those sites and maybe have an option to use Pygments if it's installed.
Also, just to note, IDLE already has Highlighting Themes in the Configure IDLE window. I'd like to see duplicates of the common themes (such as those used by Sublime Text: monokai, etc).
Aside from color scheme, can anyone name differences between the way pygments highlights code and IDLE currently does? For example, the string '%(spam)03d Hello' has the %(spam)03d part highlighted differently than the rest of the string. IDLE doesn't do that. That's one difference.
Though I'd argue that these differences (if there are any more) aren't significant enough to spend time re-doing IDLE's syntax highlighting code. I advocate being conservative in scope (at least until we have a 1.0 out).
-Al
On Sun, Dec 14, 2014 at 12:48 PM, akehrer [email protected] wrote:
I agree that including the whole Pygments module in IR or as part of the standard library may be too much to ask. But I like kevinaloys' idea of having a consistent highlighting scheme between the editor and code that a user would find at sites like readthedocs, stackoverflow, or github. It may be worthwhile to rework IDLE's default highlighter to be more in line with those sites and maybe have an option to use Pygments if it's installed.
— Reply to this email directly or view it on GitHub https://github.com/asweigart/idle-reimagined/issues/5#issuecomment-66928942 .
I was interested in how to implement this. So i hacked this: https://gist.github.com/jedie/528bee3fb86b56330e7c
IMHO it's not really effective: Because it always parse the complete text and highlight it again. But IMHO this can't be done in a other way, e.g.: block-comments in more than one line... Also boring to call "tag_remove" for all existing tags before use "tag_add"...
The currently IDLE Highlighting code is very complicated IMHO.
