rtv
rtv copied to clipboard
michael-lazar
Use markdown to format post and comment text
Some things like links can't be done, but the majority of the syntax used can be rendered fine on a terminal. Should probably use an existing markdown implementation and either add an output format that's easy for us to understand, or try to parse the HTML output that's given.
+1, I think that this is going to be difficult to implement though because we use curses for formatting, e.g.
stdscr.addstr("hello", curses.A_BOLD)
instead of plain escape characters
"\033[1mhello\033[0m"
I think a good way to do this would be to render the markdown to a list of chunks with their text and any needed formatting. Then do a loop like
for chunk, attr in chunks:
stdscr.addstr(chunk, attr)
Maybe make a function for the rendering to chunks where the chunk function is just given the markdown and screen width (And it handles the text wrapping as well, since doing it beforehand could screw with the formatting).
That way would be faster than doing it character by character, and for posts with no formatting it would be pretty much identical to the way we're currently doing it.
Have you looked into streaming Markdown parsers? Also there was this JS library that formated markdown in the terminal, you might want to look into that for inspiration.
There's also a streaming HTML parser right in the stdlib: html.parser. You can use that to parse the HTML gemerated by markdown library, and draw the text to the screen. :)
@eugene-eeo
Looks like that JS library is using direct ANSI escapes. That works fine for just printing some text out, but they do not play nice with curses, last time I checked.
Yes, the best thing to do would be to use a premade markdown parser and parse the output of that, I don't want to try to write a parser myself, that's just wasted effort and would be prone to bugs. I've used https://github.com/Anomareh/Hoep in the past and quite liked it, but really any library that is packaged (And supports all the features that reddit's markdown does) should work.
@5225225 I think we should also try the HTML solution. However if that doesn't suit us then I'm all for ~~making~~ stripping down markdown2 into a streaming parser.
What "features" of markdown can be supported, realistically? I have a branch in which I implemented bold text, but ncurses doesn't support italics/strikethrough. List support seems relatively trivial (for one line list elements, at least), and table support is realistic. The current roadblock is the way text rendering is handled is line-by-line, so any multi-line "effect" would have to be handled at the _draw_comment() method, which doesn't really handle rendering (which happens in the terminal class). Would it make sense to shove everything into _draw_comment()?
On Mon, Oct 17, 2016 at 04:01:30PM -0700, Reshef Elisha wrote:
What "features" of markdown can be supported, realistically? I have a branch in which I implemented bold text, but ncurses doesn't support italics/strikethrough. List support seems relatively trivial (for one line list elements, at least), and table support is realistic. The current roadblock is the way text rendering is handled is line-by-line, so any multi-line "effect" would have to be handled at the _draw_comment() method, which doesn't really handle rendering (which happens in the terminal class). Would it make sense to shove everything into _draw_comment()?
You could implement italics as an underline (Not many terminals support italics/strikethrough either, it's probably best to stick to more standard features. Underline should be supported pretty much everywhere).
Links can't really be done, you want to keep the full URL visible so it can be clicked on
terminals. Maybe the link format could be changed from [text](url) to something like text: url
Quoted text shouldn't be that hard, either use a vertical bar or just put > marks at the beginning of every line after text wrapping is done. Maybe an appropriate colour for the different levels.
Code blocks should be indented 4 lines or changed to a different text colour. You can't really make it monospace since everything is already monospace.
Strikethrough could maybe use a dim colour, like grey. Superscript might just need to be ignored and rendered normally, I can't think of any way to visually represent it on a terminal that still looks good and isn't complex. There's unicode characters for numbers like ² and ³, but that would be a special case.
I've got some WIP markdown code in my fork that might be useful (https://github.com/5225225/rtv/blob/markdown/mark.py). Doesn't have text wrapping or some features, but a fair few of the features work.
Looks interesting!
I see you have mistune handling the markdown, how does it deal with comments containing <em>text like this</em> will mistune add escape characters that then need to be handled in rendering or will it be indistinguishable from a mistune-parsed *text like this* and both show up as italics/underline?
I think writing a bare bones markdown parser that only handles the cases that can be rendered properly.
Reddit is nice and documented all of their implemented markdown features here: https://www.reddit.com/wiki/commenting so at least we know the scope, which doesn't seem huge.
On Tue, Oct 18, 2016 at 03:37:44PM -0700, Reshef Elisha wrote:
Looks interesting!
I see you have mistune handling the markdown, how does it deal with comments containing
? If a user posts a comment with <em>text like this</em>will mistune add escape characters that then need to be handled in rendering or will it be indistinguishable from a mistune-parsed*text like this*and both show up as italics/underline?
Neither, I think it generates escaped tags using > and <, which the python HTML parser will automatically both ignore when parsing and then convert to the angle brackets when needed.
I think writing a bare bones markdown parser that only handles the cases that can be rendered properly.
Possibly. I don't write parsers much, so I took the method of letting mistune (really any markdown parser that emits HTML will work) handle the parsing. Might be an idea to do it properly, but when I tried that there were a fair few edge cases that broke the formatting.
@michael-lazar I know this issue has been dormant for a while, but I use rtv semi-religiously, and I feel that using the mdv library would really help with our terminal-based Markdown formatting.
