[FIX] Support complete markdown specification

[gromain]

@RocketChat/core

Closes RocketChat/feature-requests#642 Support full Markdown specification Also solves: RocketChat/Rocket.Chat#7240: Special characters escape with \ (_ ` etc...) RocketChat/Rocket.Chat#6719 RocketChat/Rocket.Chat#6586: Markdown link with & sign is broken RocketChat/Rocket.Chat#5218: Italicizing links breaks them RocketChat/Rocket.Chat#4346: Support for markdown tables

Also, it may simplify work on RocketChat/Rocket.Chat#1593: support for better code highlight

This PR adds full Markdown support through markdown-it. It also supports sub- and super-scripts and checkboxes (via markdown-it plugins). I believe some cleanup is needed in the CSS file(this one has been moved out of the base.css file into its own markdown.css. The class has been switched from .body to .markdown-body in message.html).

A couple of things are broken right now:

• >>> to start a blockquote
• Strikethrough with single ~ (works with ~~)
• Bold with single * (works with ** or __) How do we deal with those style changes (bold and strikethrough and blockquote)?
• Code highlighter could use a pass on the style used I think, some languages does not display well (python for example)

Screenshots

• Tables
• Ordered and unordered lists
• Checkboxes!
• Sub-scripts and super-scripts
• Typographic replacement
• Headings and horizontal rules
• Code with highlighter
• Nested blockquotes

[gromain]

I'm not sure the build failure has anything to do with my changes. The failure happens during a call to sideNav.getChannelFromSpotlight('rocket.cat').waitForVisible(5000); in 09-channel.js.

What can I do to fix that?

[gromain]

I pulled and merged the changes from Rocket.Chat/develop, I hope it will help with the build failure.

[gromain]

The build failure is now related to badwords management. This PR add supports for horizontal rules via ___, --- or ***. Bad words are replaced by *, thus creating a horizontal rule. I will change the replacement to escape the stars. We could also change the character to x.

[gromain]

Mmmmmm, for some reason, it worked on my local tests, but not when deployed. Oh well, x it is then.

[gromain]

Hello @sampaiodiego @rodrigok and @MartinSchoeler,

Is there anything needed from my side?

[JSzaszvari]

Really hoping this change makes it in. Table support would be amazing

On Aug 9, 2017 10:50 PM, "Romain Bazile" [email protected] wrote:

Hello @sampaiodiego https://github.com/sampaiodiego @rodrigok https://github.com/rodrigok and @MartinSchoeler https://github.com/martinschoeler,

Is there anything needed from my side?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/RocketChat/Rocket.Chat/pull/7454#issuecomment-321245599, or mute the thread https://github.com/notifications/unsubscribe-auth/AA1QDwF-p9UY4ZIhEzDAgqO8yCeTIeWuks5sWasHgaJpZM4OSDiD .

[gromain]

@jszaszvari Agreed! :-D

[JSzaszvari]

Keen to find out why this one has not been looked at? This would be a super helpful, especially the tables and lists

@geekgonecrazy @graywolf336 any chance you can let @gromain know what needs to be done if anything to move forward with this?

Thanks

[JSzaszvari]

@engelgabriel ??

[Lemmmy]

Are you going to add support for nested blockquotes? E.g:

hello

world

foo

[Lemmmy]

Also - slack added a simple check to disable blockquoting with emoticons like >.> and >:( - would you be able to implement something similar?

[gromain]

@Lemmmy I forgot the nested blockquote support! But yes, it's there:

Regarding the emoticons, I'll have to have a look how to deal with this. It doesn't sound impossible though. It works for now if you escape the first > with \ though (but I agree, this is not practical at all!).

[geekgonecrazy]

@RocketChat/core can we try and get this lined up for 0.60.0 ? Better markdown support would be great to have

[vcapretz]

hey @gromain @lindoelio I fixed the conflicts in https://github.com/gromain/Rocket.Chat/pull/1, and after gromain accepts it should work well so we can merge it here to be released at 0.60.0

[gromain]

@lindoelio @vcapretz Thanks for the help and sorry for the delay, I was away for the weekend! I merged the PR, it's building now on the CI. If the build fail I'll check locally what's up with it.

[gromain]

It failed, I'll have a look at the details.

[vcapretz]

hey @gromain, they added the Marked support so now we can choose between the original one and the marked parser, so your changes should be applied here in the file rocketchat-markdown/parser/original/markdown.js.

I tried to do that in https://github.com/vcapretz/Rocket.Chat/commit/26a559df46ef7e4d467635a82e2c90ec3e85b9f1 but as you are the one who did this in first place would be great if you could check it and see if it's everything ok!

[gromain]

Hi,

If the support for marked was added, I'm not sure there is a point in adding my changes in the original parser. I modified it by using the markdown-it library (mainly because it has supports through plugins for sub-sup-scripts and checkboxes) and removing unnecessary parts. If it's now possible to chose to use marked instead of the original parser, I'm afraid most of those changes I did are not needed anymore.

[gromain]

I'm not sure when the support for marked was added, it looks like it's old, but I couldn't find any reference to it back when I forked the project...

[vcapretz]

@gromain I found this PR: https://github.com/RocketChat/Rocket.Chat/pull/7852

if you are an admin on a RC server you can choose between original and marked parsers, I don't exactly know the differences but if you feel like marked support's everything needed this PR may not be necessary :disappointed:

[vcapretz]

@rodrigok any thoughts on this? Do you think we still need to change something on the original parser?

[gromain]

I guess we can close this PR now.

To me it doesn't make sense since there is already work that was done in RocketChat/Rocket.Chat#7852 (and RocketChat/Rocket.Chat#6158).

It would be nice if the coordination on this project was more open and more communicative. I would have like to know when I asked in RocketChat/feature-requests#642 what were the thoughts of the community on the integration of Markdown. I'm glad it's finally finding its way in Rocket.Chat, but in my humble opinion, the whole project would greatly benefit from a better stewardship and openness towards exterior contribution. This experience only leaves me frustrated (and not because my changes don't make it through to master). The process is obscure at best. It would have been better if from the onset someone said "Hey, we are already considering pulling this PR, no need to sweat on it, but have a look if it can be improved". This would have been much better than just the screams of silence.

Anyway, I'll close this PR in a week if there is no further interest.

[JSzaszvari]

agreed. hoping that someone in @RocketChat/core has something to say considering this PR was opened a long time before the other markdown stuff was done and merged.

someone could have at least told the poor guy working on this one not to bother if there was plans that weren't made public.

instead multiple calls for a Comment went completely ignored..

[gromain]

Rocket.Chat is a great messaging app, but really, it would benefit greatly for more openness from the devs and a more open communication.

Closing this PR now.

[geekgonecrazy]

Sorry guys. I'd also love to improve our markdown support. Being able to add support for additional markdown would be amazing.

I don't think another option has been chosen. We are constantly fighting to keep xss vulnerability from popping up. Anything that involves HTML in the message in any form runs a risk of being exploited.

Also we just finished releasing 0.59.0 and we are just now looking at PR's to merge into 0.60.0

With a lot of UI changes in 0.59.0 it took a lot of our time.

@rocketchat/core

[gromain]

@geekgonecrazy let me know if there is any interest and feedback, I will reopen it.

[rodrigok]

@gromain Sorry about that, we was busy with our latest release and we delayed this review.

Please reopen it and I will start the review.

First question, did you change how bold and italic are handled?

[DeanVanGreunen]

Hey guys/girls, etc, etc.

having a small issue with the Chat app, desktop (just a wrapper app...) and web versions.

When typing the following: Any Message which is then Wrapped in Code, StrikeThrough, Italics and bold as below real-world example/preview which is also marked as a multi-line.

when sending/posting it in the chat, it will display as follows. (note: only name and date/time is marked out, due to company specs, hosted locally on internal servers, etc, etc..)

Both these images are of the exact same "data"/"text"

please explain what this is for and/or if it is meant to be avalible as a feature, is a bug, or if it is encryption of some sorts, what is this data and how do I undo it??? or disable this if it is a bug..

This problem can be repeated and will always reproduce the same "output/display data" if the same "input/typed data" is entered again.

Possible Feature Suggestion: Inline encryption, would be cool as well for say a group, channel or user2user chat...

I love the app by the way and would like to contribute / get involved as well.

Thank You, Dean :)

Initial Issue Report: https://stackoverflow.com/questions/48930326/unkown-data-in-chat-please-explain-rocket-chat-only/

[didierhk]

@gromain @rodrigok is this still in the works ?

[gromain]

Hi everyone, After a break related to work, I'm planning on starting to work again on this. Is there anyone already working on this, @didierhk @rodrigok ? Anything I should take into account?