.tmLanguage & VS Code Support

[MattMorgis]

This is more to start a community discussion, but support for VS Code is lacking. There is one plugin out there, but it's just a fork of TextMate support for EJS.

It looks like VS Code needs a .tmLanguage file to offer support for a language. This a grammar format TextMate uses to identify keywords, comments, control flow, etc. when rendering an .ejs file. The documentation seemed vague to me. This is why the above and only EJS extension forked the TextMate .tmLanguage file.

I have had issues with EJS support for a while. I moved from Sublime Text to Atom because of the EJS support breaking in Sublime. The best support I've seen for EJS is in WebStorm via a Java plugin from JetBrains.

I've found a list of various .tmLanguage files for EJS and tried them all in VS Code. They all have various features that kind of work. I am proposing we build one officially supported .tmLanguage file that can be shared by Sublime, VS Code, TextMate, and any other editor that supports this format (I think Atom and Webstorm also do).

I'm not sure if this is something best suited for the official EJS language repo or organization or done as a community project. I'm willing to help out and even do a bulk of the work, it's when I got into the guts of it that things got tricky and I needed some help and brainstorming. Hence me reaching out to the community to see if there is already something in works, interest in getting this off the ground, etc.

List of EJS.tmLanguage files:

• @samholmes - most elaborate I've found, used by and currently broken in Sublime
• tmBundles - repo of text mate language bundles
• @gregory-m - current VS Code extension built off of this and doesn't provide much support

Please let me know thoughts, next steps, or anything I'm missing or overlooking.

In my opinion, these would be 'must haves':

• File include's
• Comments
• JS Control Flow (if/else/loops) between closing %> and opening <%
• HTML in-line values. Ex. <img src="<%= ejs %>">

Current Screenshot of EJS in VS Code:

Current Screenshot of EJS in WebStorm:

[RyanZim]

This should be community-supported IMO. We could consider linking to a community project from the README perhaps.

[mde]

Community-supported would be ideal -- it seems like a non-trivial project unto itself. We're definitely happy to help and provide support however possible.

[ezekg]

Can we just fork the .erb package and replace the Ruby-specific matching with JS? The .erb language seems to look great for .ejs files as-is, honestly. I have the following in my VS Code preferences:

// Place your settings in this file to overwrite the default settings
{
    "files.associations": {
        "*.ejs": "erb"
    }
}

[mde]

Interesting idea! Would you like to take a crack at an adapted-for-JS fork?

[ezekg]

I gave it a try at vscode-ejs, but I can't seem to get partial blocks (e.g. if (foo) {) and matching parentheses to stay within the <% and %> patterns—it seems that you can't limit the scope of embedded languages so I've given up for now. The erb hack at least gives me decent syntax highlighting, albeit not JavaScript, but it's better than the current ejs package.

Edit: what we'll probably need to do is partially reimplement the JS grammar within the EJS grammar, fixing any multiline expectations when between EJS tags. Right now, it seems that those expectations are what's causing issues with the grammar e.g.

<% if (true) { /* The vscode JS grammar takes over here and expects a closing bracket */ %>
  <h1>Test</h1>
<% } %>

The same issue is apparent something like this as well,

<% [1, 2, 3].map(num => { /* The vscode JS grammar takes over here and expects a closing bracket AND a closing parenthesis */ %>
  <span><%= num %></span>
<% }) %>

[mde]

Of course, it's two (or is it three?) interleaved grammars. Seems like a really interesting problem to solve. :)

[MattMorgis]

@ezekg That is the exact point that I created this issue! I had a similar thought to yours where I wanted to dig up Handlebars support and see if it handles this in any way that we could fork or re-implement, but it's still on my to-do list unfortunately.

I wonder if there are any template engines who have done something similar to what you've mentioned already? I honestly don't even know what's out there these days - a quick Google brought up this list which I can try to look into.

I honestly feel like we're this close.

[markovicdenis]

Right now this extension (EJS language definition) is utterly useless. Wrong syntax highlighting when you use IF statements and whats worse, it doesnt mark closing tags in the html.

[Betanu701]

All,

I have created a new EJS Language definition. If you want to test it out and let me know if you have any issues. https://github.com/Digitalbrainstem/ejs-grammar

For now, if you open VSCode installation directory, find the other languages and create a new folder called "ejs" and place the code from my repo in there. I will be working on getting it integrated into VSCode. But you should now have access to select "Javascript EJS" from the languages.

I tried to get as much as I could in it. Let me know if I should add anything.

[Betanu701]

I have made this an actual extension in VSCode, Search ejs, the name is "Javascript EJS" Let me know if there are issues.

[MattMorgis]

@Betanu701 👏 👍 😄

I'll install this and check it out a bit later! Very excited!

[ezekg]

You're awesome, @Betanu701. Looks great for me. 👏

[mde]

This is awesome. :) Would be great to get this added to the README and Web site.

[tbergeron]

Any link to the extension? The only ones I found are:

• https://marketplace.visualstudio.com/items?itemName=DigitalBrainstem.javascript-ejs-support
• https://marketplace.visualstudio.com/items?itemName=DigitalBrainstem.javascript-ejs-support
• https://marketplace.visualstudio.com/items?itemName=QassimFarid.ejs-language-support

And none of them highlights my code properly: https://cl.ly/0f290P110G2m

[Betanu701]

@tbergeron

There are several issues you are encountering. The first one, is that you are using multiple lines within tags. Technically this does render, but as far as I am aware, EJS is not meant for this. It is meant for single lines. For instance

<% for(i=0;i<10;i++) { %> <% if(i % 2 == 0) console.log('i is even') %> <% else console.log('i is odd') %> <% } %> @ezekg correct me if I am wrong.

What is happening with the parser which is causing your issues is the language sees the open braket and wants to interpret it as javascript. From then on, it is using javascript and there is not anything built in to see the stopping delimiter. We will have to write an entire javascript parser to see the ending delimiter and break javascript to signal the end of ejs.

tmlanguage servers do not see multiple lines. They only read 1 line at a time. This is what makes what you are wanting very difficult. I would need to essentially hold the lines after an open tag in memory until I see the closing tag. In theory this seems like a basic practice, but the parser itself is not powerful enough (yet) to do that. This requires significant work with an actual language server to make this possible.

There is another contributor that is working on the language server for ejs. I am not sure how through it is, and it is not ready for prod, but you are welcome to test it.

The other issue you are encountering, is ejs tags within strings of html. This is a known issue, but again not easy to solve, as it requires writing an entire html parser as well to find anything within html tags and to find <%%>. HTML natively takes the string and hides it. If you know of a way to parser this out without the language server you are welcome to edit the language files and submit a pull request. I don't know of a way without a server though. I spent many many hours trying to solve that.

[Betanu701]

@tbergeron NOTE: examples are spaced out for clarification, otherwise it the parser on here is placing everything on 1 line ...

The issue with the old ejs server, was that on any given line if you had an open bracket, it would automatically have javascript take over and you could not get out of javascript parser. The new language file will actually stop javascript parsing at the end of a line if there is a closing ejs tag.

<% if(foo<bar) { %> that final tag would not be seen by ejs. The new one does see it and forces javascript parsing to stop.

The only issue really is the open brackets with multiline things. If you don't have multiline brackets, you can use 1 single pair of ejs tags.

This will work:

` <% for(foo=0;foo<bar;foo++) { %> '

<%

if(foo % 2 == 0) { console.log("foo is even"); }

else { console.log("foo is odd"); }

} %>

Hope this gives further clarity.

Edit:

The other option you have is putting the open bracket for the control statements on the next line. Example:

` <% for(foo=0;foo<bar;foo++) %>

<% {

if(foo % 2 == 0) { console.log("foo is even"); }

else { console.log("foo is odd"); }

} %>

[tbergeron]

I completely get what is happening now. Thanks for such a detail response! I've recently switched to VSCode and I'm looking more and more about extending it and learning how to develop extensions etc. Your explanation really helped wrapping my head around the whole language server concept (this also helped a lot https://code.visualstudio.com/docs/extensions/example-language-server)

My EJS usage might be quite funky in the provided example, but in my humble opinion, I believe the parser shouldn't expect a proper syntax but act on whatever you throw at it. Nobody uses a syntax in a lint-esque manner all the time, so this shouldn't actually break the rendering (though I understand why this is happening at the moment hehehe) I'm not ranting at all, what's been done so far is already fantastic!

I see you seem to be developing the update on the dev branch? (on this repo here: https://github.com/Digitalbrainstem/ejs-grammar/tree/dev) I'll need to figure out how to use extensions coming right off a GitHub repo but I'll sure try to help out if I can! It seems possible to fix these issues, but I totally understand it's pretty complex to accomplish! Thanks again!

[bombillazo]

This is desperately needed for ejs...