Format of embedded languages

[WhyNotHugo]

trafficstars

Vue files can have a script element with either type="javascript" (the default) or type="typescript". It also support scripts in css, scss, less, sass and a few others.

Would it be possible for the script_element and style_element to somehow indicate the language of this embedded block? Or is that not something this parser can do?

[ikatyang]

Would it be possible for the script_element and style_element to somehow indicate the language of this embedded block? Or is that not something this parser can do?

I guess you meant to add ts_script_element and similar, it's possible but it does not seem to be a good idea since there are countless possibilities for the lang value. I think it'd be better to leverage the query feature to achieve the goal, for example:

Code

<script lang="a">aaa</script>
<script lang="b">bbb</script>

Query

(script_element
  (start_tag
    (attribute
      ((attribute_name) @lang_key (#match? @lang_key "lang"))
      ((quoted_attribute_value) @lang_value (#match? @lang_value "a"))
     )
   )
   (raw_text) @a
 )
 
 (script_element
  (start_tag
    (attribute
      ((attribute_name) @lang_key (#match? @lang_key "lang"))
      ((quoted_attribute_value) @lang_value (#match? @lang_value "b"))
     )
   )
   (raw_text) @b
 )

Result (from the playground)

[artem-schander]

Hello, I hope it's okay if I join in :) I'm trying to understand how this exciting stuff works .. and maybe I'll contribute somehow when I'm comfortable.

My main goal is to get Pug templates highlighted (at least a bit) but I can not get the individual parts matched. To be honest, I did not read the Tree-Sitter docs yet, but I definitely will. However, maybe someone could point me the right direction. Here is what I got:

Code

<template lang="pug">
  .foo(:bar="baz")
  	#test Lorem Ipsum
</template>

Query

(template_element
	(start_tag
    	(attribute
    		((attribute_name) @lang_key (#match? @lang_key "lang"))
        	((quoted_attribute_value) @lang_value (#match? @lang_value "pug"))
        )
    )
    ((text) @class_selector (#match? @class_selector "\."))
)

[artem-schander]

Hello there

I've noticed some strange behaviour. If I change the filetype of a vue file to pug :setf pug the highlighting does not change noticeably. Also it stays the same after changing it back to ft vue. But then I'm getting the pug highlighting on visual selections.

It would be nice to get the highlighting not only for the selection.

[zealot128]

This query seems to work:

(template_element
	(start_tag
    	(attribute
    		((attribute_name) @lang_key (#match? @lang_key "lang"))
        	((quoted_attribute_value) @_lang (#match? @_lang "pug"))
        )
    )
    ((text) @pug)
)

... next steps should be (for nvim integration, which @Artem-Schander and me seems to be after in the end) integrating it into the query of nvim-treesitter:

My findings up to now:

• Here seems to be the perfect place to put the query https://github.com/nvim-treesitter/nvim-treesitter/blob/master/queries/vue/injections.scm
• Just pasting it there seems to have no effect.
• There is also no treesitter definition for the Pug file, yet. Maybe it is possible to "fall back" to old-school highlighting? It should be possible, like @Artem-Schander showed with the selection (which I could reproduce with setf)
• But also changing it to a different language, which I have installed (ruby, python etc.) seems to also not trigger any syntax highlighting. So there might be more steps involved to tell nvim-treesitter how to handle @pug etc.
• ...? @ikatyang have you any idea on how to integrate it into nvim, or do you have any pointers?

EDIT: Tested: When changing the @_lang in the query to a language known by treesitter, e.g. typescript, and also changing the template lang to ts, the highlighting works in general. So all the moving parts are there, Treesitter just does not know how to handle Pug files...

[zealot128]

I've implemented a pug tree parser and tried integrating it within neovim with Vue treeparser. Unfortunately, it does not work with the query. The inner content is always parsed by the Vue Treeparser. Instead, we would need a choice for the template tag:

• template tag without lang attribute -> normal parsing like now with embedded html
• template tag with lang attribute -> totally ignore everything within the template tag and return as "raw_text" similar to the script/style tag

Otherwise, the content will be already processed and is not easily passed to the Pug parser as I figured out:

[bjesus]

Hi @ikatyang , any chance #13 can get merged? Those of us who are writing Pug inside Vue files would be forever thankful 🙏

[zealot128]

@ikatyang Also, if you have any other ideas for better recognizing the language tag or adding more tests ... I would be happt for feedback, because I think the current implementation is suboptimal.

@bjesus I am running the vue + pug at the moment with this Nvim setting with lspconfig:

local parser_config = require("nvim-treesitter.parsers").get_parser_configs()
parser_config.vue.install_info.url = "https://github.com/zealot128/tree-sitter-vue.git"
parser_config.pug = {
   install_info = {
      url = "https://github.com/zealot128/tree-sitter-pug", -- local path or git repo
      -- url = "/Users/stefan/LocalProjects/tree-sitter-pug",
      files = { "src/parser.c", "src/scanner.cc" },
   },
   filetype = "pug", -- if filetype does not agrees with parser name
}

Afterwards:

:TSInstall pug
:TSInstall vue

Or :TSInstallFromSource ...

You might test it if it works with your files...

[bjesus]

@zealot128 I'd love to get it working but so far with no luck. I've followed the above steps but currently still no luck:

Not sure what I missed. TSModuleInfo shows pug ✗ ✗ ✗ ✗ while TSInstallInfo shows pug [✓] installed. TSConfigInfo gives this:

{
  ensure_installed = {},
  ignore_install = {},
  modules = {
    autopairs = {
      disable = {},
      enable = false,
      module_path = "nvim-autopairs.internal"
    },
    highlight = {
      additional_vim_regex_highlighting = false,
      custom_captures = {},
      disable = { "markdown" },
      enable = true,
      loaded = true,
      module_path = "nvim-treesitter.highlight",
      use_languagetree = true
    },
    incremental_selection = {
      disable = {},
      enable = false,
      keymaps = {
        init_selection = "gnn",
        node_decremental = "grm",
        node_incremental = "grn",
        scope_incremental = "grc"
      },
      module_path = "nvim-treesitter.incremental_selection"
    },
    indent = {
      disable = {},
      enable = false,
      module_path = "nvim-treesitter.indent"
    }
  },
  update_strategy = "lockfile"
}

Am I missing any step?

[parascent]

@bjesus I dont know what you should do. But this is what I did. I added the following code to .config/nvim/plugged/nvim-treesitter/queries/vue/injections.scm

(
  (style_element
    (start_tag
      (attribute
        (quoted_attribute_value (attribute_value) @_lang)))
    (raw_text) @scss)
  (#any-of? @_lang "scss" "postcss" "less" "stylus")
)
(
  (template_element
    (start_tag
      (attribute
        (quoted_attribute_value (attribute_value) @_lang)))
    (text) @pug)
  (#match? @_lang "pug")
)

And the following to .config/nvim/plugged/nvim-treesitter/queries/pug/highlights.scm

(comment) @comment
(tag_name) @tag
(content) @none
(quoted_attribute_value) @string
(id) @constant
(class) @constant
(attribute_name) @label

And the following to .config/nvim/plugged/nvim-treesitter/queries/pug/injections.scm

(javascript) @javascript
(
   (attribute_name) @_attribute_name
   (quoted_attribute_value (attribute_value )  @javascript)
   (#match? @_attribute_name "^(:|v-bind|v-)")
)

I got all these from comments made by @zealot128 on different github issues regarding this. Hope this helps somebody.