Feature request: Expose types of snippet globals for nicer development

[MariaSolOs]

When writing custom snippets, it would be really helpful to use type annotations for s, t, and other global snippet constructors for better editing support :)

[bew]

Related #378 (but incomplete)

[L3MON4D3]

Oh yeah, for sure! As far as I understand, this would basically be whatever is in DOC.md, but as emmylua-annotation, so lua-language-server can parse it? If so, I'd say there is a choice to make:

• either just copy+transform what is currently in DOC.md to the source-files (which could be done relatively quickly, but having to keep both in sync manually seems suboptimal, and mistakes could happen), or
• create a proper build-step for extracting these definitions and inserting them into DOC.md (initial setup seems much harder, but this seems cleaner at least)

I'd honestly be fine with either option since I don't really anticipate many major additions to luasnip, so the additional long-time-work induced by the first option may not outweigh one-time work and complexity introduced by the second one (though the second one may be fun to figure out)

Unfortunately, I really don't use annotations at all though, so my motivation for doing either is pretty low :sweat_smile:

[MariaSolOs]

@L3MON4D3 that's fair. I think that if no major API changes are predicted, then option 1 is fine.

Any reason why #378 hasn't been merged? Despite being incomplete, it's still an improvement.

[L3MON4D3]

Only that it's marked as draft, and adding annotations for a small number of api-functions only seemed a bit weird. But you're right, better that than nothing. I'll look into getting it merged

[MariaSolOs]

Sounds good. I think that splitting the work into smaller PRs will be less overwhelming than documenting the entire API at once (I think no one has the motivation for that hehe).

I’m also happy to help with this btw!

[L3MON4D3]

Sounds good. I think that splitting the work into smaller PRs will be less overwhelming than documenting the entire API at once

Ah, yeah, probably also true :D

I’m also happy to help with this btw!

Yay, hoped for just that :P Best wait until I'm done with #941 (shouldn't be long now), lots of changes to the files where api is defined in there

[UtkarshVerma]

Hi, I am also interested in this. Since the other PR is merged. Is there anything else blocking this?

[bew]

In #1117 (and https://github.com/L3MON4D3/LuaSnip/pull/1117#issuecomment-1943633183) we saw that type hinting the nodes is tricky because of the lazyness of ls table. /cc @michaeldebetaz

Now that I played with types a lot more in my config, I just thought about a very nice (and working!) workaround to be able to type node function with minimal duplications, and best DX: 👉 I am using the fact that LuaLS explores all the code, even when it is never actually executed.

With this diff

diff --git a/lua/luasnip/init.lua b/lua/luasnip/init.lua
index 96cced8..b7d59f0 100644
--- a/lua/luasnip/init.lua
+++ b/lua/luasnip/init.lua
@@ -187,6 +187,8 @@ local function jumpable(dir)
 		~= session.current_nodes[vim.api.nvim_get_current_buf()]
 end
 
+--- `true` if a snippet can be expanded at the current cursor position
+---@return boolean
 local function expandable()
 	next_expand, next_expand_params =
 		match_snippet(util.get_current_line_to_cursor(), "snippets")
@@ -851,7 +853,43 @@ local ls_lazy = {
 	post_yank = function() return require("luasnip.util.select").post_yank end,
 }
 
-ls = lazy_table({
+-- This is a tentative at auto-defining a complex type
+-- This will never be executed
+if false then
+  ---@class LS_lazy
+  _ = {
+	text_node = require("luasnip.nodes.textNode").T,
+  -- ...
+  }
+end
+
+---@class LS_static
+local ls_static = {
   expand_or_jumpable = expand_or_jumpable,
   expand_or_locally_jumpable = expand_or_locally_jumpable,
   locally_jumpable = locally_jumpable,
@@ -895,6 +933,10 @@ ls = lazy_table({
   extend_decorator = extend_decorator,
   log = require("luasnip.util.log"),
   activate_node = activate_node,
-}, ls_lazy)
+}
 
+---@class LS: LS_static, LS_lazy
+ls = lazy_table(ls_static, ls_lazy)
 return ls
diff --git a/lua/luasnip/nodes/textNode.lua b/lua/luasnip/nodes/textNode.lua
index 6022269..7071e63 100644
--- a/lua/luasnip/nodes/textNode.lua
+++ b/lua/luasnip/nodes/textNode.lua
@@ -7,6 +7,38 @@ local feedkeys = require("luasnip.util.feedkeys")
 
 local TextNode = node_mod.Node:new()
 
+-- for now...
+---@alias LS.ExtOpts table
+---@alias LS.NodeCallbacks table
+---@class LS.Node: table
+
+---@class LS.NodeOpts
+---@field node_ext_opts LS.ExtOpts?
+---@field key string?
+---@field node_callbacks LS.NodeCallbacks?
+
+--- The most simple kind of node; just text.
+---
+--- ```lua
+--- s("trigger", { t("Wow! Text!") })
+--- ```
+---
+--- This snippet expands to:
+--- ```
+--- Wow! Text!⎵
+--- ```
+--- where ⎵ is the cursor.
+---
+--- Multiline strings can be defined by passing a table of lines rather than a string:
+--- ```lua
+---     s("trigger", {
+---         t({"Wow! Text!", "And another line."})
+---     })
+--- ```
+---
+---@param static_text string|string[]
+---@param opts LS.NodeOpts?
+---@return LS.Node
 local function T(static_text, opts)
 	return TextNode:new({
 		static_text = util.to_string_table(static_text),

I get:

🎉 I'll open a PR for the base idea here, and future PRs will be able to incrementally add documentation to nodes / utils / internal functions / .... Similarly a way to generate the DOC.md can be thought about as a later PR.

[Rick-Phoenix]

Any updates on this? It would be amazing if you could actually have some docs/type completion for the various methods.

[bew]

Any updates on this? It would be amazing if you could actually have some docs/type completion for the various methods.

I opened #1318 with the base idea I mentioned in the last comment. I'm waiting for feedback from the maintainer on my approach before going forward with adding types ~everywhere.

There's still a big question on the duplication of documentation between the vim help page, the DOC.md file and soon in the codebase itself. Currently the vim help is generated from the DOC.md file. I think the goal would be to generate both from documentation comments in the codebase.

.. I think we'll start with a period with some duplication, then play with tools like https://github.com/mrcjkb/vimcats, see how much we can use them to achieve our goal 🤔 .. Interesting page about additional annotations we can use to describe some meta information for generating doc pages: https://github.com/numToStr/lemmy-help/blob/master/emmylua.md

[Rick-Phoenix]

That sounds like a really good approach actually. In the meantime I am just defining types manually on my wrappers like this, in a module that I require from the files where I define my snippets:

local _i = require("luasnip.nodes.insertNode").I

---@alias LuasnipNode table

--- Creates an interactive node (placeholder) that can be jumped to and edited.
---
---@param jump_index number Position in the jump sequence. See |luasnip-basics-jump-index|. `i(0)` has special behavior (see Luasnip#110).
---@param text string|string[]? Initial text content (single or multi-line), selected when jumped to.
---@param node_opts table? Optional table for common node settings. See |luasnip-node|.
local i = function(jump_index, text, node_opts)
  return _i(jump_index, text, node_opts)
end
_G.i = i

So at least I can have some basic info in the hover + the links to the relevant help tags

[L3MON4D3]

Hey all, sorry for the long silence 😅

There's still a big question on the duplication of documentation between the vim help page, the DOC.md file and soon in the codebase itself. Currently the vim help is generated from the DOC.md file. I think the goal would be to generate both from documentation comments in the codebase.

I agree that the current situation is suboptimal, it would be better if the documentation for the various functions lived right beside them in the code, not in some separate file. I don't think I'd try to generate all of DOC.md from the source-files (there are some sections in it where I can't think of a good place to put them in the sourcecode (motivation for various functionality), and there are links to gifs and the like in there that don't make much sense in a .lua :D). So, I'd look for a solution where only the documentation of the functions is generated from the annotations, and then inserted into a skeleton/template of DOC.md which contains all the surrounding context

vimcats would be interesting if it could generate markdown instead of vimdoc, and if it could extract the documentation for specific functions 🤔 Maybe we can repurpose it a bit, although I don't know much Rust 😅

[L3MON4D3]

I've looked into this a bit, and I think we can just extract the necessary information using lua-language-server. It has a feature where it exports all types in a codebase, so for example

lua-language-server --doc ./ --doc_out_path ./
cat doc.json | jq '.[] | select(.name == "LuaSnip.FSWatcher.Tree" and .type == "type").fields | map(.extends.view)'
cat doc.json | jq '.[] | select(.name == "LuaSnip.FSWatcher.Tree" and .type == "type").fields | map(.rawdesc)'
cat doc.json | jq '.[] | select(.name == "LuaSnip.FSWatcher.Tree" and .type == "type").fields | map(.extends.args)'

can be used to get information on the LuaSnip.FSWatcher.Tree-class (lua/luasnip/loaders/fs_watchers.lua)

.extends.view

[
  "(method) LuaSnip.FSWatcher.Tree:BufWritePost_callback(realpath: any)",
  "LuaSnip.FSWatcher.TreeCallbacks",
  "(method) LuaSnip.FSWatcher.Tree:change_child(rel: any, full: any)",
  "(method) LuaSnip.FSWatcher.Tree:change_dir(rel: any, full: any)",
  "(method) LuaSnip.FSWatcher.Tree:change_file(rel: any, full: any)",
  "number",
  "table<string, LuaSnip.FSWatcher.Tree>",
  "table<string, boolean>",
  "userdata",
  "(method) LuaSnip.FSWatcher.Tree:fs_event_callback(err: any, relpath: any, events: any)",
  "table<\"autocmd\"|\"libuv\", boolean>",
  "(method) LuaSnip.FSWatcher.Tree:new_dir(rel: any, full: any)",
  "(method) LuaSnip.FSWatcher.Tree:new_file(rel: any, full: any)",
  "unknown",
  "(method) LuaSnip.FSWatcher.Tree:remove_child(rel: any, full: any)",
  "(method) LuaSnip.FSWatcher.Tree:remove_root()",
  "boolean",
  "string",
  "string?",
  "boolean",
  "boolean",
  "(method) LuaSnip.FSWatcher.Tree:start()",
  "(method) LuaSnip.FSWatcher.Tree:stop()",
  "(method) LuaSnip.FSWatcher.Tree:stop_self()",
  "boolean"
]

and

.rawdesc

[
  " May not recognize child correctly if there are symlinks on the path from the\n child to the directory-root.\n Should be fine, especially since, I think, fs_event can recognize those\n correctly, which means that this is an issue only very seldomly.",
  null,
  null,
  null,
  null,
  "How deep the root should be monitored.",
  null,
  null,
  null,
  null,
  null,
  null,
  " these functions maintain our logical view of the directory, and call\n callbacks when we detect a change.",
  " needed by BufWritePost-callback.",
  null,
  null,
  null,
  null,
  "Set as soon as the watcher is started.",
  null,
  null,
  null,
  null,
  null,
  null
]

and

.extends.args

[
  [
    {
      "finish": [
        215,
        8
      ],
      "name": "self",
      "start": [
        215,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    },
    {
      "finish": [
        215,
        51
      ],
      "name": "realpath",
      "start": [
        215,
        43
      ],
      "type": "local",
      "view": "any"
    }
  ],
  null,
  [
    {
      "finish": [
        383,
        8
      ],
      "name": "self",
      "start": [
        383,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    },
    {
      "finish": [
        383,
        37
      ],
      "name": "rel",
      "start": [
        383,
        34
      ],
      "type": "local",
      "view": "any"
    },
    {
      "finish": [
        383,
        43
      ],
      "name": "full",
      "start": [
        383,
        39
      ],
      "type": "local",
      "view": "any"
    }
  ],
  [
    {
      "finish": [
        379,
        8
      ],
      "name": "self",
      "start": [
        379,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    },
    {
      "finish": [
        379,
        35
      ],
      "name": "rel",
      "start": [
        379,
        32
      ],
      "type": "local",
      "view": "any"
    },
    {
      "finish": [
        379,
        41
      ],
      "name": "full",
      "start": [
        379,
        37
      ],
      "type": "local",
      "view": "any"
    }
  ],
  [
    {
      "finish": [
        375,
        8
      ],
      "name": "self",
      "start": [
        375,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    },
    {
      "finish": [
        375,
        36
      ],
      "name": "rel",
      "start": [
        375,
        33
      ],
      "type": "local",
      "view": "any"
    },
    {
      "finish": [
        375,
        42
      ],
      "name": "full",
      "start": [
        375,
        38
      ],
      "type": "local",
      "view": "any"
    }
  ],
  null,
  null,
  null,
  null,
  [
    {
      "finish": [
        150,
        8
      ],
      "name": "self",
      "start": [
        150,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    },
    {
      "finish": [
        150,
        42
      ],
      "name": "err",
      "start": [
        150,
        39
      ],
      "type": "local",
      "view": "any"
    },
    {
      "finish": [
        150,
        51
      ],
      "name": "relpath",
      "start": [
        150,
        44
      ],
      "type": "local",
      "view": "any"
    },
    {
      "finish": [
        150,
        59
      ],
      "name": "events",
      "start": [
        150,
        53
      ],
      "type": "local",
      "view": "any"
    }
  ],
  null,
  [
    {
      "finish": [
        355,
        8
      ],
      "name": "self",
      "start": [
        355,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    },
    {
      "finish": [
        355,
        32
      ],
      "name": "rel",
      "start": [
        355,
        29
      ],
      "type": "local",
      "view": "any"
    },
    {
      "finish": [
        355,
        38
      ],
      "name": "full",
      "start": [
        355,
        34
      ],
      "type": "local",
      "view": "any"
    }
  ],
  [
    {
      "finish": [
        345,
        8
      ],
      "name": "self",
      "start": [
        345,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    },
    {
      "finish": [
        345,
        33
      ],
      "name": "rel",
      "start": [
        345,
        30
      ],
      "type": "local",
      "view": "any"
    },
    {
      "finish": [
        345,
        39
      ],
      "name": "full",
      "start": [
        345,
        35
      ],
      "type": "local",
      "view": "any"
    }
  ],
  null,
  [
    {
      "finish": [
        391,
        8
      ],
      "name": "self",
      "start": [
        391,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    },
    {
      "finish": [
        391,
        37
      ],
      "name": "rel",
      "start": [
        391,
        34
      ],
      "type": "local",
      "view": "any"
    },
    {
      "finish": [
        391,
        43
      ],
      "name": "full",
      "start": [
        391,
        39
      ],
      "type": "local",
      "view": "any"
    }
  ],
  [
    {
      "finish": [
        408,
        8
      ],
      "name": "self",
      "start": [
        408,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    }
  ],
  null,
  null,
  null,
  null,
  null,
  [
    {
      "finish": [
        254,
        8
      ],
      "name": "self",
      "start": [
        254,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    }
  ],
  [
    {
      "finish": [
        133,
        8
      ],
      "name": "self",
      "start": [
        133,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    }
  ],
  [
    {
      "finish": [
        140,
        8
      ],
      "name": "self",
      "start": [
        140,
        8
      ],
      "type": "self",
      "view": "LuaSnip.FSWatcher.Tree"
    }
  ],
  null
]

So there's quite a bit of info to get from there :)

Unfortunately, this does not yet work for the functions defined in require("luasnip"), but that seems like a small obstacle, I'm pretty certain we can extract their metadata as well :)

[bew]

Oh that's interesting, didn't know luals could be used like this!

If I understand what you're hinting at, once we add docs in the Lua code we can easily extract the data in some script to generate the markdown?

[L3MON4D3]

Yeah, that's what I'm thinking of :D

We'd have some kind of marker in a (not yet existing) DOC.md.template-file, and that marker contains a type and a function-name, which we can use to query doc.json

cat doc.json | jq '.[] | select(.name == "<typename>" and .type == "type").fields| .[] | select(.name == "<fname>")'

We'll have to have some mechanism for turning links to other source-files or types in the annotations to proper links in DOC.md, so there will be some complexity to this. I think it'd also be good to put this in a separate repo, it may be useful to other projects as well

[L3MON4D3]

I have created a small sample-project for testing ideas on this.

For now, I've set it up so that we look for markdown code-blocks with a specific info_string (according to the treesitter-parser :D), and this can contain instruction on what documentation to render at this position:

```lua render_region
render_fn_doc({ typename = "Cat", funcname = "meow" })
```

with the function

---@class Cat
local Cat = {}

---Make the cat meow at something, in some volume.
---@param target string What to meow at.
---@param volume number How loud to meow.
function Cat:meow(target, volume) end

results in

`Cat.meow(self:Cat, target:string, volume:number)`
Make the cat meow at something, in some volume.
* `self: Cat` 
* `target: string` What to meow at.
* `volume: number` How loud to meow.

I like this because we get proper highlighting, and it's flexible enough to accomodate all kind of weird customizations :) AFAIK what is written after the filetype of the codeblock does not matter, if that's not the case, feel free to let me know :D

[llakala]

If it's at all possible, I would love some manual annotations for the default SNIP_ENV variables - like s, fmt, etc. I can't rely on the trick used here, since my snippets are in a separate repo, so if I want type annotations in that repo without having globals everywhere, I'll need to declare them myself.

[L3MON4D3]

I've looked into that recently, could you try the idea given in the doc? Sounds like it may be applicable :eyes:

[llakala]

I've looked into that recently, could you try the idea given in the doc? Sounds like it may be applicable 👀

That's the one I was following. There's a lot of different ways listed there - which one specifically should I try?

[L3MON4D3]

I've made some more progress, follow #1353 for updates :)

[bew]

Hellow friends o/ I opened a draft PR @ #1396 with a big chunk of nodes & internal code that are now mostly type annotated 🚀 Still very much draft, but it might interest you nonetheless 🙃