Heading or trailing comment in Python class/function not included in text object

[rafikdraoui]

Describe the bug

If the first or last element of a Python class or function is a comment, then targeting it as a text object (with @class.outer, @function.inner, etc.) won't include the comment.

This isn't a real problem for me, just a curious edge case I noticed. If fixing this over-complicates other parts of the code or parser, feel free to disregard this and close, but on the off-chance there is a simple fix I'm missing, I thought I would be reporting it.

To Reproduce

With

treesitter.setup {
  textobjects = {
    select = {
      enable = true,
      keymaps = {
        ["ac"] = "@class.outer",
      },
    },
  },
}

and the Python snippet

class A:
    x = 1
    y = 2  # some comment

with the cursor anywhere on or before the character 2, then typing vac will select everything except the comment. If the cursor is on the comment, then typing vac won't do anything.

Expected behavior

With the cursor anywhere in the class, typing vac selects the whole class.

Output of :checkhealth nvim_treesitter

health#nvim_treesitter#check

Installation

• OK: git executable found.
• OK: cc executable found.

elm parser healthcheck

• OK: elm parser found.
• WARNING: No highlights.scm query found for elm
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for elm
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for elm
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for elm
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

c parser healthcheck

• OK: c parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• OK: folds.scm found.

java parser healthcheck

• OK: java parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for java
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for java
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

python parser healthcheck

• OK: python parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• OK: folds.scm found.

dart parser healthcheck

• OK: dart parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for dart
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for dart
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

lua parser healthcheck

• OK: lua parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• OK: folds.scm found.

ocaml parser healthcheck

• OK: ocaml parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for ocaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: folds.scm found.

go parser healthcheck

• OK: go parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• WARNING: No folds.scm query found for go
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

nix parser healthcheck

• OK: nix parser found.
• WARNING: No highlights.scm query found for nix
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for nix
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for nix
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for nix
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

yaml parser healthcheck

• OK: yaml parser found.
• WARNING: No highlights.scm query found for yaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for yaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for yaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for yaml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

json parser healthcheck

• OK: json parser found.
• OK: highlights.scm found.
• WARNING: No locals.scm query found for json
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for json
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for json
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

jsdoc parser healthcheck

• OK: jsdoc parser found.
• WARNING: No highlights.scm query found for jsdoc
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for jsdoc
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for jsdoc
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for jsdoc
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

markdown parser healthcheck

• OK: markdown parser found.
• WARNING: No highlights.scm query found for markdown
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for markdown
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for markdown
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for markdown
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

julia parser healthcheck

• OK: julia parser found.
• WARNING: No highlights.scm query found for julia
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for julia
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for julia
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for julia
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

html parser healthcheck

• OK: html parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for html
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for html
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

typescript parser healthcheck

• OK: typescript parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• OK: folds.scm found.

fennel parser healthcheck

• OK: fennel parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for fennel
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for fennel
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

swift parser healthcheck

• OK: swift parser found.
• WARNING: No highlights.scm query found for swift
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for swift
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for swift
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for swift
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

query parser healthcheck

• OK: query parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for query
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for query
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

cpp parser healthcheck

• OK: cpp parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• OK: folds.scm found.

regex parser healthcheck

• OK: regex parser found.
• OK: highlights.scm found.
• WARNING: No locals.scm query found for regex
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for regex
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for regex
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

ruby parser healthcheck

• OK: ruby parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for ruby
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for ruby
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

vue parser healthcheck

• OK: vue parser found.
• WARNING: No highlights.scm query found for vue
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for vue
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for vue
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for vue
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

ql parser healthcheck

• OK: ql parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for ql
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for ql
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

scala parser healthcheck

• OK: scala parser found.
• WARNING: No highlights.scm query found for scala
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for scala
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for scala
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for scala
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

toml parser healthcheck

• OK: toml parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for toml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for toml
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

rust parser healthcheck

• OK: rust parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• WARNING: No folds.scm query found for rust
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

bash parser healthcheck

• OK: bash parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for bash
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for bash
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

ocaml_interface parser healthcheck

• OK: ocaml_interface parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for ocaml_interface
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: folds.scm found.

rst parser healthcheck

• OK: rst parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• WARNING: No folds.scm query found for rst
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

css parser healthcheck

• OK: css parser found.
• OK: highlights.scm found.
• WARNING: No locals.scm query found for css
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for css
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for css
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

c_sharp parser healthcheck

• OK: c_sharp parser found.
• OK: highlights.scm found.
• WARNING: No locals.scm query found for c_sharp
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: textobjects.scm found.
• WARNING: No folds.scm query found for c_sharp
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

javascript parser healthcheck

• OK: javascript parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• OK: folds.scm found.

php parser healthcheck

• OK: php parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• WARNING: No textobjects.scm query found for php
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• OK: folds.scm found.

haskell parser healthcheck

• OK: haskell parser found.
• WARNING: No highlights.scm query found for haskell
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No locals.scm query found for haskell
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No textobjects.scm query found for haskell
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter
• WARNING: No folds.scm query found for haskell
  • ADVICE:
    • Open an issue at https://github.com/nvim-treesitter/nvim-treesitter

tsx parser healthcheck

• OK: tsx parser found.
• OK: highlights.scm found.
• OK: locals.scm found.
• OK: textobjects.scm found.
• OK: folds.scm found.

Output of nvim --version

NVIM v0.5.0-721-g3c5141d2c
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/gcc-5 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -O2 -g -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=3 -I/home/travis/build/neovim/bot-ci/build/neovim/build/config -I/home/travis/build/neovim/bot-ci/build/neovim/src -I/home/travis/build/neovim/bot-ci/build/neovim/.deps/usr/include -I/usr/include -I/home/travis/build/neovim/bot-ci/build/neovim/build/src/nvim/auto -I/home/travis/build/neovim/bot-ci/build/neovim/build/include
Compiled by travis@travis-job-5482189d-7ef7-4d79-892c-6d7ec9adbd1f

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "
/home/travis/build/neovim/bot-ci/build/neovim/build/nvim.AppDir/usr/share/nvim
"

Run :checkhealth for more info

[theHamsta]

At the moment we have the extreme shortcoming that we can only match for the range of a single node. We had some ideas how we could resolve this problem in the future. We will need some directive that produce range and not only nodes. I have some hacks but didn't implement a thorough solution yet

At the moment treesitter-textobjects (will move soon to https://github.com/nvim-treesitter/nvim-treesitter-textobjects) implements a hack that can help you with your situation:

(function_definition
 (comment)? @function.inner.start  ; <- add this line, this might help you including the leading comment
  body: (block)? @function.inner) @function.outer

You can add @<textobject-capture>.start to indicate another node that should serve as the start for your range. We use this functionality to include decorators and C++ template declarations.

I didn't implement a corresponding @function.inner.end yet.

It's not included yet for comments since the comment might by totality unrelated but appear adjacent in the syntax tree. Just imagine comments separated from the function by multiple empty lines (we might solve this with additional predicates like same-line? or same-paragraph?)

Feel free to open a PR https://github.com/nvim-treesitter/nvim-treesitter-textobjects

[theHamsta]

Strange, I don't know why this example does not work. @function.outer.start is used for function decorations.

EDIT: I found the reason, apparently the table from @function.inner.start gets lost when it's specified before @function.inner in the query text. This should be a easy to resolve issue in upstream Neovim. EDIT: found the bug in the query generation in our repo .

[vigoux]

I don't know if this still exists, but anyway, this should move to the nvim-treesitter-texobjects.

[theHamsta]

@rafikdraoui FYI: https://github.com/tree-sitter/tree-sitter-python/commit/87e874f23e1bbfc4bd160caa3bad03de250c8b9f https://github.com/tree-sitter/tree-sitter-python/issues/97

[stsewd]

This is a bug from the python parser https://github.com/tree-sitter/tree-sitter-python/issues/113, https://github.com/tree-sitter/tree-sitter-python/pull/143 should fix it. But there is still the problem of not including leading comments.

def a():
    # Here
    a = b # c

(module [0, 0] - [3, 0]
  (function_definition [0, 0] - [2, 13]
    name: (identifier [0, 4] - [0, 5])
    parameters: (parameters [0, 5] - [0, 7])
    (comment [1, 4] - [1, 10])
    body: (block [2, 4] - [2, 13]
      (expression_statement [2, 4] - [2, 9]
        (assignment [2, 4] - [2, 9]
          left: (identifier [2, 4] - [2, 5])
          right: (identifier [2, 8] - [2, 9])))
      (comment [2, 10] - [2, 13]))))

[stsewd]

leading comments are included outside the function body.