Reorganize nvimrserver

[she3o]

Hi, I'm still working on this PR. Please wait a bit before merging. I'm open to suggestions, so feel free to share your thoughts. Thanks.

[she3o]

I'm unfamiliar with compiling C code on Windows and currently don't have access to a Windows machine for testing purposes. My strategy includes segregating include/windows.h in the project. I would greatly appreciate any advice or perspectives on this approach.

[jalvesaq]

One advantage of having everything in a single file is that functions not called by R code might remain static, visible only to nvimrserver.c itself. This also avoids the need to create header files. We don't even need to declare them beforehand, we may just define them before their use.

[jalvesaq]

For Windows, I downloaded a virtual machine from Microsoft website and I run it in qemu when there is any change that might not work on Windows. But I can quickly check if the code compiles without warnings with this script:

#!/bin/sh

/usr/bin/x86_64-w64-mingw32-gcc-win32 -o /tmp/nvimrserver.exe \
  /path/to/Nvim-R/R/nvimcom/src/apps/nvimrserver.c \
  -Wall -pedantic -O3 -static-libgcc -m64 -mwindows -std=gnu99 -lws2_32

/usr/bin/x86_64-w64-mingw32-gcc-win32 -c -o /tmp/nvimcom.o \
  /path/to/Nvim-R/R/nvimcom/src/nvimcom.c -Wall -pedantic -O3 \
  -static-libgcc -m64 -mwindows -lws2_32 -L/path/to/R-4.3.2/lib \
  -I/path/to/R-4.3.2/include

[jalvesaq]

The Windows code no longer bothers me because now it is highlighted as Comment (by clangd/lsp/tree-sitter?).

[jalvesaq]

One advantage of having everything in a single file is that functions not called by R code might remain static, visible only to nvimrserver.c itself.

Actually, this is an advantage only to nvimcom.c because nvimrserver.c is not a R library. So, the main reason to not split the file doesn't exist, and we can split it to make it easier to understand and maintain...

[she3o]

The Windows code no longer bothers me because now it is highlighted as Comment (by clangd/lsp/tree-sitter?).

Wait.. That means the code is no longer necessary even on Windows? I thought it was dimmed because I am on Linux So the compiler didn't use/test that code but would be highlighted otherwise on a Windows machine

[jalvesaq]

It's necessary. But I no longer have to make any effort to skip it when reading the code because now it is completely highlighted as Comment:

[she3o]

for now, I will return nvimrserver.c to root and update makefiles accordingly to resolve this conflict.

[jalvesaq]

I think nvimrserver is too simple to have a separate directory for the .h files unless you are planning to use it as a library for something else. Even a much larger project such as nvim has the headers in the same directory as the .c files.

[she3o]

I'm currently devising a long-term plan to develop a new implementation of nvimrserver, potentially using Lua or Rust. This approach aims to offer users a choice between the original, stable application and a newer, possibly less stable version.

nvimrserver is inherently complex, supporting a wide array of features. To effectively tackle this project, my strategy involves isolating the existing C implementation of nvimrserver. This isolation will allow for a gradual integration of functionalities into the new environment, ensuring each component is robustly implemented.

I am actively learning the interfacing of C with Lua to facilitate this transition. As this progresses, I anticipate the need for specific include/ libraries tailored for Lua integration.

[jalvesaq]

Currently, we have cmp-nvim-r using the nvimrserver for completion. How about converting nvimcom+nvimrserver into the completion mechanism of an R language server that could be used by other text editors?

[she3o]

I will look into that. I haven't been able to contribute to nvimcom or any of the LSPs/parsers because they assume I understand R internals. The nice thing about nvimrserver is it doesn't interact with R objects directly.

I will revert a couple of commits from this PR to make it mergeable. I will revert the last two commits because as you mentioned they introduce a high level of complexity. but I would like to keep separate source files in the root directory since they help make the code manageable.

I might also add a couple of Doxygen comment blocks for future reference before completing this PR.

[she3o]

I would also be happy to pick any low-hanging fruits, such as eliminating outdated backward compatibility features. This will streamline the code and also allow me to familiarize myself with it.

[jalvesaq]

What "outdated backward compatibility features"? Is legacy omni-completion an example? Or, much more than this, do you mean dropping support for Vim and converting the .vim scripts into .lua?

I took part of the code that was in VimLanguage, rewrote it in C, and put it in nvimrserver.c. Nvim-R is already mostly optimized for speed.

[jalvesaq]

C code interacting with R is much easier to write today (with clangd as a language server) than it was when I wrote most of nvimcom.c code. You have only to be careful to PROTECT() an R object when it is created and UNPROTECT() it when it will no longer be used. R will crash if an R object is not protected or if the number of calls to PROTECT() and UNPROTECT() are not perfectly balanced.

[she3o]

What "outdated backward compatibility features"? Is legacy omni-completion an example? Or, much more than this, do you mean dropping support for Vim and converting the .vim scripts into .lua?

I don't know how many of the current users of nvim-R still use vim. If a substantial number of people still use Vim and nvim-R daily, phasing out Vim support would be problematic.

I've observed that nvim-R includes functionalities such as legacy omnicompletion, commenting, and syntax highlighting. Given that these features might be more efficiently handled by specialized plugins, I'm wondering if there are any plans to streamline nvim-R by delegating these aspects to other plugins.

I also have a few questions regarding future changes:

• Are there any plans to discontinue tmux integration in nvim-R?
• Is there consideration for rewriting parts of the plugin, currently in Vimscript, in Lua?
• If such a rewrite is on the agenda, do you foresee any performance or maintainability advantages in adopting Lua?

I am also wondering what would you have done differently if you had started writing Nvim-R in 2024.

I deeply appreciate the time you are taking to read, respond, and review. :blush:

[jalvesaq]

I am also wondering what would you have done differently if you had started writing Nvim-R in 2024.

I guess I would use Lua instead of VimLanguage because it would be easier for more people to contribute, and also because it would be a better time investment to learn Lua than VimLanguage.

But note that in 2009 even RStudio didn't exist. Today, we have Neovim, LSP, and a languageserver for R. I would use them and not start something new.

I don't know how many of the current users of nvim-R still use vim. If a substantial number of people still use Vim and nvim-R daily, phasing out Vim support would be problematic.

I also don't know.

I've observed that nvim-R includes functionalities such as legacy omnicompletion, commenting, and syntax highlighting. Given that these features might be more efficiently handled by specialized plugins, I'm wondering if there are any plans to streamline nvim-R by delegating these aspects to other plugins.

I no longer use the legacy omni-completion. I'm also using a plugin for commenting code. But I still see Vim's syntax highlighting as better than tree-sitter highlighting for R and Quarto. Nvim-R only highlights function names, and I prefer functions highlighted as they are loaded which is something that other plugins can't do. I also have to disable tree-sitter indenting to avoid problems in R code.

Are there any plans to discontinue tmux integration in nvim-R?

Tmux is required to run R in an external terminal. I no longer guarantee the support to start R in a split Tmux pane. The code for this is still in Nvim-R because it's stable and doesn't require maintenance.

Is there consideration for rewriting parts of the plugin, currently in Vimscript, in Lua?

Perhaps, if someone wants to do the job, although this would mean ending support for Vim.

If such a rewrite is on the agenda, do you foresee any performance or maintainability advantages in adopting Lua?

It's not on my agenda. Although I have never perceived slowness in Vimscript, I already translated to C what required ultra-speed. In fact, completion from nvimrserver.c is so fast that it could be refreshed after each keystroke, which nvim-cmp doesn't. I don't foresee any performance gain in adopting Lua, but probably more people would be able to contribute with pull requests.

[jalvesaq]

I created a file named compile_commands.json in the directory above Nvim-R:

[
  {
      "directory": "/path/to/Nvim-R/R/nvimcom/src",
      "command": "/usr/bin/clang -I /path/to/software/R-4.3.2/include -DNDEBUG -fpic -O3 -Wall -pedantic -Wformat -Wstrict-prototypes -mtune=native -c",
      "file": "*"
  },

  {
      "directory": "/path/to/Nvim-R/R/nvimcom/src/apps",
      "command": "/usr/bin/clang -O3 -Wall -pedantic -Wformat -Wstrict-prototypes -mtune=native -c",
      "file": "*"
  }
]

Perhaps we should add this to CONTRIBUTING.md. With the above options, clangd complains about "function declaration without a prototype", which can be fixed by putting void within the parentheses.

[jalvesaq]

I'm currently devising a long-term plan to develop a new implementation of nvimrserver, potentially using Lua or Rust. This approach aims to offer users a choice between the original, stable application and a newer, possibly less stable version.

Rewriting nvimrserver might not be enough to attract people to your project. Nvim-R misses some features that I can't implement because I don't have the necessary spare time. If you implemented these features, I could freeze Nvim-R development (but keep fixing bugs) and redirect all people requesting new features to your project. Some suggestions:

• Remove support for Vim and for Windows

• Translate VimScript into Lua

• Develop new features:

  • Language server (completion and object browser)

    • Implement options: ignore case, fuzzy finder, snippets.

    • Quarto:

      • Completion of valid values of cell fields

      • Completion of YAML header fields

  • Debugger server. See:

    • https://github.com/mfussenegger/nvim-dap
    • https://github.com/rcarriga/nvim-dap-ui
    • https://github.com/ManuelHentschel/VSCode-R-Debugger

[she3o]

Remove support for Vim and for Windows

I think That because Neovim runs on all platforms, and R runs on Windows as well as Unix, Nvim-R, or any similar project should support Windows to some degree. That's unless we know that barely anyone uses Neovim on Windows.

[she3o]

Hi, I just want to say that this PR is alive but I am looking for some spare time to add the finishing touches and wrap it up.

[jalvesaq]

No problem. There is no known bug in nvimrserver.c. So, there is no need of making changes that would conflict with your pull request.

[jalvesaq]

I'm currently devising a long-term plan to develop a new implementation of nvimrserver, potentially using Lua or Rust. This approach aims to offer users a choice between the original, stable application and a newer, possibly less stable version.

Another possibility is to create an organization in Github (R-Vim?) with at least the following projects:

• Vim-R (Vim9 Script)
• Nvim-R (Lua)
• cmp-nvim-r
• nvim-dap-r

We would need the help of maintainers for Vim and Windows, otherwise support for them could be frozen in their current state.

[she3o]

Another possibility is to create an organization in Github (R-Vim?) 

That would be awesome. Perhaps it should be called "R Vim toolkit", "R Vim developer's tools", "R Vim Suite", "VimR Universe", "R Vimverse", or some other variation. This will

• Elaborate the difference between the organization and already existing Nvim-R

• Not confuse newcomers who would have to learn that R-Vim is the Organization that contains Nvim-R, and that Nvim-R is actually the codebase that supports legacy Vim.

[jalvesaq]

I prefer "R Vim toolkit" and "R Vim suite". Anything with "universe" would not correspond to reality because there are some other projects that contribute to this "universe" (quarto-nvim, languageservr, nvim-treesitter, etc.). I will open a Discussion about this new organization since it goes beyond the scope of this issue. Hopefully, other people will join.

[jalvesaq]

I started the discussion: https://github.com/jalvesaq/Nvim-R/discussions

[hongyuanjia]

Remove support for Vim and for Windows

I think That because Neovim runs on all platforms, and R runs on Windows as well as Unix, Nvim-R, or any similar project should support Windows to some degree. That's unless we know that barely anyone uses Neovim on Windows.

Here for a Windows Neovim and NVim-R user for quite some years. If you guys need any help testing features on Windows, I would like to help.

[jalvesaq]

Thanks, @hongyuanjia !

[jalvesaq]

@she3o could you do the pull request on tmp-R-Nvim immediately? We can fix bugs and implement the remaining changes later.

[she3o]

Yes, as soon as I have a computer