Ideas for a future major version

[hl037]

I would like to open a thread for some suggestions I would have for a future UltiSnips Version.

This is definitively not a roadmap nor some feature request, just some idea I would love to debate... Even if they never get implemented

[hl037]

While digging in UltiSnips code, I remarked that a lot of stuff are simply hacks because vim would not respond immediately to change from python.

Came to me an idea : Why not use co-routines ?

Here are some point on it :

1. (Golden rule) ONLY 1 co-routine should be executing at once, other thing would only be event handling/dispatching (The goal is not doing async tasks, but only use they ability to be paused and resumed).

2. All function/methods of Ultisnips are marked async

3. All SnippetManager methods are wrapped by a non-async function that would handle the async part (This way, the vim part does not need changes, since it still work with non-async code).

4. An "async vim_dispatch()" function is created (see below for the implementation). This function permits to pause the python code until vim dispatches its own event queue and then resume.

Implementation

The vim_dispatch would actually be rather simple : it would simply call vim.command('call UltiSnips#Reenter()') and await a future.

Here is a POC :

import asyncio
import nest_asyncio # needed because of some silly closed-minded people... https://bugs.python.org/issue22239#msg225883
nest_asyncio.apply()

loop = asyncio.new_event_loop()

class VimResumed():
  slingleton = None
  resumable = None
  
  def __init__(self):
    self.fut = loop.create_future()
  
  async def wait(self):
    # vim.command('call UltiSnips#resume()')
    print('call UltiSnips#resume()')
    self.resumable.fut.set_result(True)
    print('WAITING FOR VIM...')
    await self.fut
  
  def resume(self):
    self.fut.set_result(True)
    self.resumable.resume()
  
  @classmethod
  def init(cls):
    cls.singleton = cls()
  
VimResumed.init()

async def vim_resumed():
  await VimResumed.singleton.wait()

def resume():
  VimResumed.singleton.resume()
  
  

class Resumeable():
  def __init__(self, fn, *args, **kwargs):
    self.fn = fn
    self.args = args
    self.kwargs = kwargs
    
    loop.create_task(self._exec_or_wait())
    self.resume()
  
  def resume(self):
    self.fut = loop.create_future()
    try:
      running_loop = asyncio.get_running_loop()
    except:
      running_loop = None
    asyncio.set_event_loop(loop)
    try:
      loop.run_until_complete(self._wait())
      #loop.create_task(self._wait())
      #loop.run_forever()
    finally:
      if running_loop is not None:
        asyncio.set_event_loop(running_loop)
    
  async def _wait(self):
    await self.fut
    
  async def _exec_or_wait(self):
    VimResumed.resumable = self
    await self.fn(*self.args, **self.kwargs)
    if not self.fut.done():
      self.fut.set_result(True)
    VimResumed.resumable = None
    


def non_async_wrapper(fn):
  def main(*args, **kwargs):
    Resumeable(fn, *args, **kwargs)
  return main


@non_async_wrapper
async def fn():
  print('WAITING')
  await vim_resumed()
  print('RESUMED')
  print('DONE')
  
print('START')
fn()
print('VIM DOES ITS STUFF')
resume()
print('ENDED')
  

...Note that next-asyncio is a single file, BSD-2 so it could be provided directly inside Ultisnips, or cloned as a git submodule

[SirVer]

Such a refactor could be really nice, maybe UltiSnips could move towards being more Editor agnostic too in such a move which would be fantastic. The question is the cost of such a dependency, i.e. we would probably loose support for older python version doing this which might or might not be okay. The current LTS of ubuntu (18.04) has python 3.6, which was usually my guidepost for support for old python versions. Last time I dropped python 3.5 support, people were not happy (#1240, #1208).

[hl037]

This is why I called it "Ideas for a future major version".

Actually, such code could be made 3.6-compatible by droping the usage of the async/await keywords, at the cost of readability. However, it would be very nice if ubuntu moved at least to 3.7...

I had not though about being editor independence, but indeed now you say it, such a move could make it easier. However, I myself don't know much other editors, except Qt Creator and Kate-part (I was never able to use emacs^^').

[SirVer]

I do not think this needs to wait for a major version, in fact this is just an implementation detail, i.e. the user will not care about it. However, the concept of coroutines here is a very attractive one, it would simplify the code quite a bit I think. It feels like a fairly large refactoring though, so not something to be taken on lightly.

[hl037]

Actually, such code could be made 3.6-compatible by droping the usage of the async/await keywords, at the cost of readability. However, it would be very nice if ubuntu moved at least to 3.7...

I checked again the documentation... it is already available in python 3.5... https://docs.python.org/3.5/library/asyncio-task.html ...But I don't know if the nested loop package I provided is compatible

[hl037]

I am quite busy too right now... sorry for the lack of activity...

I think the async feature could help with other vim plugins... I think it's a good idea to provided it as a third-party helper lib. I have just started to implement it. I will link here the repo once I get a first working code.

[SirVer]

if it is a third-party helper we will need to vendor it into UltiSnips though (which is fine from my point of view), because dealing with python dependencies in a plugin for Vim is a nightmare.

[hl037]

I was actually more thinking about a "vim plugin" dependency, This way, it could be integrated in several ways :

• Asking the user to add it too ( two "Plug" lines)
• Providing it as a git-submodule, and tweaking the runtime path
• An "init" script that clone the sub-module, and copy paste it in the UltiSnips tree.

(By the way, it would be really nice if vim plugin managers supported dependencies, because I have a strong feeling many plugins reinvent the wheel, and get their code more and more complex...)

I agree, a python general dependency is not acceptable because it add a complete step to the installation

[rebelot]

what about parsing LSP style snippet format? With more neovim users taking advantage of integrated LSP, there's an interest in increasing compatibility with existing vscode extensions. (i.e. vim-vsnip is doing that).

Also, if I grasped that making ultisnips editor agnostic would be a good idea, I would lean towards LSP snippet format. Usually (neo)Vim users will have a completion plugin, and modern ones (at least on neovim) all deal with LSP protocol. Also, "adapter" plugins exist, which mock an LSP providing snippets and completions (i.e. null-ls). As of now, finding a working combination of completion-engine/snippet-plugin/language-extension that won't require crazy config overhead is not so trivial, but IMHO this is what users (including myself) expect from an editor in 2021.