E5108: Error executing lua ...es/neovim/.config/nvim/plugged/vim-ultest/lua/ultest.lua:60: attempt to index local 'test' (a userdata value)

[fnune]

Describe the bug

While trying to run a TypeScript (filetype typescript) test with UltestDebugNearest, I run into this:

E5108: Error executing lua ...es/neovim/.config/nvim/plugged/vim-ultest/lua/ultest.lua:60: attempt to index local 'test' (a userdata value)

The vim-ultest.log file contains this:

10:02:19 | INFO | MainThread | logging.py:create_logger:74 | Logger created
10:02:19 | DEBUG | MainThread | __init__.py:__init__:40 | Handler created

Additional context

nvim --version

NVIM v0.5.0-dev+1231-g48e805728
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/cc -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -O2 -g -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=always -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=3 -I/build/nvim/parts/nvim/build/build/config -I/build/nvim/parts/nvim/build/src -I/build/nvim/parts/nvim/build/.deps/usr/include -I/usr/include -I/build/nvim/parts/nvim/build/build/src/nvim/auto -I/build/nvim/parts/nvim/build/build/include
Compiled by root@lgw01-amd64-055

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/share/nvim"

Run :checkhealth for more info

python --version

Python 3.8.6

[fnune]

I believe this has to do with my config:

let test#strategy = "tslime"
let test#filename_modifier = ':p'

EDIT: nevermind, it doesn't (from reading the ultest.lua file).

[rcarriga]

It looks like the test file is not detected as a test file which is odd since it uses vim-test. Can you share the name of the file? That is how vim-test knows if it is a test file

[fnune]

For what it's worth it was happening on Python test files, too.

The filename is forms.service.spec.ts and the filetype is typescript.

One important detail is that this is a monorepo, and my test is in a nested package. Maybe that's why it's not figuring things out.

I solved the same problem in vim-test by forking vim-test and adding two commits:

• https://github.com/fnune/vim-test/commit/93c93e490e994f9b661b297d100c90e8e30c6e54
• https://github.com/fnune/vim-test/commit/f4b0f1c9868b81b784e696e0114694779a2626c5

I did that because vim-test also didn't think my file was a test file. Those patches above fixed it.

[fnune]

Related vim-test issue: https://github.com/vim-test/vim-test/issues/272

[rcarriga]

OK I think I see the issue. So essentially you need a way to run a test from a directory that is not the current one? That should be simple enough. I'll configure the tests to run from g:test#project_root if it exists.

For debugging with nvim-dap, that's a slightly different issue. I think you'll need to provide the project root as part of the adapter config (see https://github.com/mfussenegger/nvim-dap/blob/master/doc/dap.txt#L88) which is totally untouched by vim-ultest. Should be relatively easy to also just get the g:test#project_root variable.

[rcarriga]

So there was a number of issues that you raised so I'll address separately.

1. The test#filename_modifier variable will now be respected
2. Tests will now run with g:test#project_root as the current working directory if it exists
3. I've added g:ultest_pre_run which should be a function name, which will be called before any test is run with Ultest, UltestNearest, UltestDebugNearest and the corresponding <Plug> mappings

As I said before, you will have to manually configure nvim-dap to respect the project root.

Also the reason vim-test doesn't detect your file as a test is because it can't tell if your runner is installed. Simply explicitly state what your runner is so for example with javascript/typescript and jest

let g:test#javascript_runner = "jest"

Please let me know if anything isn't working as expected!

[fnune]

Thanks a lot for your time/attention!

As I said before, you will have to manually configure nvim-dap to respect the project root.

So setting g:test#project_root isn't enough?

I've updated my plugins but I still don't get the right results. The error I was getting previously (E5108: Error executing lua ...es/neovim/.config/nvim/plugged/vim-ultest/lua/ultest.lua:60: attempt to index local 'test' (a userdata value) no longer appears when I run UltestDebugNearest (or any of the other commands).

This is my config now:

let g:test#javascript_runner = "jest"
let g:test#python_runner = "pytest"
let g:test#filename_modifier = ':p'

I essentially don't get any output regardless of what I run, and regardless of whether I open Neovim in the tests directory or project root. Is my Neovim build too old? NVIM v0.5.0-dev+1231-g48e805728.

[fnune]

We can discard the "Neovim too old" idea. I just ran Ultest on a project that's a bit simpler and everything's working fine: this time the directory containing package.json is also the directory from which I ran Neovim.

[rcarriga]

Are you setting the g:test#project_root variable as well? You'd probably get an error if that was the issue but just in case.

Otherwise can you supply a reproduction of one of these files? I'd say python would be simplest, as I don't need a requirements file but javascript works too. I imagine there's some complication that I'm missing

[fnune]

Trying to build a reproduction.

It's unrelated but I believe this:

let g:test#javascript_runner = "jest"
let g:test#python_runner = "pytest"

Is wrong, and instead one should use:

let g:test#javascript#runner = "jest"
let g:test#python#runner = "pytest"

[fnune]

https://github.com/fnune/repro-issue-31-vim-ultest

Copying the README here:

1. Run yarn
2. Open Neovim in the root directory
3. Open packages/app/src/test_does_not_work.spec.ts
4. Run Ultest there
5. Notice there's no output

To "fix" it:

1. Add test dependencies to the root of the repo (this is a bad idea): yarn add -W -D jest typescript @types/jest
2. Confirm that the test suite now works

So the problem is that the test dependencies aren't installed in the same directory where Neovim was opened. Then vim-ultest (or vim-test) thinks this is not a test file.

[fnune]

What's weird is that I couldn't reproduce it with Python. I think it's because I use a virtual env and the dependencies are elsewhere? I built the same dir structure and it was working correctly in my attempt to reproduce the issue, but it still wasn't working on my production repo.

[fnune]

One important detail about the reproduction: running yarn creates a node_modules directory on the root of the repo but not in the workspace package directory. However, the package.json entries for the test dependencies are listed in packages/app/package.json and not package.json, which makes me think vim-ultest doesn't think the dependencies are installed (even if they are).

[rcarriga]

Trying to build a reproduction.

It's unrelated but I believe this:

let g:test#javascript_runner = "jest"
let g:test#python_runner = "pytest"

Is wrong, and instead one should use:

let g:test#javascript#runner = "jest"
let g:test#python#runner = "pytest"

Completely correct, sorry for the leading you the wrong way!

Thank you for that reproduction, that is perfect. Unfortunately it looks like the issue might be something with your config. I cloned that repo, opened the test file from the root directory and could run the test without installing anything (obviously failed due to no executable).

To be clear, vim-ultest does absolutely no resolution of runners, leaving it all to vim-test. Also with the config of let g:test#javascript#runner = "jest" vim-test will not even attempt to check for a package.json or if jest is installed.

I'm running with the latest vim-test/vim-ultest, may I ask if you are running with your fork of vim-test or the regular version?

[fnune]

I'm sorry it took me so long to answer. I must have lost the notification.

To test this, I was running the regular version of vim-test and vim-ultest.

[rcarriga]

OK I think I misunderstood before, but I've got it now. I've fully emulated how vim-test changes directory using the project root. I can confirm that with following config I have your example repo working with jest outputting correctly. You'll likely need to make it more robust to check if you're in a monorepo or not but it's a simple proof of concept anyway.

let g:test#javascript#runner = "jest"

function UltestSetRoot(file_name)
  let g:test#project_root = join((split(a:file_name, "/"))[0:1], "/")
endfunction

let g:ultest_pre_run = "UltestSetRoot"

[tlindsay]

I'm receiving the same error when I try to run RSpec tests.

Error detected while processing function ultest#run_file:
line   11: E5108: Error executing lua /Users/plindsay/.vim/plugins/vim-ultest/lua/ultest.lua:60: attempt to index local 'test' (a number value)

Setting test#project_root doesn't seem to change anything, and :TestFile succeeds, so the issue seems to be with Ultest, specifically.

NVIM v0.5.0-dev+1329-gc46d98146
Build type: Release
LuaJIT 2.1.0-beta3
Compilation: clang -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -DNDEBUG -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=3 -I/tmp/neovim-20210518-45231-72e2i3/build/config -I/tmp/neovim-20210518-45231-72e2i3/src -I/usr/local/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/opt/gettext/include -I/tmp/neovim-20210518-45231-72e2i3/build/src/nvim/auto -I/tmp/neovim-20210518-45231-72e2i3/build/include
Compiled by [email protected]

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/local/Cellar/neovim/HEAD-c46d981_2/share/nvim"

Run :checkhealth for more info

[rcarriga]

@tlindsay This seems to be a separate issue manifesting in a similar way. Could you please open a separate issue with the log file and an example file would be very helpful thanks!

[tlindsay]

@rcarriga Sorry for the noise on this one. Turns out I needed to run :UpdateRemotePlugins, and then everything started working

[fnune]

With this snippet:

let g:test#javascript#runner = "jest"

function UltestSetRoot(file_name)
  let g:test#project_root = join((split(a:file_name, "/"))[0:1], "/")
endfunction

let g:ultest_pre_run = "UltestSetRoot"

I get this when trying to run Ultest:

error caught in async handler '/home/fausto/.dotfiles/neovim/.config/nvim/plugged/vim-ultest/rplugin/python3/ultest:function:_ultest_run_
nearest [[0, 'packages/memfault-app-frontend/src/utils/crc16.test.ts']]'
Traceback (most recent call last):
  File "/home/fausto/.dotfiles/neovim/.config/nvim/plugged/vim-ultest/rplugin/python3/ultest/__init__.py", line 86, in _run_nearest
    self.handler.run_nearest(*args)
  File "/home/fausto/.dotfiles/neovim/.config/nvim/plugged/vim-ultest/rplugin/python3/ultest/handler/__init__.py", line 148, in run_neare
st
    self._runner.run(
  File "/home/fausto/.dotfiles/neovim/.config/nvim/plugged/vim-ultest/rplugin/python3/ultest/handler/runner/__init__.py", line 43, in run

    self._run_group(tree, file_tree, file_name, on_start, on_finish, env)
  File "/home/fausto/.dotfiles/neovim/.config/nvim/plugged/vim-ultest/rplugin/python3/ultest/handler/runner/__init__.py", line 159, in _r
un_group
    cmd = self._vim.sync_call("ultest#adapter#build_cmd", tree[0], scope)
  File "/home/fausto/.dotfiles/neovim/.config/nvim/plugged/vim-ultest/rplugin/python3/ultest/vim_client/__init__.py", line 127, in sync_c
all
    return self._eval(expr, sync=True)
  File "/home/fausto/.dotfiles/neovim/.config/nvim/plugged/vim-ultest/rplugin/python3/ultest/vim_client/__init__.py", line 140, in _eval
    return self._vim.eval(expr, async_=not sync)
  File "/usr/lib/python3/dist-packages/pynvim/api/nvim.py", line 295, in eval
    return self.request('nvim_eval', string, **kwargs)
  File "/usr/lib/python3/dist-packages/pynvim/api/nvim.py", line 182, in request
    res = self._session.request(name, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/pynvim/msgpack_rpc/session.py", line 104, in request
    raise self.error_wrapper(err)
pynvim.api.common.NvimError: Vim(cd):E344: Can't find directory "packages/memfault-app-frontend" in cdpath

Apparently even though project_root is now OK, some variable still has packages/memfault-app-frontend as the root...?

[fnune]

Without the snippet and only this as my config:

let g:test#javascript#runner = "jest"

The tests seem to run when running Ultest:

However, when running UltestNearest, a different picture:

It seems like it's not parsing as TypeScript but as JavaScript, so it chokes on keywords like interface and as.

The summary seems to be working fine (barring the wrong test results):

When running UltestDebug:

Error detected while processing function ultest#run_file:
line   11:
E5105: Error while calling lua chunk: ...es/neovim/.config/nvim/plugged/vim-ultest/lua/ultest.lua:91: attempt to index field 'fn' (a nil
value)
Press ENTER or type command to continue

[rcarriga]

Ah sorry about that, there was a fairly large code change recently and this functionality isn't set in stone yet so the bug went unnoticed. Please try with the latest commit and the snippet I gave previously.

I wouldn't expect it to work at all with no pre run function so I wouldn't bother trying to decipher the javascript/typescript mixup

[fnune]

That's a : character. All the tests fail with that output.

Here's my config:

let g:test#javascript#runner = "jest"

function UltestSetRoot(file_name)
  let g:test#project_root = join((split(a:file_name, "/"))[0:1], "/")
endfunction

let g:ultest_pre_run = "UltestSetRoot"

FWIW Python now fails with ERROR: file or directory not found: www/tests/unit/test_traces_api.py. pwd outputs the same directory www is in, so www's parent.

---

At this point I think it's best if I somehow increase my involvement and try to debug it with my own setup? I'm interested in seeing this succeed but I'm afraid I'm wasting a lot of your time...

[fnune]

Without g:ultest_pre_run Python works just fine. TypeScript goes back to the "tis not JavaScript" problem, but funnily enough a full run with Ultest passes all the tests (even if some shouldn't pass because I change the assertions).

[rcarriga]

Oh that quick function is very much only designed for the example repo you provided so your python repo might not be the same. The : character showing generally means that the test runner command failed to be found. Could you check the logfile agan with debug logging as before and the error should be pretty obvious.

I'm more than happy to figure this one out, I'll have need of it myself anyway but if you'd like to try debug yourself of course feel free. You can supply the env var ULTEST_DEBUG=1 and debugpy will be wait for a debugger to attach so you can step through the code as it runs. Pretty much everything of interest is happening in rplugin/python3/ultest/handler/runner/__init__.py in the _run_group method at line 148, it makes several calls to autoload/ultest/adapter.vim as well.