Ultest does nothing silently

[wookayin]

Describe the bug

When I run :UltestNearest, or any commands that would run test cases, it does nothing and no error messages are produced.

Log files:

00:13:40 | INFO | MainThread | logging.py:create_logger:101 | Logger created                                                                                                                                                                                                            
00:13:40 | DEBUG | MainThread | __init__.py:__init__:44 | Handler created
00:13:40 | INFO | MainThread | tracker.py:_init_test_file:97 | Initialising test file H$OME/SOMEPROJECT/data_test.py
00:13:40 | DEBUG | MainThread | __init__.py:_handle_coroutine:60 | Starting job with group update_positions
00:13:40 | INFO | MainThread | tracker.py:_async_update:57 | Updating positions in $HOME/SOMEPROJECT/data_test.py
00:13:40 | DEBUG | MainThread | file.py:parse_file_structure:25 | Converted pattern {'test': ['\\v^\\s*%(async )?def (test_\\w+)'], 'namespace': ['\\v^\\s*class (\\w+)']} to {'test': [re.compile('^\\s*(?:async )?def (test_\\w+)')], 'namespace': [re.compile('^\\s*class (\\w+)')]}
00:13:40 | DEBUG | MainThread | tracker.py:_async_update:84 | New test $HOME/SOMEPROJECT found in $HOME/SOMEPROJECT/data_test.py
00:13:40 | DEBUG | MainThread | tracker.py:_remove_old_positions:129 | No tests removed
00:13:40 | DEBUG | MainThread | __init__.py:_handle_coroutine:80 | Finished job with group update_positions
00:13:42 | INFO | MainThread | __init__.py:run_nearest:128 | Running nearest test in SOMEPROJECT/data_test.py at line 67

:UltestSummary also does not print anything -- it shows nothing but the file name.

? SOMEPROJECT/data_test.py

To Reproduce My plugin config:

" Tests                                                        
Plug 'vim-test/vim-test'                                       
Plug 'rcarriga/vim-ultest', { 'do': function('UpdateRemote') } 

I did not have any other configurations.

Expected behavior

Something runs, or error messages are produced.

Screenshots

N/A

Additional context

• vim-test commands :TestNearest works fine, running in a new tab.
• I have a normal python3 environments.
• This happened for me both on Linux and MacOS.
• neovim 0.6.1 release and NVIM v0.7.0-dev+963-g082ff2190 (nightly)

Where should I start troubleshooting?

[wookayin]

There are two bugs around.

With the following test suite:

import sys
import pytest
class TestA:
  def testFoo(self):                             
    assert False
  def test_bar(self):       
    assert False

It can detect only one test method because the method is using camel-case naming convention:

✖ a_test.py
  ✖ TestA          
    ? test_bar     

which should be also detected, as in plain pytest runnre and in vim-test.

Also, when there is no test method detected in any either case, it should say informative some error message. Hope this helps. Or any message about progress -- e.g., test is running, has completed.

[wookayin]

For the first one, test pattern are specified in g:test#python#patterns (or g:ultest_patterns). This can be confirmed by checking:

echo ultest#adapter#get_patterns(CURRENT_FILE_NAME)

For python, we by default have test_ pattern, but this can be configured as follows:

let g:test#python#patterns = {
-    \ 'test': ['\v^\s*%(async )?def (test_\w+)'],
+    \ 'test': ['\v^\s*%(async )?def (test\w+)'],
     \ 'namespace': ['\v^\s*class (\w+)']
     \}  

It was not very obvious to figure out. It would be great to add some warning messages when no tests were detected, and/or execution of test operations.

[rcarriga]

Hi thanks for looking into this.

For the first one, test pattern are specified in g:test#python#patterns (or g:ultest_patterns).

This is actually not correct, using g:ultest_patterns is purely for runners which do not use pattern matching and so ultest needs custom patterns to find them. vim-test will not use these patterns to find the tests and so there will be inconsistency.

Also, when there is no test method detected in any either case, it should say informative some error message. Hope this helps. Or any message about progress -- e.g., test is running, has completed.

I don't think is necessary since the summary already shows what has been detected. There is no further information that vim-ultest can show.

For your actual problem of camel case names, this has come up before and there's nothing vim-ultest can do as this is a vim-test problem (See https://github.com/rcarriga/vim-ultest/issues/39#issuecomment-843503282).

To demonstrate this is vim-test you can use this file

def test_a():
    assert True
    
def testB():
    assert False

and run :TestNearest on the last line. It will pass because it ran test_a.