neovim-qt icon indicating copy to clipboard operation
neovim-qt copied to clipboard

cmake test fail for 0.2.17

Open xyproto opened this issue 1 year ago • 7 comments

Hi,

I'm packaging 0.2.17 for Arch Linux, but one of the tests seems to fail now:

==> Starting check()...
ninja: Entering directory `build'
[0/1] cd /build/neovim-qt/src/build && /usr/bin/ctest --force-new-ctest-process
Test project /build/neovim-qt/src/build
      Start  1: tst_neovimobject
 1/20 Test  #1: tst_neovimobject .................   Passed    1.53 sec
      Start  2: tst_neovimconnector
 2/20 Test  #2: tst_neovimconnector ..............   Passed   10.82 sec
      Start  3: tst_callallmethods
 3/20 Test  #3: tst_callallmethods ...............   Passed    2.01 sec
      Start  4: tst_encoding
 4/20 Test  #4: tst_encoding .....................   Passed    0.04 sec
      Start  5: tst_msgpackiodevice
 5/20 Test  #5: tst_msgpackiodevice ..............   Passed    0.04 sec
      Start  6: tst_shell
 6/20 Test  #6: tst_shell ........................   Passed    3.85 sec
      Start  7: tst_main
 7/20 Test  #7: tst_main .........................   Passed    0.07 sec
      Start  8: tst_qsettings
 8/20 Test  #8: tst_qsettings ....................***Failed    0.34 sec
      Start  9: tst_input_mac
 9/20 Test  #9: tst_input_mac ....................   Passed    0.05 sec
      Start 10: tst_input_unix
10/20 Test #10: tst_input_unix ...................   Passed    0.04 sec
      Start 11: tst_input_win32
11/20 Test #11: tst_input_win32 ..................   Passed    0.04 sec
      Start 12: tst_input_common_mac
12/20 Test #12: tst_input_common_mac .............   Passed    0.03 sec
      Start 13: tst_input_common_unix
13/20 Test #13: tst_input_common_unix ............   Passed    0.03 sec
      Start 14: tst_input_common_win32
14/20 Test #14: tst_input_common_win32 ...........   Passed    0.04 sec
      Start 15: test_cell
15/20 Test #15: test_cell ........................   Passed    1.02 sec
      Start 16: test_shellcontents
16/20 Test #16: test_shellcontents ...............   Passed    0.91 sec
      Start 17: test_shellwidget
17/20 Test #17: test_shellwidget .................   Passed    0.06 sec
      Start 18: bench_scroll
18/20 Test #18: bench_scroll .....................   Passed    0.34 sec
      Start 19: bench_cell
19/20 Test #19: bench_cell .......................   Passed    0.36 sec
      Start 20: bench_paint
20/20 Test #20: bench_paint ......................   Passed    1.02 sec

95% tests passed, 1 tests failed out of 20

Total Test time (real) =  22.65 sec

The following tests FAILED:
	  8 - tst_qsettings (Failed)
Errors while running CTest
Output from these tests are in: /build/neovim-qt/src/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
FAILED: CMakeFiles/test.util
cd /build/neovim-qt/src/build && /usr/bin/ctest --force-new-ctest-process
ninja: build stopped: subcommand failed.

Here is the PKGBUILD file I'm working on:

pkgname=neovim-qt
pkgver=0.2.17
pkgrel=1
pkgdesc='GUI for Neovim'
arch=(x86_64)
url='https://github.com/equalsraf/neovim-qt'
license=(ISC)
depends=(neovim qt5-svg hicolor-icon-theme)
makedepends=(cmake git ninja xorg-server-xvfb)
source=("git+$url#commit=3f05de82ecb5c1a24a7572081ae59e419eb059b8") # tag: v0.2.17
b2sums=('SKIP')

build() {
  cmake \
    -B build \
    -D CMAKE_BUILD_TYPE=Release \
    -D CMAKE_C_FLAGS="$CFLAGS" \
    -D CMAKE_CXX_FLAGS="$CXXFLAGS" \
    -D CMAKE_EXE_LINKER_FLAGS="$LDFLAGS" \
    -D CMAKE_INSTALL_LIBDIR=lib \
    -D CMAKE_INSTALL_PREFIX=/usr \
    -D USE_SYSTEM_MSGPACK=on \
    -G Ninja \
    -S $pkgname
  ninja -C build
}

check() {
  xvfb-run ninja -v -C build test
}

package() {
  DESTDIR="$pkgdir" ninja -C build install
  install -Dm644 -t "$pkgdir/usr/share/licenses/$pkgname" "$pkgname/LICENSE"
}

Thanks for creating neovim-qt!

xyproto avatar Jul 30 '22 09:07 xyproto

Can we get a more verbose output from the test run? Usually we can see either debug/or warning messages for each test. They usually include the line/assertion that failed.

Maybe they are available in that log file /build/neovim-qt/src/build/Testing/Temporary/LastTest.log

equalsraf avatar Jul 30 '22 18:07 equalsraf

@ xyproto

The failing test is new, tst_qsettings. I'm curious to see what is causing the failure...

If you cannot find the logs, you should be able to manually run the executable: ./build/bin/tst_qsettings

The easiest workaround is to disable tests for you PKGBUILD. The build will be much quicker without the test code and the executables don't need to be packaged anyways.

Here is an example CMake configure for a no-test build: cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DENABLE_TESTS=OFF ..

jgehrig avatar Aug 02 '22 02:08 jgehrig

The issue might be:

FAIL!  : NeovimQt::TestQSettings::GuiFont() Compared values are not the same
   Actual   (window.shell()->fontDesc()): "Monospace:h11"
   Expected (fontDesc)                  : "DejaVu Sans Mono:h20"
   Loc: [/build/neovim-qt/src/neovim-qt/test/tst_qsettings.cpp(106)]

Here is the full LastTest.log:

Start testing: Aug 04 20:13 CEST
----------------------------------------------------------
1/20 Testing: tst_neovimobject
1/20 Test: tst_neovimobject
Command: "/build/neovim-qt/src/build/bin/tst_neovimobject"
Directory: /build/neovim-qt/src/build/test
"tst_neovimobject" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestNeovimObject *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestNeovimObject::initTestCase()
PASS   : TestNeovimObject::eventTypes()
PASS   : TestNeovimObject::extDecodeApi0()
PASS   : TestNeovimObject::extDecodeApi1()
PASS   : TestNeovimObject::extDecodeApi2()
PASS   : TestNeovimObject::cleanupTestCase()
Totals: 6 passed, 0 failed, 0 skipped, 0 blacklisted, 1502ms
********* Finished testing of TestNeovimObject *********
<end of output>
Test time =   1.53 sec
----------------------------------------------------------
Test Passed.
"tst_neovimobject" end time: Aug 04 20:13 CEST
"tst_neovimobject" time elapsed: 00:00:01
----------------------------------------------------------

2/20 Testing: tst_neovimconnector
2/20 Test: tst_neovimconnector
Command: "/build/neovim-qt/src/build/bin/tst_neovimconnector"
Directory: /build/neovim-qt/src/build/test
"tst_neovimconnector" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of NeovimQt::Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : NeovimQt::Test::initTestCase()
QWARN  : NeovimQt::Test::reconnect() MsgpackIO fatal error "IO device needs to be sequential"
PASS   : NeovimQt::Test::reconnect()
PASS   : NeovimQt::Test::isReady()
QWARN  : NeovimQt::Test::encodeDecode() Encoding String into MsgpackIODevice without an encoding (defaulting to utf8)
QWARN  : NeovimQt::Test::encodeDecode() Decoding String from MsgpackIODevice without an encoding (defaulting to utf8)
PASS   : NeovimQt::Test::encodeDecode()
QWARN  : NeovimQt::Test::connectToNeovimTCP() Neovim fatal error "Connection refused"
PASS   : NeovimQt::Test::connectToNeovimTCP()
QWARN  : NeovimQt::Test::connectToNeovimSocket() Neovim fatal error "QLocalSocket::connectToServer: Invalid name"
PASS   : NeovimQt::Test::connectToNeovimSocket()
PASS   : NeovimQt::Test::connectToNeovimEnvEmpty()
QWARN  : NeovimQt::Test::connectToSocket(relative) QProcess: Destroyed while process ("nvim") is still running.
PASS   : NeovimQt::Test::connectToSocket(relative)
QWARN  : NeovimQt::Test::connectToSocket(./relative) Neovim fatal error "QLocalSocket: Remote closed"
QWARN  : NeovimQt::Test::connectToSocket(./relative) QProcess: Destroyed while process ("nvim") is still running.
PASS   : NeovimQt::Test::connectToSocket(./relative)
QWARN  : NeovimQt::Test::connectToSocket(absolute) Neovim fatal error "QLocalSocket: Remote closed"
QWARN  : NeovimQt::Test::connectToSocket(absolute) QProcess: Destroyed while process ("nvim") is still running.
PASS   : NeovimQt::Test::connectToSocket(absolute)
QWARN  : NeovimQt::Test::metadataTimeout() Neovim fatal error "QLocalSocket: Remote closed"
QWARN  : NeovimQt::Test::metadataTimeout() Request 0 timed out: 0
QWARN  : NeovimQt::Test::metadataTimeout() Neovim fatal error "Neovim is taking too long to respond"
PASS   : NeovimQt::Test::metadataTimeout()
PASS   : NeovimQt::Test::cleanupTestCase()
Totals: 12 passed, 0 failed, 0 skipped, 0 blacklisted, 10391ms
********* Finished testing of NeovimQt::Test *********
<end of output>
Test time =  10.42 sec
----------------------------------------------------------
Test Passed.
"tst_neovimconnector" end time: Aug 04 20:13 CEST
"tst_neovimconnector" time elapsed: 00:00:10
----------------------------------------------------------

3/20 Testing: tst_callallmethods
3/20 Test: tst_callallmethods
Command: "/build/neovim-qt/src/build/bin/tst_callallmethods"
Directory: /build/neovim-qt/src/build/test
"tst_callallmethods" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestCallAllMethods *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestCallAllMethods::initTestCase()
PASS   : TestCallAllMethods::vim_get_current_buffer()
PASS   : TestCallAllMethods::vim_list_runtime_paths()
PASS   : TestCallAllMethods::vim_call_function()
PASS   : TestCallAllMethods::callAll()
PASS   : TestCallAllMethods::buffer_line_count()
PASS   : TestCallAllMethods::vim_var()
PASS   : TestCallAllMethods::tabpage()
PASS   : TestCallAllMethods::cleanupTestCase()
Totals: 9 passed, 0 failed, 0 skipped, 0 blacklisted, 1928ms
********* Finished testing of TestCallAllMethods *********
<end of output>
Test time =   1.97 sec
----------------------------------------------------------
Test Passed.
"tst_callallmethods" end time: Aug 04 20:13 CEST
"tst_callallmethods" time elapsed: 00:00:01
----------------------------------------------------------

4/20 Testing: tst_encoding
4/20 Test: tst_encoding
Command: "/build/neovim-qt/src/build/bin/tst_encoding"
Directory: /build/neovim-qt/src/build/test
"tst_encoding" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestEncoding *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestEncoding::initTestCase()
PASS   : TestEncoding::encodeString()
PASS   : TestEncoding::map()
PASS   : TestEncoding::stringsAreBinaryNotUtf8()
PASS   : TestEncoding::cleanupTestCase()
Totals: 5 passed, 0 failed, 0 skipped, 0 blacklisted, 8ms
********* Finished testing of TestEncoding *********
<end of output>
Test time =   0.04 sec
----------------------------------------------------------
Test Passed.
"tst_encoding" end time: Aug 04 20:13 CEST
"tst_encoding" time elapsed: 00:00:00
----------------------------------------------------------

5/20 Testing: tst_msgpackiodevice
5/20 Test: tst_msgpackiodevice
Command: "/build/neovim-qt/src/build/bin/tst_msgpackiodevice"
Directory: /build/neovim-qt/src/build/test
"tst_msgpackiodevice" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of NeovimQt::Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : NeovimQt::Test::initTestCase()
QWARN  : NeovimQt::Test::invalidDevice() MsgpackIO fatal error "IO device needs to be sequential"
PASS   : NeovimQt::Test::invalidDevice()
PASS   : NeovimQt::Test::defaultValues()
PASS   : NeovimQt::Test::setEncoding()
QWARN  : NeovimQt::Test::setInvalidEncoding() MsgpackIO fatal error "Unsupported encoding (invalid-encoding)"
PASS   : NeovimQt::Test::setInvalidEncoding()
QWARN  : NeovimQt::Test::recvError() Encoding String into MsgpackIODevice without an encoding (defaulting to utf8)
PASS   : NeovimQt::Test::recvError()
PASS   : NeovimQt::Test::notification()
PASS   : NeovimQt::Test::request()
PASS   : NeovimQt::Test::checkVariant()
PASS   : NeovimQt::Test::msgId()
QWARN  : NeovimQt::Test::timeout() MsgpackIO fatal error "IO device needs to be sequential"
QWARN  : NeovimQt::Test::timeout() Request 0 timed out: 0
PASS   : NeovimQt::Test::timeout()
PASS   : NeovimQt::Test::cleanupTestCase()
Totals: 12 passed, 0 failed, 0 skipped, 0 blacklisted, 5ms
********* Finished testing of NeovimQt::Test *********
<end of output>
Test time =   0.04 sec
----------------------------------------------------------
Test Passed.
"tst_msgpackiodevice" end time: Aug 04 20:13 CEST
"tst_msgpackiodevice" time elapsed: 00:00:00
----------------------------------------------------------

6/20 Testing: tst_shell
6/20 Test: tst_shell
Command: "/build/neovim-qt/src/build/bin/tst_shell"
Directory: /build/neovim-qt/src/neovim-qt
"tst_shell" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of NeovimQt::TestShell *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : NeovimQt::TestShell::initTestCase()
PASS   : NeovimQt::TestShell::benchStart()
RESULT : NeovimQt::TestShell::benchStart():
     27 msecs per iteration (total: 54, iterations: 2)
PASS   : NeovimQt::TestShell::startVarsShellWidget()
PASS   : NeovimQt::TestShell::startVarsMainWindow()
PASS   : NeovimQt::TestShell::gviminit()
PASS   : NeovimQt::TestShell::guiShimCommands()
PASS   : NeovimQt::TestShell::CloseEvent(Normal Exit: q)
PASS   : NeovimQt::TestShell::CloseEvent(Exit with Code 1: cq)
PASS   : NeovimQt::TestShell::CloseEvent(Exit with Code 2: 2cq)
PASS   : NeovimQt::TestShell::CloseEvent(Exit with Code 255: 255cq)
PASS   : NeovimQt::TestShell::CloseEvent(Exit Code Overflow: 256cq)
PASS   : NeovimQt::TestShell::GetClipboard(empty *)
PASS   : NeovimQt::TestShell::GetClipboard(set *)
PASS   : NeovimQt::TestShell::GetClipboard(empty +)
PASS   : NeovimQt::TestShell::GetClipboard(set +)
PASS   : NeovimQt::TestShell::GetClipboard(paste *)
PASS   : NeovimQt::TestShell::GetClipboard(paste +)
PASS   : NeovimQt::TestShell::SetClipboard(empty *)
PASS   : NeovimQt::TestShell::SetClipboard(set *)
PASS   : NeovimQt::TestShell::SetClipboard(empty +)
PASS   : NeovimQt::TestShell::SetClipboard(set +)
PASS   : NeovimQt::TestShell::SetClipboard(paste *)
PASS   : NeovimQt::TestShell::SetClipboard(paste +)
PASS   : NeovimQt::TestShell::cleanupTestCase()
Totals: 24 passed, 0 failed, 0 skipped, 0 blacklisted, 4023ms
********* Finished testing of NeovimQt::TestShell *********
<end of output>
Test time =   4.06 sec
----------------------------------------------------------
Test Passed.
"tst_shell" end time: Aug 04 20:13 CEST
"tst_shell" time elapsed: 00:00:04
----------------------------------------------------------

7/20 Testing: tst_main
7/20 Test: tst_main
Command: "/build/neovim-qt/src/build/bin/tst_main"
Directory: /build/neovim-qt/src/neovim-qt
"tst_main" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of NeovimQt::Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : NeovimQt::Test::initTestCase()
PASS   : NeovimQt::Test::runsWithoutDISPLAY(--help)
PASS   : NeovimQt::Test::runsWithoutDISPLAY(--version)
PASS   : NeovimQt::Test::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped, 0 blacklisted, 24ms
********* Finished testing of NeovimQt::Test *********
<end of output>
Test time =   0.06 sec
----------------------------------------------------------
Test Passed.
"tst_main" end time: Aug 04 20:13 CEST
"tst_main" time elapsed: 00:00:00
----------------------------------------------------------

8/20 Testing: tst_qsettings
8/20 Test: tst_qsettings
Command: "/build/neovim-qt/src/build/bin/tst_qsettings"
Directory: /build/neovim-qt/src/neovim-qt
"tst_qsettings" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of NeovimQt::TestQSettings *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : NeovimQt::TestQSettings::initTestCase()
PASS   : NeovimQt::TestQSettings::OptionLineGrid()
PASS   : NeovimQt::TestQSettings::OptionPopupMenu()
PASS   : NeovimQt::TestQSettings::OptionTabline()
FAIL!  : NeovimQt::TestQSettings::GuiFont() Compared values are not the same
   Actual   (window.shell()->fontDesc()): "Monospace:h11"
   Expected (fontDesc)                  : "DejaVu Sans Mono:h20"
   Loc: [/build/neovim-qt/src/neovim-qt/test/tst_qsettings.cpp(106)]
PASS   : NeovimQt::TestQSettings::GuiScrollBar()
PASS   : NeovimQt::TestQSettings::GuiTreeView()
PASS   : NeovimQt::TestQSettings::cleanupTestCase()
Totals: 7 passed, 1 failed, 0 skipped, 0 blacklisted, 277ms
********* Finished testing of NeovimQt::TestQSettings *********
<end of output>
Test time =   0.32 sec
----------------------------------------------------------
Test Failed.
"tst_qsettings" end time: Aug 04 20:13 CEST
"tst_qsettings" time elapsed: 00:00:00
----------------------------------------------------------

9/20 Testing: tst_input_mac
9/20 Test: tst_input_mac
Command: "/build/neovim-qt/src/build/bin/tst_input_mac"
Directory: /build/neovim-qt/src/build/test
"tst_input_mac" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestInputMac *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestInputMac::initTestCase()
PASS   : TestInputMac::AltSpecialCharacters()
PASS   : TestInputMac::LessThanModifierKeys()
PASS   : TestInputMac::SpecialKeys()
PASS   : TestInputMac::KeyboardLayoutUnicodeHexInput()
PASS   : TestInputMac::CtrlCaretWellFormed()
PASS   : TestInputMac::ShiftModifierLetter()
PASS   : TestInputMac::GermanKeyboardLayout()
PASS   : TestInputMac::cleanupTestCase()
Totals: 9 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms
********* Finished testing of TestInputMac *********
<end of output>
Test time =   0.04 sec
----------------------------------------------------------
Test Passed.
"tst_input_mac" end time: Aug 04 20:13 CEST
"tst_input_mac" time elapsed: 00:00:00
----------------------------------------------------------

10/20 Testing: tst_input_unix
10/20 Test: tst_input_unix
Command: "/build/neovim-qt/src/build/bin/tst_input_unix"
Directory: /build/neovim-qt/src/build/test
"tst_input_unix" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestInputUnix *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestInputUnix::initTestCase()
PASS   : TestInputUnix::LessThanModifierKeys()
PASS   : TestInputUnix::SpecialKeys()
PASS   : TestInputUnix::CtrlCaretWellFormed()
PASS   : TestInputUnix::ShiftModifierLetter()
PASS   : TestInputUnix::GermanKeyboardLayout()
PASS   : TestInputUnix::ControlSpace()
PASS   : TestInputUnix::cleanupTestCase()
Totals: 8 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms
********* Finished testing of TestInputUnix *********
<end of output>
Test time =   0.03 sec
----------------------------------------------------------
Test Passed.
"tst_input_unix" end time: Aug 04 20:13 CEST
"tst_input_unix" time elapsed: 00:00:00
----------------------------------------------------------

11/20 Testing: tst_input_win32
11/20 Test: tst_input_win32
Command: "/build/neovim-qt/src/build/bin/tst_input_win32"
Directory: /build/neovim-qt/src/build/test
"tst_input_win32" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestInputWin32 *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestInputWin32::initTestCase()
PASS   : TestInputWin32::LessThanModifierKeys()
PASS   : TestInputWin32::SpecialKeys()
PASS   : TestInputWin32::CtrlCaretWellFormed()
PASS   : TestInputWin32::ShiftModifierLetter()
PASS   : TestInputWin32::GermanKeyboardLayout()
PASS   : TestInputWin32::FrenchBepoKeyboardLayout()
PASS   : TestInputWin32::cleanupTestCase()
Totals: 8 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms
********* Finished testing of TestInputWin32 *********
<end of output>
Test time =   0.03 sec
----------------------------------------------------------
Test Passed.
"tst_input_win32" end time: Aug 04 20:13 CEST
"tst_input_win32" time elapsed: 00:00:00
----------------------------------------------------------

12/20 Testing: tst_input_common_mac
12/20 Test: tst_input_common_mac
Command: "/build/neovim-qt/src/build/bin/tst_input_common_mac"
Directory: /build/neovim-qt/src/build/test
"tst_input_common_mac" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestInputCommon *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestInputCommon::initTestCase()
PASS   : TestInputCommon::LessThanKey()
PASS   : TestInputCommon::ModifierOnlyKeyEventsIgnored()
PASS   : TestInputCommon::VolumeKeysIgnored()
PASS   : TestInputCommon::ShiftKeyEventWellFormed()
PASS   : TestInputCommon::CapsLockIgnored()
PASS   : TestInputCommon::AltGrAloneIgnored()
PASS   : TestInputCommon::AltGrKeyEventWellFormed()
PASS   : TestInputCommon::ShiftSpaceWellFormed()
PASS   : TestInputCommon::ShiftBackSpaceWellFormed()
PASS   : TestInputCommon::MouseLeftClick()
PASS   : TestInputCommon::MouseRightClick()
PASS   : TestInputCommon::MouseMiddleClick()
PASS   : TestInputCommon::cleanupTestCase()
Totals: 14 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms
********* Finished testing of TestInputCommon *********
<end of output>
Test time =   0.03 sec
----------------------------------------------------------
Test Passed.
"tst_input_common_mac" end time: Aug 04 20:13 CEST
"tst_input_common_mac" time elapsed: 00:00:00
----------------------------------------------------------

13/20 Testing: tst_input_common_unix
13/20 Test: tst_input_common_unix
Command: "/build/neovim-qt/src/build/bin/tst_input_common_unix"
Directory: /build/neovim-qt/src/build/test
"tst_input_common_unix" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestInputCommon *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestInputCommon::initTestCase()
PASS   : TestInputCommon::LessThanKey()
PASS   : TestInputCommon::ModifierOnlyKeyEventsIgnored()
PASS   : TestInputCommon::VolumeKeysIgnored()
PASS   : TestInputCommon::ShiftKeyEventWellFormed()
PASS   : TestInputCommon::CapsLockIgnored()
PASS   : TestInputCommon::AltGrAloneIgnored()
PASS   : TestInputCommon::AltGrKeyEventWellFormed()
PASS   : TestInputCommon::ShiftSpaceWellFormed()
PASS   : TestInputCommon::ShiftBackSpaceWellFormed()
PASS   : TestInputCommon::MouseLeftClick()
PASS   : TestInputCommon::MouseRightClick()
PASS   : TestInputCommon::MouseMiddleClick()
PASS   : TestInputCommon::cleanupTestCase()
Totals: 14 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms
********* Finished testing of TestInputCommon *********
<end of output>
Test time =   0.03 sec
----------------------------------------------------------
Test Passed.
"tst_input_common_unix" end time: Aug 04 20:13 CEST
"tst_input_common_unix" time elapsed: 00:00:00
----------------------------------------------------------

14/20 Testing: tst_input_common_win32
14/20 Test: tst_input_common_win32
Command: "/build/neovim-qt/src/build/bin/tst_input_common_win32"
Directory: /build/neovim-qt/src/build/test
"tst_input_common_win32" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of TestInputCommon *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : TestInputCommon::initTestCase()
PASS   : TestInputCommon::LessThanKey()
PASS   : TestInputCommon::ModifierOnlyKeyEventsIgnored()
PASS   : TestInputCommon::VolumeKeysIgnored()
PASS   : TestInputCommon::ShiftKeyEventWellFormed()
PASS   : TestInputCommon::CapsLockIgnored()
PASS   : TestInputCommon::AltGrAloneIgnored()
PASS   : TestInputCommon::AltGrKeyEventWellFormed()
PASS   : TestInputCommon::ShiftSpaceWellFormed()
PASS   : TestInputCommon::ShiftBackSpaceWellFormed()
PASS   : TestInputCommon::MouseLeftClick()
PASS   : TestInputCommon::MouseRightClick()
PASS   : TestInputCommon::MouseMiddleClick()
PASS   : TestInputCommon::cleanupTestCase()
Totals: 14 passed, 0 failed, 0 skipped, 0 blacklisted, 0ms
********* Finished testing of TestInputCommon *********
<end of output>
Test time =   0.03 sec
----------------------------------------------------------
Test Passed.
"tst_input_common_win32" end time: Aug 04 20:13 CEST
"tst_input_common_win32" time elapsed: 00:00:00
----------------------------------------------------------

15/20 Testing: test_cell
15/20 Test: test_cell
Command: "/build/neovim-qt/src/build/bin/test_cell"
Directory: /build/neovim-qt/src/build/src/gui/shellwidget/test
"test_cell" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : Test::initTestCase()
PASS   : Test::cellDefault()
RESULT : Test::cellDefault():
     0.0000089 msecs per iteration (total: 75, iterations: 8388608)
PASS   : Test::cellValue()
RESULT : Test::cellValue():
     0.0000090 msecs per iteration (total: 76, iterations: 8388608)
PASS   : Test::cellValueRgb()
RESULT : Test::cellValueRgb():
     0.00000661 msecs per iteration (total: 111, iterations: 16777216)
PASS   : Test::cellWidth()
PASS   : Test::cellBg()
PASS   : Test::cellOperatorEquals()
PASS   : Test::cellStyleEquivalent()
PASS   : Test::cleanupTestCase()
Totals: 9 passed, 0 failed, 0 skipped, 0 blacklisted, 1004ms
********* Finished testing of Test *********
<end of output>
Test time =   1.03 sec
----------------------------------------------------------
Test Passed.
"test_cell" end time: Aug 04 20:13 CEST
"test_cell" time elapsed: 00:00:01
----------------------------------------------------------

16/20 Testing: test_shellcontents
16/20 Test: test_shellcontents
Command: "/build/neovim-qt/src/build/bin/test_shellcontents"
Directory: /build/neovim-qt/src/build/src/gui/shellwidget/test
"test_shellcontents" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : Test::initTestCase()
PASS   : Test::cellDefault()
RESULT : Test::cellDefault():
     0.075 msecs per iteration (total: 77, iterations: 1024)
QWARN  : Test::resize() Invalid shell size -10 -10
QWARN  : Test::resize() Invalid shell size 0 0
PASS   : Test::resize()
PASS   : Test::resizeBench()
RESULT : Test::resizeBench():
     0.0000048 msecs per iteration (total: 81, iterations: 16777216)
PASS   : Test::clearRow()
PASS   : Test::clearRowBench()
RESULT : Test::clearRowBench():
     0.00083 msecs per iteration (total: 55, iterations: 65536)
PASS   : Test::clearRegion()
PASS   : Test::clearAll()
PASS   : Test::scrollRegion()
PASS   : Test::scrollRegion2()
PASS   : Test::put()
PASS   : Test::cases()
PASS   : Test::cleanupTestCase()
Totals: 13 passed, 0 failed, 0 skipped, 0 blacklisted, 875ms
********* Finished testing of Test *********
<end of output>
Test time =   0.90 sec
----------------------------------------------------------
Test Passed.
"test_shellcontents" end time: Aug 04 20:13 CEST
"test_shellcontents" time elapsed: 00:00:00
----------------------------------------------------------

17/20 Testing: test_shellwidget
17/20 Test: test_shellwidget
Command: "/build/neovim-qt/src/build/bin/test_shellwidget"
Directory: /build/neovim-qt/src/build/src/gui/shellwidget/test
"test_shellwidget" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : Test::initTestCase()
PASS   : Test::clearRegion()
PASS   : Test::fontDescriptionFromQFont()
PASS   : Test::fontDescriptionToQFont()
PASS   : Test::cleanupTestCase()
Totals: 5 passed, 0 failed, 0 skipped, 0 blacklisted, 32ms
********* Finished testing of Test *********
<end of output>
Test time =   0.06 sec
----------------------------------------------------------
Test Passed.
"test_shellwidget" end time: Aug 04 20:13 CEST
"test_shellwidget" time elapsed: 00:00:00
----------------------------------------------------------

18/20 Testing: bench_scroll
18/20 Test: bench_scroll
Command: "/build/neovim-qt/src/build/bin/bench_scroll"
Directory: /build/neovim-qt/src/build/src/gui/shellwidget/test
"bench_scroll" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : Test::initTestCase()
PASS   : Test::benchScrollRegion()
RESULT : Test::benchScrollRegion():
     0.14 msecs per iteration (total: 75, iterations: 512)
PASS   : Test::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted, 299ms
********* Finished testing of Test *********
<end of output>
Test time =   0.32 sec
----------------------------------------------------------
Test Passed.
"bench_scroll" end time: Aug 04 20:13 CEST
"bench_scroll" time elapsed: 00:00:00
----------------------------------------------------------

19/20 Testing: bench_cell
19/20 Test: bench_cell
Command: "/build/neovim-qt/src/build/bin/bench_cell"
Directory: /build/neovim-qt/src/build/src/gui/shellwidget/test
"bench_cell" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : Test::initTestCase()
PASS   : Test::benchCell()
RESULT : Test::benchCell():
     0.0000097 msecs per iteration (total: 82, iterations: 8388608)
PASS   : Test::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted, 331ms
********* Finished testing of Test *********
<end of output>
Test time =   0.36 sec
----------------------------------------------------------
Test Passed.
"bench_cell" end time: Aug 04 20:13 CEST
"bench_cell" time elapsed: 00:00:00
----------------------------------------------------------

20/20 Testing: bench_paint
20/20 Test: bench_paint
Command: "/build/neovim-qt/src/build/bin/bench_paint"
Directory: /build/neovim-qt/src/build/src/gui/shellwidget/test
"bench_paint" start time: Aug 04 20:13 CEST
Output:
----------------------------------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-builduser'
********* Start testing of Test *********
Config: Using QtTest library 5.15.5, Qt 5.15.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 12.1.0), arch unknown
PASS   : Test::initTestCase()
PASS   : Test::benchPaint()
RESULT : Test::benchPaint():
     0.00012 msecs per iteration (total: 63, iterations: 524288)
PASS   : Test::benchPaintLigatures()
RESULT : Test::benchPaintLigatures():
     0.00011 msecs per iteration (total: 59, iterations: 524288)
PASS   : Test::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped, 0 blacklisted, 1004ms
********* Finished testing of Test *********
<end of output>
Test time =   1.04 sec
----------------------------------------------------------
Test Passed.
"bench_paint" end time: Aug 04 20:13 CEST
"bench_paint" time elapsed: 00:00:01
----------------------------------------------------------

End testing: Aug 04 20:13 CEST

I'll try adding ttf-dejavu as a make dependency and see what happens.

xyproto avatar Aug 04 '22 18:08 xyproto

Adding ttf-dejavu as a make dependency makes the package build and the tests pass here.

xyproto avatar Aug 04 '22 18:08 xyproto

@xyproto

Adding ttf-dejavu as a make dependency makes the package build and the tests pass here.

I'm glad you found a solution! I'll take a look at removing this dependency, so we don't require ttf-dejavu moving forward.

I may also make -DENABLE_TESTS=OFF the default state for CMake configure.

For packaging, I would recommend disabling the tests: -DENABLE_TESTS=OFF

I can't think of any benefit to building the test and running the tests for packaging...

jgehrig avatar Aug 04 '22 18:08 jgehrig

Strange the tests were supposed to use the deja vu ttf fonts in the third-party folder.

equalsraf avatar Aug 04 '22 18:08 equalsraf

@jgehrig Thanks for the input on CMake and testing. Most Arch Linux packages doesn't include a check() function that runs tests, but it's an advantage if they do, since it's an extra safeguard and might also help when porting packages to new platforms.

There's also a checkdepends array that is used, so that people that wants to build the package from source can skip the check/test dependencies if they want.

For me, just adding ttf-dejavu to the checkdepends array solves the issue, since the tests now runs merrily within a chroot+virtual X server.

xyproto avatar Aug 04 '22 18:08 xyproto