luatest icon indicating copy to clipboard operation
luatest copied to clipboard

Not all assert fails from `:exec` parsed correctly

Open DifferentialOrange opened this issue 1 year ago • 0 comments

Run with luatest master https://github.com/tarantool/luatest/commit/18859f6a67d7f92bda12297869d56928d9d7dc88.

Test preset

Consider the following test.lua:

local t = require('luatest')
local g = t.group('integration')

local helper = require('helper')

g.before_all(function(cg)
    cg.server = t.Server:new({
        env = {
            LUA_PATH = helper.LUA_PATH,
        }
    })

    cg.server:start{wait_until_ready = true}
end)

g.after_all(function(cg)
    cg.server:drop()
end)

with the following helper.lua:

local fio = require('fio')

local helper = {}

local root = fio.dirname(fio.abspath(package.search('helper')))

helper.LUA_PATH = root .. '/?.lua;' ..
    root .. '/?/init.lua;' ..
    root .. '/.rocks/share/tarantool/?.lua;' ..
    root .. '/.rocks/share/tarantool/?/init.lua'

return helper

Bad case

If something complicated is unsuccessfully asserted:

g.test_exec = function(cg)
    cg.server:exec(function()
        local json = require("json")
        local body = '{"Space":{"customers":{"Format":[{"Name":"id","Type":"unsigned","IsNullable":false},' ..
                     '{"Name":"bucket_id","Type":"unsigned","IsNullable":false},{"Name":"name","Type":"str' ..
                     'ing","IsNullable":false},{"Name":"age","Type":"number","IsNullable":false}]}}}'
        t.assert_equals(json.decode(body), {
                customers = {
                    format = {
                        {name = 'id', type = 'unsigned', is_nullable = false},
                        {name = 'bucket_id', type = 'unsigned', is_nullable = false},
                        {name = 'name', type = 'string', is_nullable = false},
                        {name = 'age', type = 'number', is_nullable = false},
                    },
                },
            })
    end)
end

test errors (E) and luatest output is raw:

./.rocks/bin/luatest ./test.lua 
Tarantool version is 2.11.0-0-g247a9a418
E

Tests with errors:
------------------

1) integration.test_exec
{"status":"fail","class":"LuatestError","message":"...iseevgeorgy\/Development\/sandbox\/luatest-repro\/\/\/test.lua:26: expected: \n{\n    customers = {\n        format = {\n            {is_nullable = false, name = \"id\", type = \"unsigned\"},\n            {is_nullable = false, name = \"bucket_id\", type = \"unsigned\"},\n            {is_nullable = false, name = \"name\", type = \"string\"},\n            {is_nullable = false, name = \"age\", type = \"number\"},\n        },\n    },\n}\nactual: \n{\n    Sp
stack traceback:
	...iseevgeorgy/Development/sandbox/luatest-repro///test.lua:21: in function 'integration.test_exec'
	...
	[C]: in function 'xpcall'
artifacts:
	server -> /tmp/t/artifacts/server-KOcN_5Jwubog
Ran 1 tests in 0.233 seconds, 0 succeeded, 1 errored

Failed tests:

integration.test_exec

Good case

At the same time, if a simple assert fails inside the :exec:

g.test_exec = function(cg)
    cg.server:exec(function()
        t.assert_equals(1, 2)
    end)
end

test fails (F) and luatest output is pretty:

./.rocks/bin/luatest ./test.lua 
Tarantool version is 2.11.0-0-g247a9a418
F

Failed tests:
-------------

1) integration.test_exec
...iseevgeorgy/Development/sandbox/luatest-repro///test.lua:41: expected: 2, actual: 1
stack traceback:
	...iseevgeorgy/Development/sandbox/luatest-repro///test.lua:40: in function 'integration.test_exec'
	...
	[C]: in function 'xpcall'
artifacts:
	server -> /tmp/t/artifacts/server-JEXyZRyZ9t6u
Ran 1 tests in 0.235 seconds, 0 succeeded, 1 failed

Failed tests:

integration.test_exec

DifferentialOrange avatar Sep 26 '23 11:09 DifferentialOrange