libyang icon indicating copy to clipboard operation
libyang copied to clipboard

`yanglint_interactive` test is unstable

Open jktjkt opened this issue 2 years ago • 2 comments

I'm occasionally getting this failure:

60/60 Test #59: yanglint_interactive ..............***Failed   10.90 sec
Tests running in interp:  /nix/store/mxcx6ldm6zsp2i9c503cxafk3nsr6wix-tcl-8.6.13/bin/tclsh
Tests located in:  /home/jkt/work/cesnet/gerrit/github/CESNET/libyang/tools/lint/tests/interactive
Tests running in:  /home/jkt/work/prog/_build/czechlight-clang16-asan-ubsan/libyang/tools/lint
Temporary files stored in /home/jkt/work/prog/_build/czechlight-clang16-asan-ubsan/libyang/tools/lint
Test files run in separate interpreters
Running tests that match:  *
Skipping test files that match:  l.*.test
Only running test files that match:  *.test
Tests began at Thu Jun 15 15:36:35 CEST 2023
add.test
clear.test
completion.test
data_default.test
data_format.test
data_in_format.test
data_merge.test
data_not_strict.test
data_operational.test
data_present.test
data_type.test
data_xpath.test
debug.test
extdata.test


==== extdata_schema_mount_tree Print tree output of a model with Schema Mount FAILED
==== Contents of test case:
ly_cmd "clear -y"
    ly_cmd "searchpath $mdir"
    ly_cmd "load modsm"
    ly_cmd "extdata $ddir/modsm_ctx_ext.xml"
    ly_cmd "print -f tree modsm" "--mp root.*--rw lfl/"
---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: >>> Check-failed timeout
    while executing
"error "$error_head timeout""
    invoked from within
"expect {
            -re "${pattern}.*\r\n${prompt}$" {}
            -re $failure_pattern {
                error "unexpected output:\n$expect_out(buf..."
    (procedure "ly_cmd" line 28)
    invoked from within
"ly_cmd "print -f tree modsm" "--mp root.*--rw lfl/""
    ("uplevel" body line 5)
    invoked from within
"uplevel 1 $script"
---- errorCode: NONE
---- Test cleanup failed:
>>> Check-failed timeout
---- errorInfo(cleanup): >>> Check-failed timeout
    while executing
"error "$error_head timeout""
    invoked from within
"expect eof"
    (procedure "ly_exit" line 3)
    invoked from within
"ly_exit"
    ("uplevel" body line 1)
    invoked from within
"uplevel 1 $cleanup"
---- errorCode(cleanup): NONE
==== extdata_schema_mount_tree FAILED

feature.test
list.test
load.test
print.test
searchpath.test

Tests ended at Thu Jun 15 15:36:46 CEST 2023
all.tcl:        Total   120     Passed  113     Skipped 6       Failed  1
Sourced 19 Test Files.
Files with failing tests: extdata.test
Number of tests skipped for each constraint:
        2       !ctest
        4       [ly_cmd_exists "debug"]


98% tests passed, 1 tests failed out of 60

Total Test time (real) =  10.91 sec

The following tests FAILED:
         59 - yanglint_interactive (Failed)

I'm not fluent in expect, which means that I won't be debugging this one myself :).

jktjkt avatar Jun 15 '23 13:06 jktjkt

Most likely, the problem is that the timeout is set low. Could you please increase the timeout, I don't know, maybe 5 seconds? Here it is: https://github.com/CESNET/libyang/blob/f048ba5277227d721e0c39b162e8e7510aeb742c/tools/lint/tests/interactive/ly.tcl#L6 Let me know if it helped.

lePici avatar Jun 15 '23 13:06 lePici

thanks, it looks non-flaky with #2048

jktjkt avatar Jun 15 '23 14:06 jktjkt