scryer-prolog get_char/2 bugs

With cea1353fbb0ae7966e99fcb4eb1210ccb20a4712 and https://github.com/LogtalkDotOrg/logtalk3/blob/master/tests/prolog/predicates/get_char_2/tests.lgt:

$ logtalk_tester -p scryer -g "set_logtalk_flag(clean,off)" -w
% Batch testing started @ 2022-05-12 08:19:20
%         Logtalk version: 3.56.0-b01
%         Scryer Prolog version: 0.9.0
%
% logtalk/tests/prolog/predicates/get_char_2
%         22 tests: 0 skipped, 8 passed, 14 failed (0 flaky)
%         completed tests from object tests in 7 seconds
%         clause coverage n/a
%
% Compilation errors/warnings and failed unit tests
% (compilation errors/warnings might be expected depending on the test)
!     iso_get_char_2_01: failure (in 0.0 seconds)
!       test goal failed but should have succeeded
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 34-42
!     iso_get_char_2_02: failure (in 0.0 seconds)
!       test goal failed but should have succeeded
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 42-48
!     iso_get_char_2_03: failure (in 0.0 seconds)
!       test goal failed but should have succeeded
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 48-54
!     iso_get_char_2_04: failure (in 0.0 seconds)
!       test goal failed but should have succeeded
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 54-59
!     iso_get_char_2_06: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(permission_error(input,stream,user_output),A)
!         but got  error(permission_error(input,stream,user_output),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 66-69
!     sics_get_char_2_08: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(type_error(in_character,1),A)
!         but got  error(type_error(character,1),can_be/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 74-78
!     sics_get_char_2_09: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(type_error(in_character,1),A)
!         but got  error(type_error(character,1),can_be/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 78-82
!     sics_get_char_2_11: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(existence_error(stream,A),B)
!         but got  error(domain_error(stream_or_alias,'$dropped_value'),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 86-90
!     sics_get_char_2_12: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(existence_error(stream,A),B)
!         but got  error(domain_error(stream_or_alias,'$dropped_value'),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 90-94
!     sics_get_char_2_16: failure (in 0.0 seconds)
!       test goal throws an error but should have succeeded: error(type_error(character,end_of_file),can_be/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 109-117
!     lgt_get_char_2_18: failure (in 0.0 seconds)
!       test goal throws an error but should have succeeded: error(type_error(character,end_of_file),can_be/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 123-131
!     lgt_get_char_2_19: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(permission_error(input,stream,s),A)
!         but got  error(permission_error(input,stream,s),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 131-135
!     lgt_get_char_2_21: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(type_error(in_character,ab),A)
!         but got  error(type_error(character,ab),can_be/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 139-143
!     lgt_get_char_2_22: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(type_error(in_character,ab),A)
!         but got  error(type_error(character,ab),can_be/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 143-147
%
% Failed tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - iso_get_char_2_01 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - iso_get_char_2_02 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - iso_get_char_2_03 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - iso_get_char_2_04 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - iso_get_char_2_06 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - sics_get_char_2_08 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - sics_get_char_2_09 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - sics_get_char_2_11 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - sics_get_char_2_12 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - sics_get_char_2_16 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - lgt_get_char_2_18 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - lgt_get_char_2_19 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - lgt_get_char_2_21 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - lgt_get_char_2_22 @ tests
%
% 1 test sets: 1 completed, 0 skipped, 0 broken, 0 timedout, 0 crashed
% 22 tests: 0 skipped, 8 passed, 14 failed (0 flaky)
%
% Batch testing ended @ 2022-05-12 08:19:30

Logtalk's lgtunit uses get_char/2 in its predicates that simplify testing I/O, so type_error(character,end_of_file) at the sics_get_char_2_16 and lgt_get_char_2_18 tests is specially problematic.

Also, the failure of the iso_get_char_2_06 abd lgt_get_char_2_19 tests exposes a bug likely in subsumes_term/2, which is used to check that the expected error subsumes the actual error. This is an issue that affects tests for other predicates as well.

May 12 '22 07:05 pmoura

Thanks for the latest fixes. With 7a9b71cc03018ebea9fde0529d6d10a484626e7b:

$ logtalk_tester -p scryer -g "set_logtalk_flag(clean,off)" -w -t 480 -s $HOME/
% Batch testing started @ 2022-05-23 09:25:06
%         Logtalk version: 3.56.0-b09
%         Scryer Prolog version: 0.9.0
%
% logtalk/tests/prolog/predicates/get_char_2
%         22 tests: 0 skipped, 18 passed, 4 failed (0 flaky)
%         completed tests from object tests in 15 seconds
%         clause coverage n/a
%
% Compilation errors/warnings and failed unit tests
% (compilation errors/warnings might be expected depending on the test)
!     iso_get_char_2_06: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(permission_error(input,stream,user_output),A)
!         but got  error(permission_error(input,stream,user_output),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 66-69
!     sics_get_char_2_11: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(existence_error(stream,A),B)
!         but got  error(domain_error(stream_or_alias,'$dropped_value'),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 86-90
!     sics_get_char_2_12: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(existence_error(stream,A),B)
!         but got  error(domain_error(stream_or_alias,'$dropped_value'),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 90-94
!     lgt_get_char_2_19: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(permission_error(input,stream,s),A)
!         but got  error(permission_error(input,stream,s),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 131-135
%
% Failed tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - iso_get_char_2_06 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - sics_get_char_2_11 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - sics_get_char_2_12 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - lgt_get_char_2_19 @ tests
%
% 1 test sets: 1 completed, 0 skipped, 0 broken, 0 timedout, 0 crashed
% 22 tests: 0 skipped, 18 passed, 4 failed (0 flaky)
%
% Batch testing ended @ 2022-05-23 09:25:25

Likely all domain_error(stream_or_alias,'$dropped_value') could be changed to existence_error(stream,Stream) exceptions?

The other two remaining failures are due to the implicit subsumes_term/2 unexpectedly failing.

May 23 '22 08:05 pmoura

I'm not able to reproduce the subsumes_terms/2 bug.

May 23 '22 17:05 mthom

Odd. Are you trying to reproduce those bugs just by typing the test goals at the top-level or by running the Logtalk tests? With 084fc845902f7b435b0e5e44aaeba023bc28958f, I still get:

$ logtalk_tester -p scryer -g "set_logtalk_flag(clean,off)" -w -t 480 -s $HOME/
% Batch testing started @ 2022-05-23 18:42:46
%         Logtalk version: 3.56.0-b09
%         Scryer Prolog version: 0.9.0
%
% logtalk/tests/prolog/predicates/get_char_2
%         22 tests: 0 skipped, 20 passed, 2 failed (0 flaky)
%         completed tests from object tests in 19 seconds
%         clause coverage n/a
%
% Compilation errors/warnings and failed unit tests
% (compilation errors/warnings might be expected depending on the test)
!     iso_get_char_2_06: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(permission_error(input,stream,user_output),A)
!         but got  error(permission_error(input,stream,user_output),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 66-69
!     lgt_get_char_2_19: failure (in 0.0 seconds)
!       test goal throws the wrong error:
!         expected error(permission_error(input,stream,s),A)
!         but got  error(permission_error(input,stream,s),get_char/2)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/get_char_2/tests.lgt between lines 131-135
%
% Failed tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - iso_get_char_2_06 @ tests
logtalk/tests/prolog/predicates/get_char_2/tests.lgt - lgt_get_char_2_19 @ tests
%
% 1 test sets: 1 completed, 0 skipped, 0 broken, 0 timedout, 0 crashed
% 22 tests: 0 skipped, 20 passed, 2 failed (0 flaky)
%
% Batch testing ended @ 2022-05-23 18:43:09

@mthom I'm compiling Scryer on macOS using, I believe, an up-to-date version of Rust.

May 23 '22 17:05 pmoura

Are you trying to reproduce those bugs just by typing the test goals at the top-level or by running the Logtalk tests?

Typing them at the top-level. I manually reproduced some of the logtalk tests by importing some lgtunit predicates into Scryer.

May 24 '22 22:05 mthom

Are you trying to reproduce those bugs just by typing the test goals at the top-level or by running the Logtalk tests?

Typing them at the top-level. I manually reproduced some of the logtalk tests by importing some lgtunit predicates into Scryer.

When executing the test goals at the top-level, the call stack is not the same. E.g. lgtunit (necessarily) wraps the test goal using a catch/3 construct to capture and check any errors. That likely explains the different results as the environment is not the same. You may get some insight on the bug by instrumenting the subsumes_term/2 implementation and then running one of the tests that fail:

$ cd logtalk3/tests/prolog/predicates/get_char_2
$ scryerlgt
...
?- {tester}.
...
?- tests::run(iso_get_char_2_06).
...

May 24 '22 22:05 pmoura

Do these issues still arise with the latest git version?

Jul 04 '23 21:07 triska

I tried them just now and they pass.

Jul 12 '23 19:07 mthom

With 3f5dbc16804ca349bf79a8d8f58d067c303f948f, I get:

$ logtalk_tester -p scryer
% Batch testing started @ 2023-07-12 20:59:01
%         Logtalk version: 3.68.0-b01
%         Scryer Prolog version: 0.9.1
%
% Documents/Logtalk/logtalk3/tests/prolog/predicates/get_char_2
%         22 tests: 0 skipped, 22 passed, 0 failed (0 flaky)
%         completed tests from object tests in 5 seconds
%         clause coverage n/a
%
% 1 test sets: 1 completed, 0 skipped, 0 broken, 0 timedout, 0 crashed
% 22 tests: 0 skipped, 22 passed, 0 failed (0 flaky)
%
% Batch testing ended @ 2023-07-12 20:59:08

$ cd ../peek_char_2/
✔ ~/Documents/Logtalk/logtalk3/tests/prolog/predicates/peek_char_2 [master|✚ 1…73⚑ 65] 
21:00 $ logtalk_tester -p scryer
% Batch testing started @ 2023-07-12 21:00:28
%         Logtalk version: 3.68.0-b01
%         Scryer Prolog version: 0.9.1
%
% Documents/Logtalk/logtalk3/tests/prolog/predicates/peek_char_2
%         21 tests: 0 skipped, 21 passed, 0 failed (0 flaky)
%         completed tests from object tests in 5 seconds
%         clause coverage n/a
%
% 1 test sets: 1 completed, 0 skipped, 0 broken, 0 timedout, 0 crashed
% 21 tests: 0 skipped, 21 passed, 0 failed (0 flaky)
%
% Batch testing ended @ 2023-07-12 21:00:34

But:

% iso_put_char_2_01: success (in 0.000000000 seconds)
% iso_put_char_2_02: success (in 0.000000000 seconds)
% iso_put_char_2_03: success (in 0.000000000 seconds)
% iso_put_char_2_04: success (in 0.000000000 seconds)
% iso_put_char_2_05: success (in 0.000000000 seconds)
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/machine/system_calls.rs:3105:44
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I will create a separate issue.

Jul 12 '23 20:07 pmoura

scryer-prolog scryer-prolog copied to clipboard

get_char/2 bugs

scryer-prolog
scryer-prolog copied to clipboard