Showing covered/not covered instances

[hakank]

When debugging a model it would be helpful to see which instances that are covered/not covered after the (tentative) programs.

Here is a (fake) example for examples/alleven to show what I mean:

% NEW BEST PROG 1:
f(A):-empty(A).
% Precision:n/a, Recall:0.00, TP:0, FN:4, TN:5, FP:0

TP:
-

FN:
-

TN: 
neg(f([78, 49])).
neg(f([78, 49, 10])).
neg(f([78, 2, 49, 3])).
neg(f([4, 78, 2, 49, 3])).
neg(f([62, 10,20,30,4,2,8,23,44])).

FP:
pos(f([34, 62, 10])).
pos(f([62, 10])).
pos(f([62, 10, 2, 18, 4])).
pos(f([62, 10,20,30,4,2,8,22,44])).





% NEW BEST PROG 7:
f(A):-last(A,B),even(B).
% Precision:0.67, Recall:1.00, TP:4, FN:0, TN:3, FP:2

TP:
pos(f([34, 62, 10])).
pos(f([62, 10])).


FN:
neg(f([4, 78, 2, 49, 3])).
neg(f([62, 10,20,30,4,2,8,23,44])).


TN: 
neg(f([78, 49])).
neg(f([78, 49, 10])).
neg(f([78, 2, 49, 3])).

FP:
pos(f([62, 10, 2, 18, 4])).
pos(f([62, 10,20,30,4,2,8,22,44])).


% BEST PROG 125:
f(A):-empty(A).
f(A):-head(A,B),tail(A,C),even(B),f(C).
% Precision:1.00, Recall:1.00, TP:4, FN:0, TN:5, FP:0

TP:
pos(f([34, 62, 10])).
pos(f([62, 10])).
pos(f([62, 10, 2, 18, 4])).
pos(f([62, 10,20,30,4,2,8,22,44])).


FN:
-

TN: 
neg(f([78, 49])).
neg(f([78, 49, 10])).
neg(f([78, 2, 49, 3])).
neg(f([4, 78, 2, 49, 3])).
neg(f([62, 10,20,30,4,2,8,23,44])).

FP:
-

[andrewcropper]

That could be helpful when there are only a small number of examples, but could become cumbersome with many examples. I'll look into adding an extra --super-debug flag which would give that additional information.

[hakank]

That's would be really great!

And I agree that this is only relevant when using a small amount of examples (and when one have a good intuition about the correct solution).