rebis-dev_...: `pio`: Unexpected IO interaction

[haijinSk]

Version:

rebis-dev_opt-in-offset-tbl-concurrency, https://github.com/mthom/scryer-prolog/tree/rebis-dev_opt-in-offset-tbl-concurrency

https://github.com/mthom/scryer-prolog/tree/60695e8cf84be5a10cdec9f6c7333dc8aaadb0e6

Creating an auxiliary file and seeing that the file is empty:

$ touch text.txt
$ cat text.txt 
$ 

A file to consult:

:- use_module(library(pio)).
:- use_module(library(dcgs)).

test :-
  phrase_from_file(seq(Str),"text.txt"),
  writeToEmptyFile(Str).

writeToEmptyFile([_|_]) :-
  write(text_in_file_already),nl.
writeToEmptyFile([]) :-
  write(writing_to_file),nl,
  phrase_to_file(seq("Hello..."),"text.txt").

The unexpected interaction (the auxiliary text file is empty):

?- test.
writing_to_file
   true
;  text_in_file_already % Unexpected "info-printing", if the auxiliary text file was empty at the beginning.
true                        
;  false.

With the empty file, the expected interaction (as with master) is like this:

?- test.
writing_to_file % As expected (only this "info-printing"), if the auxiliary text file was empty at the beginning.
   true
;  false.

Markus @triska, thank you for your answer.

[haijinSk]

In other words, with the empty text file, this:

test2(Str) :-
  phrase_from_file(seq(Str),"text.txt"),
  phrase_to_file(seq("Hello..."),"text.txt").

has two solutions:

?- test2(Str).
   Str = [] 
;  Str = "Hello..." % The unexpected solution/binding, if the text file was empty at the beginning.
;  false.

but only this: Str = [] is expected, if the text file is empty at the beginning.

[mthom]

@triska @haijinSk The issue seems to date to the changes of 025ec377. render_step is called twice in both master and rebis-dev, but in rebis-dev, Chars is unified to [H,e,l,l,o,.,.,.] the second time whereas in master it's unified to [].

[triska]

@haijinSk, @bakaq: The intention of 025ec37 was to obtain the speed for files we had at (and before) b04d845ec05b1b8b402cf61ab8df13a3d1cec742, i.e., to bypass the machinery for non-repositionable streams (for which support was added in 35d0042be1cd83914d3362fafbd1b5d90c4a155a) when possible.

There is apparently a mistake in this commit or somewhere else in Scryer and I will try to find it. I would greatly appreciate if you could also take a look at this, thank you a lot!

[triska]

I have reduced it to the following self-contained program:

:- use_module(library(pio)).
:- use_module(library(dcgs)).

test(Str) :-
        File = "testfile.out",
        phrase_to_file(seq("abc"), File),
        phrase_from_file(seq(Str), File),
        phrase_to_file(seq("123456"), File).

With master, I get:

?- test(Str).
   Str = "abc"
;  false.

Whereas with rebis-dev, I get:

?- test(Str).
   Str = "abc"
;  Str = "abc456"
;  false.

library(pio) therefore treats repositionable streams in rebis-dev different from master. And indeed it works correctly also on rebis-dev with pio.pl from before 025ec37733984b0240c319c15aaeeaacbe378be6.

UPDATE: And the same phenomenon apparently arises also with rebis-dev and pio.pl at b04d845ec05b1b8b402cf61ab8df13a3d1cec742!

[triska]

I think I may have mixed up versions in the comparison above: I now tried pio.pl from b04d845ec05b1b8b402cf61ab8df13a3d1cec742 (i.e., a version way before 025ec37733984b0240c319c15aaeeaacbe378be6) with rebis-dev, and I can reproduce the issue even there:

?- test(Str).
   Str = "abc"
;  Str = "abc456"
;  false.

This may be a hint that something in the underlying engine is wrong.

[triska]

And, as my last update for today on this specific issue, I now noticed that also with master, I get with pio.pl as it was at b04d845:

?- test(Str).
   Str = "abc"
;  Str = "abc456"
;  false.

From what I can tell so far, this entire issue may be a consequence of writing to a file while a choice-point is pending. For this reason, the next read on backtracking "sees" what an intermittent write has written starting at the previously current position of the file.

Note that seq//1 leaves a choice-point when reading from a file:

?- phrase_from_file(seq(Cs), "testfile.out").
   Cs = "123456"
;  false.

I therefore ask: Is the observed behaviour not actually what one would naturally expect upon writing to a file "during" a read?

[haijinSk]

I was thinking about it as well... If the observed "new behaviour" is natural or not... and what should be expected in a Prolog system... But I'm not the right person to know that, to decide. As always, only, I can see, or I saw, in this case, rebis-dev and then master, Trealla, and SWI Prolog. And reactions of Prolog experts on my question and report.

Naively (not thinking about how Prolog works), seeing it as imperative steps, I would expect reading from the file only once, and then writing to the file without the "interaction". To have that behaviour with the new pio, phrase_from_file/2 in once/1 can be used, as I see it.

Please, tell me, if I should close this.

[triska]

I think we have seen the following sequence with Scryer:

1. In its initial version generously contributed by @notoria, library(pio) did read the entire file at once: d6772c5ca5359d9e5a8de15a110e10408e213275.
2. With the move to reading the file lazily (40c3e31955598bb37e516b777ca30c6d6ac2ad93), the issue here observed already arose.
3. With the awesome generalization contributed by @bakaq in 35d0042be1cd83914d3362fafbd1b5d90c4a155a, repositionable streams were handled in such a way that the issue here observed went away.
4. With the introduction of specialized branches for repositionable streams in 025ec37733984b0240c319c15aaeeaacbe378be6, we now again see the issue that was present already in (2).

So, it seems that we now see a behaviour that was already observable at a previous point. Personally, I think it can be considered acceptable. If it is unclear what should happen here, maybe we can leave this issue open and somehow tag it as something where more input or consideration is needed, even if it is not entirely clear at the moment whether this is a mistake or anything should be done here.

[notoria]

This is related to how the stream works (peek_char/2). With this test:

test :-
    File = "file.out",
    Os0 = [type(text),reposition(false),eof_action(eof_code)],
    open(File, write, So, Os0), % stream-position `at`
    put_char(So, a), % stream-position `at`
    Os1 = [type(text),reposition(false),eof_action(eof_code)],
    open(File, read, Si, Os1), % stream-position `not`
    peek_char(Si, a), % No change to stream-position
    get_char(Si, a), % Update stream-position from `not` to `at`
    peek_char(Si, C0), % No change to stream-position
    C0 == end_of_file,
    put_char(So, b), % Comment this.
    peek_char(Si, C1),
    (   C0 == C1
    ->  true
    ;   portray_clause(user_output, unexpected-C1)
    ),
    close(Si),
    close(So).

One expects C0 to be identical to C1 but that's not the case.

The stream Si and So are supposed to be independent. If the stream is end-of-stream then peek_char/2 unifies with end_of_file.