`readChar` blocks after `peekChar` on stdin

[adrianwong]

Description

New to Nim, so apologies if I'm making an obvious mistake.

import std/streams

var s = stdin.newFileStream()
stdout.write "> "
echo "peek: ", s.peekChar()
echo "read: ", s.readChar()

Nim Version

Nim Compiler Version 2.0.0 [MacOSX: amd64] Compiled at 2023-09-18 Copyright (c) 2006-2023 by Andreas Rumpf

active boot switches: -d:release

Current Output

> 1
peek: 1
# Blocks on readChar()

Expected Output

> 1
peek: 1
read: 1
# Program terminates

Possible Solution

No response

Additional Information

No response

[litlighilit]

stdin is not peek-able (IIRC)
What's more, in my test on Linux, s.peekChar() will explicitly cause Error: unhandled exception: cannot retrieve file position [IOError], which differs from what you met on MacOSX: silently failing and blocking.
Maybe os-dependent behavior?
Later I'll test on Windows

[litlighilit]

Maybe os-dependent behavior? Later I'll test on Windows

On Windows, this behaves the same as MacOSX, i.e. blocking silently when peeking

[litlighilit]

But I find one interesting thing:
blocking as it's really, it's in fact waiting for your another input,
which means peekChar somewhat behaves the same as readChar
(p.s. tested on Windows)

[demotomohiro]

When you create FileStream, proc peekChar*(s: Stream): char indirectly calls proc fsPeekData(s: Stream, buffer: pointer, bufLen: int): int. https://github.com/nim-lang/Nim/blob/devel/lib/pure/streams.nim

fsPeekData indirectly calls setFilePos and getFilePos proc in syncio module. https://github.com/nim-lang/Nim/blob/devel/lib/std/syncio.nim

setFilePos calls fseek and getFilePos calls ftell in C stdlib.

So

import std/streams

var s = stdin.newFileStream()
stdout.write "> "
echo "peek: ", s.peekChar()
echo "read: ", s.readChar()

is almost same to following C code excepts following code doesn't call ftell:

#include <stdio.h>
#include <errno.h>
#include <string.h>

char readChar() {
  char c = '\0';
  size_t ret = fread(&c, sizeof(c), 1, stdin);
  printf("ret = %zu\n", ret);
  if(ret != 1) {
    if(feof(stdin) != 0) {
      printf("EOF\n");
    }
    if(ferror(stdin) != 0) {
      printf("Error\n");
    }

    return '\0';
  }

  return c;
}

int main() {

  char c = readChar();
  if(c == '\0') {
    return 1;
  }

  printf("peek: %c\n", c);

  if(fseek(stdin, 0, SEEK_SET) != 0) {
    int e = errno;
    printf("fseek failed because ... %s\n", strerror(e));
    return 1;
  }

  c = readChar();
  if(c == '\0') {
    return 1;
  }
  printf("read: %c\n", c);

  return 0;
}

I tested Nim code on my Linux.

$ nim c testpeek.nim 
$ ./testpeek 
testpeek.nim(5) testpeek
nim-2.1.1/lib/pure/streams.nim(484) peekChar
nim-2.1.1/lib/pure/streams.nim(322) peekData
nim-2.1.1/lib/pure/streams.nim(1341) fsPeekData
nim-2.1.1/lib/pure/streams.nim fsGetPosition
nim-2.1.1/lib/std/syncio.nim(779) getFilePos
nim-2.1.1/lib/std/syncio.nim(158) raiseEIO
Error: unhandled exception: cannot retrieve file position [IOError]
> $ echo 11 | ./testpeek 
testpeek.nim(5) testpeek
nim-2.1.1/lib/pure/streams.nim(484) peekChar
nim-2.1.1/lib/pure/streams.nim(322) peekData
nim-2.1.1/lib/pure/streams.nim(1341) fsPeekData
nim-2.1.1/lib/pure/streams.nim fsGetPosition
nim-2.1.1/lib/std/syncio.nim(779) getFilePos
nim-2.1.1/lib/std/syncio.nim(158) raiseEIO
Error: unhandled exception: cannot retrieve file position [IOError]
> $ echo 11 > test.txt
$ ./testpeek < test.txt 
> peek: 1
read: 1

getFilePos failed when stdin reads from keyboard or pipe. It works when stdin reads from a file

I saved above C code to teststdin.c. Then compiled and tested on my Linux:

$ gcc -o teststdin teststdin.c
$ ./teststdin 
1
ret = 1
peek: 1
fseek failed because ... Illegal seek
$ echo 11 | ./teststdin 
ret = 1
peek: 1
fseek failed because ... Illegal seek
$ echo 11 > test.txt
$ ./teststdin < test.txt 
ret = 1
peek: 1
ret = 1
read: 1

fseek failed when stdin reads from keyboard or pipe. It works when stdin reads from a file.

I don't have neither MacOS nor Windows. But if you don't get error, setFilePos/fseek or getFilePos/ftell works for stdin even if it reads from keyboard or pipe on these OS?

[litlighilit]

But if you don't get error, setFilePos/fseek or getFilePos/ftell works for stdin even if it reads from keyboard or pipe on these OS?

Not that,
I found that on Windows:

peekChar somewhat behaves the same as readChar

Also, I've tested teststdin.c on Windows [^1],
in which I found fseek(stdin, ...) will do nothing but return 0 fseek(stdin, 0, SEEK_SET) will set position of stdin to the begining of the current buffer (after cleaning buffer) here's my test:

3  <- first input
ret = 1
peek: 3
4  <- second input
ret = 1
read: 4

As mentioned above, if no second input, it'll block.

[^1]: using gcc in mingw-w64

[litlighilit]

fseek(stdin, 0, SEEK_SET) will set position of stdin to the begining of the current buffer (after cleaning buffer)

However, what's notable is that fseek for stdin is not portable, e.g. not work on Linux (as mentioned above)

Therefore, in turn, in Nim peekChar shall not used for a stdin based FileStream.

[litlighilit]

Solution:

While setFilePos is not for stdin,

peekDataImpl, called with length of 1 as its argument by peekChar, can call ungetc instead, which seems to run on stdin

I may try to implement later.

[litlighilit]

I may try to implement later.

See https://github.com/litlighilit/Nim/commit/3032c748ec3b67aa3f5669cffae94771959bd51f

But it looks ugly, as std/streams used to have no importc pragma.

So I myself even don't wanna PR it, considering it only solves peekChar of FileStream on stdin

[demotomohiro]

Pipes are also streams that you cannot peek or fseek. So streamwrapper was created for peeking a pipe: https://github.com/nim-lang/Nim/blob/devel/lib/pure/streamwrapper.nim

It seems newPipeOutStream proc also works for FileStream:

import std/[streams, streamwrapper]

var s = stdin.newFileStream().newPipeOutStream()
stdout.write "> "
echo "peek: ", s.peekChar()
echo "read: ", s.readChar()