`isatty` is not required to set errno

[jgallagher]

trafficstars

Hi! On illumos, nix's isatty() function can return Err(UnknownErrno) for non-TTY file descriptors:

$ cat src/main.rs
use std::io;
use std::os::unix::io::AsRawFd;

fn main() {
    dbg!(nix::unistd::isatty(io::stdout().as_raw_fd()));
}
$ cargo run | cat
[src/main.rs:7] nix::unistd::isatty(io::stdout().as_raw_fd()) = Err(
    UnknownErrno,
)

Digging in a bit, we found that the issue is that illumos currently chooses not to set errno in isatty(), but nix expects errno to be set to ENOTTY if isatty() returns 0, and returns the error we're seeing otherwise. I think illumos's behavior here is not ideal (and we're also filing a bug there), but is technically allowed by the wording on the POSIX manpage (emphasis mine):

The isatty() function shall return 1 if fildes is associated with a terminal; otherwise, it shall return 0 and may set errno to indicate the error.

Would you be opposed to changing the implementation of isatty() to only look at the return value and not errno?