Filesystem I/O

[ralt]

How should one do filesystem I/O using cl-async?

I'm thinking about something like this:

(as:open "foo"
  :if-exists :overwrite
  :open-cb (lambda (file-stream)))

But it obviously doesn't exist in cl-async, so I'm wondering what I'm missing.

Should I use the streamish class? As in:

(with-open-file (file-stream "foo" :direction :input :if-exists :overwrite)
  (let ((streamish (make-instance 'as:async-input-stream :streamish file-stream)))
    (as:stream-read-sequence streamish buf)))

From what I understand in the code, that's not how it should work though, since my example is not event-loop aware (and I/O would fail pretty hard on it, e.g. EAGAIN is not handled).

If I understand the code correctly, a new class must be added that implements the defmethod for an async-input-stream. Does it already exist and I missed it? Am I reading the code wrong?

[ralt]

It looks like there's no binding for libuv's uv_fs_open, so I guess filesystem I/O isn't supported by cl-async?

[orthecreedence]

Sorry for the lag, I'm not sure if we ever wrapped libuv's async file i/o (it has been years since I've worked on cl-async materially, so my memory is a bit foggy). There are a few things you do...fork/implement/issue a PR or you can use something like lparallel to achieve the same thing libuv does...start a thread pool with N many workers, and feed file i/o jobs to the thread pool, essentially sidestepping the blocking file i/o. Lastly, you can go the nginx route and just do blocking file i/o, reading the file in acceptably-sized chunks so as not to block the event loop for too long,

Hope this helps.

[Plisp]

libuv has a uv_fs_t type that provides support for what you describe. It employs libuv's own thread pool (which you could emulate on the lisp side) as os multiplexing facilities cannot handle filesystem i/o (descriptor is always readable). I'll be working on this soon.