How to read response as a continuous stream?

[blaind]

I'm trying to make a GET request where the response is a (slow) continuous stream. However, the response reading results to a core dump.

async fn connect_to_server(&mut self) -> Result<()> {
    let mut reader = surf::get("http://...[server-address]").await?;

    println!("{:?}", reader.status());

    let mut buf = [0_u8; 1024];

    loop {
        let n = reader.read(&mut buf).await?;
        println!("Read data {:?} {}", &buf[..16], n);
    }

    Ok(())
}

stdout says:

200
Read data [13, 13, 23, ...] 12
Read data [15, 99, 121, ...] 887
Read data [15, 99, 121, ...] 0
thread 'async-task-driver' panicked at 'Receiver::next_message called after `None`', src/libcore/option.rs:1166:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
Aborted (core dumped)

Any ideas how to solve this? Here's the relevant part of the backtrace:

  11: core::option::Option<T>::expect
             at /rustc/4cf7673076e6975532213e494dd3f7f9d8c2328e/src/libcore/option.rs:345
  12: futures_channel::mpsc::Receiver<T>::next_message
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-channel-preview-0.3.0-alpha.17/src/mpsc/mod.rs:842
  13: <futures_channel::mpsc::Receiver<T> as futures_core::stream::Stream>::poll_next
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-channel-preview-0.3.0-alpha.17/src/mpsc/mod.rs:912
  14: <sluice::pipe::chunked::Reader as futures_io::if_std::AsyncRead>::poll_read
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/sluice-0.4.1/src/pipe/chunked.rs:83
  15: <sluice::pipe::PipeReader as futures_io::if_std::AsyncRead>::poll_read
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/sluice-0.4.1/src/pipe/mod.rs:43
  16: <chttp::body::Body as futures_io::if_std::AsyncRead>::poll_read
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/chttp-0.5.5/src/body.rs:157
  17: <alloc::boxed::Box<T> as futures_io::if_std::AsyncRead>::poll_read
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-io-preview-0.3.0-alpha.17/src/lib.rs:352
  18: <surf::http_client::Body as futures_io::if_std::AsyncRead>::poll_read
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/surf-1.0.1/src/http_client/mod.rs:83
  19: <&mut T as futures_io::if_std::AsyncRead>::poll_read
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-io-preview-0.3.0-alpha.17/src/lib.rs:352
  20: <surf::response::Response as futures_io::if_std::AsyncRead>::poll_read
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/surf-1.0.1/src/response.rs:246
  21: <&mut T as futures_io::if_std::AsyncRead>::poll_read
             at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-io-preview-0.3.0-alpha.17/src/lib.rs:352

[CryZe]

This is like a duplicate of #36

[blaind]

Now that #36 might be fixed, using commit https://github.com/rustasync/surf/commit/13485cb398799d5838bbd604463f18a4f4848fd4 the reading works for longer (a few seconds), but still crashes. Using reader.read_exact(&mut buf) is a bit better - crash after ~30 seconds.

I tried using isahc directly, and no problem with it (version 0.7.0).

async fn connect_to_server(&mut self) -> Result<()> {
        let mut response = Request::get("http://URL")
            .body(())?
            .send_async()
            .await?;

        println!("Status: {}", response.status());
        let body = response.body_mut();

        let mut buf = [0_u8; 1024];

        loop {
            body.read_exact(&mut buf).await?;
            println!("Read data {:?}", &buf[..16]);
        }
        Ok(())
}

[yoshuawuyts]

This seems to be a bug, and we should investigate further. Thanks heaps for reporting!