bug: Azure or OpenAi Stream api not working.

[mztlive]

I don't know what happened. I will get this error if I use azure or openai's stream agent

model is gpt-4o

thread 'tokio-runtime-worker' panicked at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/rig-core-0.12.0/src/providers/openai/streaming.rs:132:51:
Should have at least one choice

[linear[bot]]

RIG-747 bug: Azure or OpenAi Stream api not working.

[joshua-mo-143]

Hey - just had a look at the codebase for the v0.12 release, at the time when you get the panic how far along in streaming does it typically get before it panics?

[mztlive]

Hey - just had a look at the codebase for the v0.12 release, at the time when you get the panic how far along in streaming does it typically get before it panics?

This is my test code

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    // Create streaming agent with a single context prompt
    let agent = openai::Client::from_url(
        "xxxxxxx",
        "xxxxxx",
    )
    .agent(openai::GPT_4O)
    .preamble("Be precise and concise.")
    .temperature(0.5)
    .build();

    // Stream the response and print chunks as they arrive
    let mut stream = agent
        .stream_prompt("When and where and what type is the next solar eclipse?")
        .await?;

    stream_to_stdout(agent, &mut stream).await?;

    Ok(())
}


// I added logs to this method to check send_compatible_streaming_request
// Near Line 100
  for line in text.lines() {
                let mut line = line.to_string();

                println!("line: {}", line);

                // If there was a remaining part, concat with current line
                if partial_data.is_some() {
                    line = format!("{}{}", partial_data.unwrap(), line);
                    partial_data = None;
                }


// print result is 
// Response: line: data: {"choices":[],"created":0,"id":"","model":"","object":""}

// stacktrace is:

thread 'main' panicked at /Users/huangjiajiang/Development/rig/rig-core/src/providers/openai/streaming.rs:130:51:
Should have at least one choice
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/panicking.rs:75:14
   2: core::panicking::panic_display
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/panicking.rs:261:5
   3: core::option::expect_failed
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/option.rs:2024:5
   4: core::option::Option<T>::expect
             at /Users/huangjiajiang/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/option.rs:933:21
   5: rig::providers::openai::streaming::send_compatible_streaming_request::{{closure}}::{{closure}}
             at ./rig-core/src/providers/openai/streaming.rs:130:30
   6: <async_stream::async_stream::AsyncStream<T,U> as futures_core::stream::Stream>::poll_next
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/async-stream-0.3.6/src/async_stream.rs:56:13
   7: <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/futures-core-0.3.31/src/stream.rs:130:9
   8: futures_util::stream::stream::StreamExt::poll_next_unpin
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/futures-util-0.3.31/src/stream/stream/mod.rs:1638:9
   9: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/futures-util-0.3.31/src/stream/stream/next.rs:32:9
  10: rig::streaming::stream_to_stdout::{{closure}}
             at ./rig-core/src/streaming.rs:94:43
  11: openai_streaming::main::{{closure}}
             at ./rig-core/examples/openai_streaming.rs:21:42
  12: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /Users/huangjiajiang/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9
  13: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/runtime/park.rs:284:60
  14: tokio::task::coop::with_budget
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/task/coop/mod.rs:167:5
  15: tokio::task::coop::budget
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/task/coop/mod.rs:133:5
  16: tokio::runtime::park::CachedParkThread::block_on
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/runtime/park.rs:284:31
  17: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/runtime/context/blocking.rs:66:9
  18: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  19: tokio::runtime::context::runtime::enter_runtime
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/runtime/context/runtime.rs:65:16
  20: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  21: tokio::runtime::runtime::Runtime::block_on_inner
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/runtime/runtime.rs:370:45
  22: tokio::runtime::runtime::Runtime::block_on
             at /Users/huangjiajiang/.cargo/registry/src/rsproxy.cn-e3de039b2554c837/tokio-1.44.1/src/runtime/runtime.rs:340:13
  23: openai_streaming::main
             at ./rig-core/examples/openai_streaming.rs:23:5
  24: core::ops::function::FnOnce::call_once
             at /Users/huangjiajiang/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[0xMochan]

Seems like an assumption was broken by the provider here. Our code shouldn't panic, but should be easy to resolve.

Thanks for the report!

[yavens]

This was fixed by #388, albeit unknowingly

[joshua-mo-143]

Ok looks like we need to do a release next week then if there is availability to do so.

In the meantime @mztlive you can remedy the issue by using the main Rig branch from Github. I'll keep this issue open until we release a new version as it is still an ongoing problem until then.

[aurexav]

Ok looks like we need to do a release next week then if there is availability to do so.

Please!

[joshua-mo-143]

The relevant release has been made for now, so I'll close this issue. For anyone else encountering this issue, you need Rig ^=0.13 (0.13 or above) to fix this issue.

If there's anything else, please open a new GitHub issue. Thank you!