Error when using multi-turn: Expected the last role to be User or Tool (or Assistant with prefix True).

[sopaco]

use crate::tools::{outlet::TextSaver, perfetto::SqlExecutor};
use anyhow::Result;
use rig::agent::Agent;
use rig::completion::{Completion, Prompt};
use rig::providers::mistral::CompletionModel;
use rig::{
    client::{CompletionClient, ProviderClient},
    providers,
};

...
        let sql_executor = SqlExecutor::default();
        let text_saver = TextSaver::default();

        let llm_client = providers::mistral::Client::from_env();

        // Create agent with tools
        let agent = llm_client
            .agent("devstral-small-latest")
            .preamble(format!(include_str!("prompts/analysis_trace_sys.tpl"), platform).as_str())
            .max_tokens(100000)
            .temperature(0.1)
            .tool(sql_executor)
            .tool(text_saver)
            .build();

        // Agent is now ready to use with the configured tools
        let kb_all_about_perfetto_sql = include_str!("./KB/all-about-perfetto-sql.md");

        // Create the initial prompt
        let initial_prompt = format!(
            include_str!("prompts/analysis_trace.tpl"),
            trace_path, out_path, kb_all_about_perfetto_sql
        );

        // Start the agentic loop
        // self.agentic_loop(&agent, initial_prompt, 0).await?;
        let response = agent.prompt(&initial_prompt).multi_turn(10).await?;

My code is as above, and it reports an error when running:

Error: CompletionError: ProviderError: {"object":"error","message":"Expected last role User or Tool (or Assistant with prefix True) for serving but got assistant","type":"invalid_request_message_order","param":null,"code":"3230"}

Caused by:
    ProviderError: {"object":"error","message":"Expected last role User or Tool (or Assistant with prefix True) for serving but got assistant","type":"invalid_request_message_order","param":null,"code":"3230"}

rig-core: 0.17.1

[linear[bot]]

RIG-887 Error when using multi-turn: Expected the last role to be User or Tool (or Assistant with prefix True).

[sopaco]

It does indeed appear that the most recent entry in chat_history does not contain a tool message. However, based on the execution flow in prompt_request, it seems that the tool message was actually retrieved.

[0xMochan]

You likely don't want to use .multi_turn(1). By default, if a tool is called, it'll complete the round-trip (returning the Tool message back to the model to get rendered). Only if the agent wants to call multiple tools would you result in a crash.

I'm still sorta surprised this is crashing here, maybe this is a mistral specific behavior that's different here or there's something i'm missing,

[sopaco]

You likely don't want to use .multi_turn(1). By default, if a tool is called, it'll complete the round-trip (returning the Tool message back to the model to get rendered). Only if the agent wants to call multiple tools would you result in a crash.

I'm still sorta surprised this is crashing here, maybe this is a mistral specific behavior that's different here or there's something i'm missing,

I set up multiple tools by calling agent.tool. Is there an issue with the framework's adaptation to Mistral?

[joshua-mo-143]

It looks like this is apparently supposed to be True under some circumstances (specifically when conditioning a model to use a prefixed response), given the picture below. However I have no idea what to do about this without a message dump.

[sopaco]

It looks like this is apparently supposed to be True under some circumstances (specifically when conditioning a model to use a prefixed response), given the picture below. However I have no idea what to do about this without a message dump.

I have captured the following HTTP communication data:

rig_multi-turn_mistral.log.txt

[aleksmeshr]

The issue could be fixed by providing a message with the expected Tool role instead of User for a tool result: https://github.com/0xPlaygrounds/rig/pull/1130

[aleksmeshr]

The fix #1130 was just merged. @sopaco, could you please test whether the issue in your use case has been resolved?