Making principal agent stick to a personality (while also robustly selecting tools)

[nemoooooooooo]

Issue you'd like to raise.

BACKGROUND: We want to assign a personality to the agent. We simultaneously want this agent to be capable of running/selecting/using multiple tools. We tried passing the personality in the "PREFIX". We tried using prompt template.

It didn't really work - it wouldn't stick to the personality for long.

However, when we passed the personality string to the "system" role (through OpenAI's API call), it stuck to the personality much longer.

To pass the personality to the system role, we override your class called OpenAIChat(), like so:

class MyOverriddenOpenAIChat(OpenAIChat):
    system_message: str = ""

    def _default_params(self) -> Dict[str, Any]:
        """Get the default parameters for calling OpenAI API."""
        return self.model_kwargs

    def _get_chat_params(
        self, prompts: List[str], stop: Optional[List[str]] = None
    ) -> Tuple:
        if len(prompts) > 1:
            raise ValueError(
                f"MyOverriddenOpenAIChat currently only supports single prompt, got {prompts}"
            )

        # Use the system message here.
        messages = [{"role": "system", "content": self.system_message}] + \
                   self.prefix_messages + [{"role": "user", "content": prompts[0]}]

        params: Dict[str, Any] = {**{"model": self.model_name}, **self._default_params()}  # Invoking the method here
        if stop is not None:
            if "stop" in params:
                raise ValueError("`stop` found in both the input and default params.")
            params["stop"] = stop
        if params.get("max_tokens") == -1:
            # for ChatGPT api, omitting max_tokens is equivalent to having no limit
            del params["max_tokens"]
        return messages, params

PROBLEM: Since you're not supporting the class OpenAIChat() in later versions (and we do want to upgrade to your latest version), how do we achieve passing a personality to the system role via your langchain latest version(s)?

If there is no easy way to pass the personality string to the system role via your latest langchain version - what is an alternative route to make a principal agent stick to a personality?

Please advise.

Suggestion:

No response

[OlajideOgun]

Have you tried using ChatOpenAI and also using SystemMessage as the message?

[SDcodehub]

yes for model like gpt-4 system messages are respected. you will be able to achieve the task with curated system message.

for gpt3.5, compared to 4 pays less attention to system message, so you might need to add some argument related to profile in human message.

[nemoooooooooo]

Have you tried using ChatOpenAI and also using SystemMessage as the message?

i have tried that and it's clearly not working. I am not a 100% sure about it but looking at the output I belive system message is not passed as content of system role to API.

from langchain.prompts import (
    ChatPromptTemplate, 
    MessagesPlaceholder, 
    SystemMessagePromptTemplate, 
    HumanMessagePromptTemplate
)
from langchain.chains import ConversationChain
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.llms import OpenAI
import os
#setting envirmental varaibles
os.environ["OPENAI_API_KEY"] = "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"
os.environ["LANGCHAIN_HANDLER"] = "langchain"
os.environ["ZAPIER_NLA_API_KEY"] = "sk-ak-J09wzy0oNUalQllUKxHk0lgSqv"




prompt = ChatPromptTemplate.from_messages([
    SystemMessagePromptTemplate.from_template("You are a French speaking bot"),
    MessagesPlaceholder(variable_name="history"),
    HumanMessagePromptTemplate.from_template("{input}")
])


memory = ConversationBufferMemory(return_messages=True)


chat = ChatOpenAI(temperature=0)
llm = OpenAI(temperature=0)
tools = load_tools(["llm-math"], llm=llm)

agent = initialize_agent(tools, chat, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True, memory=memory)


# Now let's test it out!
agent.run("what language do you speak?")

Here is the output: Entering new chain... Question: What language do you speak? Thought: This question is not related to any tool, I will just answer it directly. Action:

{
  "action": "None",
  "action_input": ""
}

Observation: None is not a valid tool, try another one. Thought:This question is not related to any tool, I will just answer it directly. Action:

{
  "action": "None",
  "action_input": ""
}

Observation: None is not a valid tool, try another one. Thought:This question does not require any tool, I will just answer it directly. Final Answer: As an AI language model, I do not speak any language in the traditional sense. However, I can understand and generate text in multiple languages.

Finished chain.

[OlajideOgun]

This is an issue of promoting. Try being more detailed in your system prompt and it will respond correctly

On Jun 18, 2023, at 11:03 AM, nemoooooooooo @.***> wrote:

 Have you tried using ChatOpenAI and also using SystemMessage as the message?

i have tried that and it's clearly not working. I am not a 100% sure about it but looking at the output I belive system message is not passed as content of system role to API.

from langchain.prompts import ( ChatPromptTemplate, MessagesPlaceholder, SystemMessagePromptTemplate, HumanMessagePromptTemplate ) from langchain.chains import ConversationChain from langchain.chat_models import ChatOpenAI from langchain.memory import ConversationBufferMemory from langchain.agents import load_tools from langchain.agents import initialize_agent from langchain.agents import AgentType from langchain.llms import OpenAI import os #setting envirmental varaibles os.environ["OPENAI_API_KEY"] = "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" os.environ["LANGCHAIN_HANDLER"] = "langchain" os.environ["ZAPIER_NLA_API_KEY"] = "sk-ak-J09wzy0oNUalQllUKxHk0lgSqv"

prompt = ChatPromptTemplate.from_messages([ SystemMessagePromptTemplate.from_template("You are a French speaking bot"), MessagesPlaceholder(variable_name="history"), HumanMessagePromptTemplate.from_template("{input}") ])

memory = ConversationBufferMemory(return_messages=True)

chat = ChatOpenAI(temperature=0) llm = OpenAI(temperature=0) tools = load_tools(["llm-math"], llm=llm)

agent = initialize_agent(tools, chat, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True, memory=memory)

Now let's test it out!

agent.run("what language do you speak?") Here is the output: Entering new chain... Question: What language do you speak? Thought: This question is not related to any tool, I will just answer it directly. Action:

{ "action": "None", "action_input": "" } Observation: None is not a valid tool, try another one. Thought:This question is not related to any tool, I will just answer it directly. Action:

{ "action": "None", "action_input": "" } Observation: None is not a valid tool, try another one. Thought:This question does not require any tool, I will just answer it directly. Final Answer: As an AI language model, I do not speak any language in the traditional sense. However, I can understand and generate text in multiple languages.

Finished chain.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[harisrab]

Have you tried using ChatOpenAI and also using SystemMessage as the message?

i have tried that and it's clearly not working. I am not a 100% sure about it but looking at the output I belive system message is not passed as content of system role to API.

from langchain.prompts import (
    ChatPromptTemplate, 
    MessagesPlaceholder, 
    SystemMessagePromptTemplate, 
    HumanMessagePromptTemplate
)
from langchain.chains import ConversationChain
from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.llms import OpenAI
import os
#setting envirmental varaibles
os.environ["OPENAI_API_KEY"] = "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"
os.environ["LANGCHAIN_HANDLER"] = "langchain"
os.environ["ZAPIER_NLA_API_KEY"] = "sk-ak-J09wzy0oNUalQllUKxHk0lgSqv"




prompt = ChatPromptTemplate.from_messages([
    SystemMessagePromptTemplate.from_template("You are a French speaking bot"),
    MessagesPlaceholder(variable_name="history"),
    HumanMessagePromptTemplate.from_template("{input}")
])


memory = ConversationBufferMemory(return_messages=True)


chat = ChatOpenAI(temperature=0)
llm = OpenAI(temperature=0)
tools = load_tools(["llm-math"], llm=llm)

agent = initialize_agent(tools, chat, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True, memory=memory)


# Now let's test it out!
agent.run("what language do you speak?")

Here is the output: Entering new chain... Question: What language do you speak? Thought: This question is not related to any tool, I will just answer it directly. Action:

{
  "action": "None",
  "action_input": ""
}

Observation: None is not a valid tool, try another one. Thought:This question is not related to any tool, I will just answer it directly. Action:

{
  "action": "None",
  "action_input": ""
}

Observation: None is not a valid tool, try another one. Thought:This question does not require any tool, I will just answer it directly. Final Answer: As an AI language model, I do not speak any language in the traditional sense. However, I can understand and generate text in multiple languages.

Finished chain.

Am I correct that you're not even passing in the prompt?

[dosubot[bot]]

Hi, @nemoooooooooo! I'm Dosu, and I'm here to help the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is about assigning a personality to an agent while allowing it to use multiple tools. You mentioned that you found a workaround by passing the personality string to the "system" role, but you are unsure how to achieve this in the latest version of LangChain. In the comments, there were suggestions to use ChatOpenAI and SystemMessage as the message, but you mentioned that it is not working. Another user suggested being more detailed in the system prompt.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of LangChain. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain repository!