ChatGPT Plugins vs. LangChain agent

[marialovesbeans]

I'm wondering how does ChatGPT plugin work under-the-hood? Are they using the same techique as Langchain agent? Basically sending prompt and manually parse out the result -> and feed result back to another plugin in subsequential steps if necessary.

Or is it using a superior technique than LangChain agent? I can't find a mention of LangChain in their doc.

Any thoughts are welcome!

[Ynng]

In comparing ChatGPT plugins to Langchain, it appears that ChatGPT's support for third-party plugins is relatively limited.

Developers using ChatGPT are restricted to defining specific actions or HTTP endpoints for the language model to call. This makes it challenging, if not impossible, for third-party developers to create tools that needs a LLM, such as Palmath. (Although ChatGPT does offer an official equivalent called "Code Interpreter")

[diemus]

It seems that Langchain has a competitor now.

[zachschillaci27]

It looks similar at least in how the plugins (i.e. Langchain tools) are described to the model, which allows ChatGPT to choose which tool to use for a particular problem. I'm not sure if there's a possibility of chaining together multiple tools in a single agent call, but I assume it would have this capability

[zachschillaci27]

Also they directly link to Langchain in the introduction section (follow the 'ideas' reference)

[kennethleungty]

From the above video on their blog post (https://openai.com/blog/chatgpt-plugins), it seems like the multiple tools were called upon based on a single agent call. This is a demo, so I am sure they are displaying their smoothest/most accurate example. Based on that demo alone, it does seem very impressive compared to Langchain agent.

Anyone with insights on what their agent does differently compared to Langchain's agents?

[zachschillaci27]

From the above video on their blog post (https://openai.com/blog/chatgpt-plugins), it seems like the multiple tools were called upon based on a single agent call. This is a demo, so I am sure they are displaying their smoothest/most accurate example. Based on that demo alone, it does seem very impressive compared to Langchain agent.

Anyone with insights on what their agent does differently compared to Langchain's agents?

As of now, perhaps they're doing some zero-shot type decision capability using a ReACT/MRKL framework. But given they also cite the toolformer paper, I wouldn't be surprised if they fine-tune later versions of GPT-4 to better use some of the more tightly integrated tools (e.g. Wolfram Alpha)

[tonyabracadabra]

It is likely they used some kind of zero shot toolformer like what https://github.com/minosvasilias/toolformer-zero does

[ericwangqing]

Actually, IMO, the most interesting things here are 

1. how ChatGPT figure out the steps and the order of execution these plugins, 
2. and how ChatGPT pass the result of a plugin to the next , is there anything like the output parser of LangChain implicitly working behind the scene?

[tonyabracadabra]

Actually, IMO, the most interesting things here are 

1. how ChatGPT figure out the steps and the order of execution these plugins,
2. and how ChatGPT pass the result of a plugin to the next , is there anything like the output parser of LangChain implicitly working behind the scene?

1. The order is determined in the fly just like langchain agent does
2. They don’t have to parse the result as the result is simply returned from the tool they use in raw format; The reason why OutputParser exists in langchain is because in many cases, by chaining LLMs we need to parse the result generated by LLM, but this won’t apply to the result returned from a tool.

[ericwangqing]

The order is determined in the fly just like langchain agent does

@tonyabracadabra does that mean for users enable some plugins, before really answer a message, OpenAI inserts a planning conversation to LLM?

[tonyabracadabra]

The order is determined in the fly just like langchain agent does

@tonyabracadabra does that mean for users enable some plugins, before really answer a message, OpenAI inserts a planning conversation to LLM?

That’s one possible way (unarguably the simplest way) but I do believe OpenAI may come up with something beyond that to achieve better performance and lower cost, an example for the alternatives might be https://arxiv.org/abs/2212.04089 or its variants that utilizes the compositional nature of the latent space, so whenever certain plugins are selected you are basically using a model with a few thin plugins weight layers that are pre-trained for performance optimization.

[ericwangqing]

That’s one possible way (unarguably the simplest way) but I do believe OpenAI may come up with something beyond that to achieve better performance and lower cost.

according to the official doc of OpenAI, I guess currently they are doing the simplest way, and may collecting training data for the next stage to cut down cost.

[biedanian]

From my understanding, langchain allows LLM to use tools through pre-set prompt templates. I don’t know if the chatgpt plugin is also done in the same way. Hypothetically, what is the difference between the plugin model and langchain? Has openai done training for the plugin to make its own method work better?

[ericwangqing]

Technically speaking, they are the same. However, commercially speaking, LangChain allows us to choose LLM and master business logic while OpenAI controls everything without choice.

[dosatos]

How does it differ from token usage point of view? Do I understand right that the OpenAI Plugins provide the context related token usage for free?

[yeagob]

But there is a way to use gpt plugins from an api call?

[dosubot[bot]]

Hi, @marialovesbeans! I'm Dosu, and I'm here to help the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, you were asking for clarification on how the ChatGPT plugin works compared to the LangChain agent. In the comments, there were discussions about the limitations of ChatGPT's support for third-party plugins and how it compares to LangChain. Some users also speculated on the order of execution of plugins and the possibility of chaining multiple tools in a single agent call. There were mentions of zero-shot toolformers and potential future improvements in ChatGPT's capabilities.

Now, we would like to know if this issue is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding, and please don't hesitate to reach out if you have any further questions or concerns!