Implementation of Reflexion in Langchain

[viraniaman94]

Is there a plan to implement Reflexion in Langchain as a separate agent (or maybe an add-on to existing agents)?

https://arxiv.org/abs/2303.11366

Sample implementation: https://github.com/GammaTauAI/reflexion-human-eval/blob/main/reflexion.py

[slavakurilyak]

It's time for a reflexion, pun intended!

The authors (@noahshinn024 et al.) "hypothesize that LLMs possess an emergent property of self-reflection."

GPT-4 can significantly improve its performance by designing and executing tests to critique its own performance, as shown here with its results on the AlfWorld test.

On the HumanEval coding test, GPT-4 went from 67% to 88% accuracy, an impressive leap, using self-reflective loops. In other words, GPT-4 becomes 30% more accurate when asked to critique itself.

Here's how it works:

Here's how to combine it with ReAct:

Here are five ways to get started with Reflexion:

1. Improved code generation: Integrate Reflexion and LangChain to develop an AI-powered code generator that learns from its own generated code, continuously refining its output for increased accuracy and efficiency.
2. Improved chatbot performance - Use Reflexion for enhancing LangChain chatbots by learning from errors and optimizing future choices.
3. Improved conversational agents - Apply Reflexion to LangChain conversational agents for learning from past interactions and refining future replies.
4. Efficient recommendation systems - Use self-reflection data storage in LangChain's memory modules for improved recommendation system development.
5. Accurate text classification - Integrate Reflexion and LangChain's text classification for an evolving, precise tool.

We are shifting the “accuracy bottleneck” from correct syntactic and semantic code generation to correct syntactic and semantic test generation. @noahshinn024

Resources

• Official implementation of reflexion by @noahshinn024
• Spin-off project: Mastering HumanEval with Reflexion by @GammaTauAI
• Great explainer: GPT 4 Can Improve Itself by AI Explained

Hack

Here's a quick way to implement reflection, inspired by Language Models can Solve Computer Tasks:

Reflection Prompt:

"Review your previous answer and find problems with your answer"

The idea behind this prompt is to encourage the LLM to critically evaluate its own output and identify any problems with it. By doing so, the LLM can improve its output by generating an updated version that addresses the identified problems.

Post-Reflection Prompt:

"Based on the problems you found, improve your answer."

The idea behind this prompt is to encourage the LLM to use the identified problems as feedback to generate an updated version of its output that addresses those problems. By doing so, the LLM can improve its output quality and accuracy.

[jwm4]

I agree with everything @slavakurilyak wrote, and I think something along these lines would be very useful for langchain. With that said, I'd prefer to see the RCI technique described in Language Models can Solve Computer Tasks be split off into a separate "Implementation of RCI in Langchain" issue rather than framing it as a "hack" solution to "Implementation of Reflexion in Langchain" -- which is related, but distinct in my opinion. Implementing RCI in Langchain sounds like it would be a fairly quick, easy win.

[slavakurilyak]

In the words of @noahshinn024, "Reflexion is not a replacement for ReAct or any other decision-guiding approach! It is a simple retry technique that can be used (to) enhance other approaches."

[klein-t]

So what is the state of the integration? Is somebody already working on it?

[noahshinn]

So what is the state of the integration? Is somebody already working on it?

Currently busy with other work but would be open to call with someone if they are interested in implementing a version of Reflexion. In the meantime, I upvote the comment about using RCI for quick implementation.

[sorendunn]

@azamiftikhar1000 Any progress on implementing Reflexion in Langchain? What are the main bottlenecks? I am no expert but would be more than happy to help where I can (I am currently building on the original Reflexion implementation for another project I am working on).

[RobRoyce]

Also interested in helping to implement!

[UmerHA]

@sorendunn @RobRoyce I've also started implementing it. Think I'll be done in 2-3 days.

I'll send you the code so you can comment! If you're interested in helping feel free to message me on Discord (RicChilligerDude#7589) and we can collaborate

Edit: Here's the current code: https://github.com/UmerHA/langchain/tree/2316-reflexion/langchain/agents/

[UmerHA]

Hey @vowelparrot, Reflexion works now. 🎉

Because Reflexion introduces the concept of trials, which Agent/AgentExecutor currently don't handle, I had to duplicate a lot of code. Can we have a call to speak about your vision for the langchain agent architecture?

Integrating Reflexion, SmartGPT, Plan & Execute, ... means changing some of stuff. Would love to talk about that with you first, before diving in. :)

Edit 15-05-23: Imo Relfexion only makes sense for Action Agents, not Plan & Execute Agents. I have therefore extended AgentExecutor (of Action Agents) to handle Reflexion. See https://github.com/hwchase17/langchain/pull/4737.

[dosubot[bot]]

Hi, @viraniaman94! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is about implementing Reflexion in LangChain as a separate agent or add-on. There have been some interesting discussions in the comments, including a suggestion by @slavakurilyak that GPT-4 could significantly improve its performance. @jwm4 also suggested splitting the implementation of RCI into a separate issue. It seems like there is progress towards resolving the issue, with interest expressed by @UmerHA and others in helping to implement Reflexion in LangChain.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contributions to the LangChain project!

[UmerHA]

Hi, @viraniaman94! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is about implementing Reflexion in LangChain as a separate agent or add-on. There have been some interesting discussions in the comments, including a suggestion by @slavakurilyak that GPT-4 could significantly improve its performance. @jwm4 also suggested splitting the implementation of RCI into a separate issue. It seems like there is progress towards resolving the issue, with interest expressed by @UmerHA and others in helping to implement Reflexion in LangChain.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contributions to the LangChain project!

Imo this issue is still open. The relevant PR (https://github.com/langchain-ai/langchain/pull/4737) is waiting for input from the LC team

[dosubot[bot]]

@baskaryan Could you please help @viraniaman94 with this issue? They have indicated that it is still relevant and there is a relevant PR waiting for input from the LangChain team. Thank you!

[dosubot[bot]]

Hi, @viraniaman94

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue "Implementation of Reflexion in Langchain" was opened to inquire about implementing Reflexion in Langchain as a separate agent or add-on to existing agents, with discussions on potential benefits, suggestions to split the implementation of RCI into a separate issue, and expressions of interest from contributors like UmerHA in helping with the implementation. UmerHA indicated that it is still open and relevant, with a relevant PR waiting for input from the LangChain team.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

[robtaylor]

For other folk looking, seems the last status was that #4737 needs reimplementing using Runnables (LCEL) in a template. https://github.com/langchain-ai/langchain/blob/master/templates/docs/CONTRIBUTING.md