camel-ai
[Feature Request] Implement Learn-by-interact
Required prerequisites
- [x] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
- [ ] Consider asking first in a Discussion.
Motivation
Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments
Autonomous agents powered by large language models (LLMs) have the potential to enhance human capabilities, assisting with digital tasks from sending emails to performing data analysis. The abilities of existing LLMs at such tasks are often hindered by the lack of high-quality agent data from the corresponding environments they interact with. We propose Learn-by-interact, a data-centric framework to adapt LLM agents to any given environments without human annotations. Learn-by-interact synthesizes trajectories of agent-environment interactions based on documentations, and constructs instructions by summarizing or abstracting the interaction histories, a process called backward construction. We assess the quality of our synthetic data by using them in both training-based scenarios and training-free in-context learning (ICL), where we craft innovative retrieval approaches optimized for agents. Extensive experiments on SWE-bench, WebArena, OSWorld and Spider2-V spanning across realistic coding, web, and desktop environments show the effectiveness of Learn-by-interact in various downstream agentic tasks -- baseline results are improved by up to 12.2% for ICL with Claude-3.5 and 19.5% for training with Codestral-22B. We further demonstrate the critical role of backward construction, which provides up to 14.0% improvement for training. Our ablation studies demonstrate the efficiency provided by our synthesized data in ICL and the superiority of our retrieval pipeline over alternative approaches like conventional retrieval-augmented generation (RAG). We expect that Learn-by-interact will serve as a foundation for agent data synthesis as LLMs are increasingly deployed at real-world environments.
https://arxiv.org/abs/2501.10893
Solution
No response
Alternatives
No response
Additional context
No response
- for Task formulation, we can make it based on https://github.com/camel-ai/camel/pull/1515;
- for Agentic data synthesis, we have self-instruct method integrated here: https://github.com/camel-ai/camel/blob/master/examples/datagen/self_instruct/self_instruct.py;
- what we need to add is the backward construction method mentioned in the paper;
- for Filtering, we have some filters here: https://github.com/camel-ai/camel/tree/master/camel/datagen/self_instruct/filter, but we still can expand this part with more filter methods;
- we can use the synthesized data as demonstration examples in ICL to showcase the benefit and add the example under
examples/datagen
overall, what we can implement is
- add backward construction method
- add more filters if necessary
- create example based on ReAct agent, self-instruct pipeline, backward construction method, filter to generate data, and use the synthesized data as demonstration example
Let me know if you have any thought!
