habitat-lab
habitat-lab copied to clipboard
[hitl][hab-llm] New controller to work with habitat-llm Agents
Motivation and Context
This PR creates LLMController
based on SingleAgentBaselineController
which houses basic init
, reset
and act
code for calling habitat-llm agents.
~Confused: Do I need to update SingleAgentAccessMgr
class as well to be compatible with habitat-agents related to habitat-llm? The previously mentioned class just houses a lot of extra code which is useless in given setup. In habitat-llm setup the action
is directly sent from policies running wherever habitat-llm code is running, without needing any policy setup or inference from SingleAgentAccessMgr
class.~
SingleAgentAccessMgr
is also not needed, as inference comes from habitat-llm policies directly.
How Has This Been Tested
Types of changes
- [Docs change] Addition or changes to the documentation
- [Refactoring] Large changes to the code that improve its functionality or performance
- [Dependency Upgrade] Upgrades one or several dependencies in habitat
- [Bug Fix] (non-breaking change which fixes an issue)
- [Development] A pull request that add new features to the habitat-lab task and environment codebase. Development Pull Requests must be small (less that 500 lines of code change), have unit testing, very extensive documentation and examples. These are typically new tasks, environments, sensors, etc... The review process for these Pull Request is longer because these changes will be maintained by our core team of developers, so make sure your changes are easy to understand!
- [Experiment] A pull request that add new features to the habitat-baselines training codebase. Experiments Pull Requests can be any size, must have smoke/integration tests and be isolated from the rest of the code. Your code additions should not rely on other habitat-baselines code. This is to avoid dependencies between different parts of habitat-baselines. You must also include a README that will document how to use your new feature or trainer. You will be the maintainer of this code, if the code becomes stale or is not supported often enough, we will eventually remove it.
Checklist
- [ ] My code follows the code style of this project.
- [ ] I have updated the documentation if required.
- [ ] I have read the CONTRIBUTING document.
- [ ] I have completed my CLA (see CONTRIBUTING)
- [ ] I have added tests to cover my changes if required.
@zephirefaith You'll probably want to have this double-checked by someone who is familiar with the LLM configs so that we don't misstep here.