OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

Add integration test with dummy agent

Open rbren opened this issue 1 year ago • 5 comments

This will help with QA tasks--making sure the agent is able to read/write files, run commands, etc

rbren avatar Apr 23 '24 19:04 rbren

Two cents:

  1. Is it possible to move the DummyAgent to test package? If something is solely for testing purpose, it should ideally not reside in the main package. There might be a challenge to register the agent, though.
  2. I know why you think it's e2e testing but I feel like it's more like a special kind of unit testing.

li-boxuan avatar Apr 25 '24 07:04 li-boxuan

Yeah I was thinking e2e isn't quite right 😄 maybe integration?

I kind of want the DummyAgent to show up in the UI, at least for devs--it helps to quickly test all the UI features without having to set everything up, wait for the LLM, pay money, etc.

rbren avatar Apr 25 '24 15:04 rbren

Is it possible to add a new test to run this agent and put the test under test/integration/? That way, this agent doesn't need a dedicated GitHub Actions yaml file, and can be run with pytest.

li-boxuan avatar Apr 25 '24 17:04 li-boxuan

Hi @rbren , I'm happy to take a look at this but:

  1. It seems some tests are failing or at least stalled?
  2. I also added a dummy agent in this PR: https://github.com/OpenDevin/OpenDevin/blob/46bd83678a09c9f20ab53b30b7f2278932ace11b/agenthub/dummy_agent/agent.py#L11

Feel free to get rid of my dummy agent and replace it with yours, I don't think it should break any of the unit tests I created.

neubig avatar Apr 29 '24 17:04 neubig

@neubig I think we're good to go!

rbren avatar Apr 30 '24 00:04 rbren