agency-swarm icon indicating copy to clipboard operation
agency-swarm copied to clipboard

Agency creation failing with hallucination

Open OscarsOrbit opened this issue 1 year ago • 1 comments

Hello

I have ran several tests for this and I am unable to get a simple agency created. Not sure if it's due to the folder structure not being picked up properly, but today as I tried again, I noticed something:

When I looked inside my OpenAI assistants page, I saw the created assistants don't have proper instructions sent. I am not sure if it's supposed to look like this? I'm guessing this should have the actual instructions and not just a reference to the file path:

image

This is after running a clean install of agency swarm.

I asked for a relatively small task just to test it, like so:

image

I'm not sure this is the correct flow, because only the "GenesisCEO" agent is acting here. Is this the intended behavior? It looked different in your video demo @VRSEN

Also, check out where the agency has been added:

image

It's been added in the path /agency-swarm/notebooks/(agencyname)/(agencyname)/

Is this the intended behavior? - my guess would have been that the agency would be added to the /agency-swarm/agency/(agencyname) path and the agents (which, none were created) would be added to the /agency-swarm/agency/(agentname) path.

Am I missing something here?

The CEO agent seems to think the agency was successfully created, but it doesn't give me instructions to run the agency.py file even like it's supposed to at the end when the task has been completed.

Any help to get this working correctly is highly appreciated.

OscarsOrbit avatar Jan 18 '24 17:01 OscarsOrbit

I encountered the same issue with the genesis agency (hallucination, adding nested agents to notebooks path) and think I'm going to end up just custom creating the agency I need. Definitely following here to see if this is intended or a known issue

TylerAlmeida avatar Jan 19 '24 02:01 TylerAlmeida

Seems like this issue is only happening on windows @OscarsOrbit. Can you please try to modify this method in agent.py https://github.com/VRSEN/agency-swarm/blob/6bbd3c75bcf2dc932b41763cc0b18bbfee3bd33d/agency_swarm/agents/agent.py#L461 and let me know if you manage to fix this? The path is incorrect for some reason

VRSEN avatar Jan 29 '24 02:01 VRSEN

@VRSEN, just to let you know I've pulled the latest code on my WSL machine, and I'm getting the same error.

Let me know if there's a particular way you want the get_class_folder_path to be adjusted, and I'll try running that on my way home from work 😊

Satal avatar Jan 29 '24 07:01 Satal

@Satal thanks to @VRSEN for showing where to make changes, I have done the following and it worked for me:

1- Use pathlib to handle the path/directory by adding the following line to the top of the agent.py file:

from pathlib import Path

2- Find the following lines in your agent.py file:

def get_class_folder_path(self): return os.path.abspath(os.path.dirname(inspect.getfile(self.__class__))) 3- Replace them with our newly added pathlib method:

def get_class_folder_path(self): return str(Path(__file__).parent.resolve())

4- Run the Genesis Agency Creator (I ran it in the gradio window) and now the agents are being properly created with their manifestos, files, etc. as seen in VRSEN's video demo.

I am not sure if this is the best way to go about it, but it worked for me. Please check on your end @Satal and confirm if you are also having some success with this.

I will do some more testing of the framework myself. The path issue seems to be resolved for me though with those changes. Cheers!

OscarsOrbit avatar Jan 29 '24 08:01 OscarsOrbit

I'm experiencing this on MAC even after the changes proposed from @OscarsOrbit It keeps using the last agency-created folder to nest underneath.

vlada-kopric avatar Jan 29 '24 16:01 vlada-kopric

@Satal @vlada-kopric @OscarsOrbit can you guys please reinstall from main and let me know if the problem persists? Thanks!

VRSEN avatar Jan 30 '24 04:01 VRSEN

Unfortunately, I'm still not seeing any agents created :-(

The following was all done on a Windows 11 laptop, using WSL2 (Ubuntu), Google Chrome (on laptop) for the browser) and Python 3.11.7. rm -rf'd the old agency swarm dir git cloned main created/activated a new conda env pip install -r requirements.txt Opened notebooks dir Ran jupyter notebook Opened genesis_agency.ipynb Changed the openai key ran all steps Opened gradio using link Sent " I need an agency to create multi-user laravel applications. For the front-end it should use Tailwind CSS and alpineJS"

I am seeing the following error at the beginning image

Also is the contents of agency_manifesto.md supposed to look like this? image

There is no manifesto.md

image

Satal avatar Jan 30 '24 08:01 Satal

Hey @Satal, if you ran the first cell in the notebook, you probably installed it from pip again, which has an old version. Try uninstalling it again and running pip install . from the root directory.

VRSEN avatar Jan 30 '24 11:01 VRSEN

Thanks @VRSEN, sorry about that. I've done it again but not running the pip install.

I'm now getting image It appears that one thing is going inside another.

Also when you go into agency.py this is the code I have image

Satal avatar Jan 30 '24 16:01 Satal

@Satal did you check the instructions of the genesis agency? Do they still show 'manifesto.md' or are they correct?

VRSEN avatar Jan 31 '24 02:01 VRSEN