gptscript icon indicating copy to clipboard operation
gptscript copied to clipboard

Add more info to system prompt to allow for the tools to process the user provided prompts in a more orderly way.

Open sangee2004 opened this issue 11 months ago • 3 comments

When i need the tool to process a set of steps , it is not enough that i state the steps in an order like this example:

Tools:  github.com/gptscript-ai/vision,  github.com/gptscript-ai/image-generation , sys.write, sys.read, sys.download

1. Generate an image of a cow grazing in the open green meadows and return only the url of the image.
2. Download the url of the image generated in step 1 and write to a file cow.png.
3. Describe the image at cow.png in the style of a Wes Anderson character.

When i run the above script , i always end up getting the following error:

10:59:00 sent     [main]
10:59:00 started  [sys.read(2)] [input={"filename": "cow.png"}]
10:59:00 sent     [sys.read(2)]
10:59:00 started  [https://raw.githubusercontent.com/gptscript-ai/image-generation/b9d9ed60c25da7c0e01d504a7219d1c6e460fe80/tool.gpt(3)] [input={"prompt": "a cow grazing in the open green meadows", "quality": "hd"}]
10:59:00 sent     [https://raw.githubusercontent.com/gptscript-ai/image-generation/b9d9ed60c25da7c0e01d504a7219d1c6e460fe80/tool.gpt(3)]
10:59:00 failed to run tool [] cmd [/usr/bin/env python3 /Users/sangeethahariharan/Library/Caches/gptscript/repos/b9d9ed60c25da7c0e01d504a7219d1c6e460fe80/python3.12/cli.py --prompt=a cow grazing in the open green meadows --size= --quality=hd --number=]: context canceled
2024/03/07 10:59:00 open cow.png: no such file or directory

When i include the following lines additionally in prompt - Follow all steps in order and move to the next step only after the first step is done completely. , it works as expected.

Tools:  github.com/gptscript-ai/vision,  github.com/gptscript-ai/image-generation , sys.write, sys.read, sys.download

Follow all steps in order and move to the next step only after the first step is done completely.

1. Generate an image of a cow grazing in the open green meadows and return only the url of the image.
2. Download the url of the image generated in step 1 and write to a file cow.png.
3. Describe the image at cow.png in the style of a Wes Anderson character.

Should we consider adding something along the lines of Follow all steps in order and move to the next step only after the first step is done completely. to our existing system prompt ?

sangee2004 avatar Mar 07 '24 19:03 sangee2004

We have done this.. did it change anything in a meaningful way? I've heard some conflicting reports.

drpebcak avatar Apr 18 '24 23:04 drpebcak

After the change, I still see steps execute in parallel. I usually need to add "NEVER RUN STEPS IN PARALLEL!" to get gpt-4o to play nice.

njhale avatar Jul 09 '24 05:07 njhale

Hmph. @ibuildthecloud any thoughts here?

cjellick avatar Jul 11 '24 23:07 cjellick