Anton Osika

- [ ] Fix Napoleon - [ ] Fix the TODO strings in documentation - [ ] Use a custom domain name for the project - [ ] Rebuild if...

# Chain of thought role playing prompt that can answer e.g. SAT questions well Credit goes to reddit user [oddlyspecificnumber7](https://old.reddit.com/user/oddlyspecificnumber7). [His post](https://old.reddit.com/r/ChatGPT/comments/10bpzjb/chatgpt_scores_80_correct12_out_of_15_on_sample/) where he got 12/15 SAT questions right. I...

Add step to review all code, file by file, and rewrite any parts that miss necessary functionality

5

This should resolve issues like #119

Run the benchmark with GPT 3.5 over different —steps-config

4

We have scripts/benchmark.py. If we run it over more configs and store the results to RESULTS.md we will clearly be able to see what works and what does not. Would...

“Ask for feedback” step.

3

Create a step that asks “did it run/work/perfect”?, and store to memory folder. And let the benchmark.py script check that result, and convert it to a markdown table like benchmark/RESULTS.md...

Create a new "--steps-config" that runs a loop and asks for feedback in the end and feeds it back to GPT Engineer to fix

1

I think the "catch KeyboardInterrupt" in scripts/benchmark.py is a good pattern for being able to shut down the run, so that it goes to the "ask for input" step

New release not visible on sidebar

2

# Issue Template ## Expected Behavior Version 0.0.4 visible on https://github.com/AntonOsika/gpt-engineer right sidebar ## Current Behavior Version 0.0.3 visible on https://github.com/AntonOsika/gpt-engineer right sidebar ## More info: If one goes to...

Print and store how many tokens were used in memory/logs

6

In this way, we can also store this to benchmark results. A huge increase in tokens will not be worth a minor improvement in benchmark resultss.