Let's level up the smol developer.

[linsomniac]

Defining Requirements

I really struggled with getting my first prompt to feed to smol-dev, partly because my first prompt was way, way too big and included too much information.

Let's get smol-dev to ask for the requirements it needs to get started. I used this:

System prompt: You are a prompt generation robot. you need to gather information about user goals, objecives, examples of preferred output, and other relevant content. The prompt should include all of the necessary information that was provided to you. Ask follow up questions to the user until you have confidence you can produce the perfect prompt. Your return should be formatted clearly and optimized for gpt interactions. Start by asking user goals, desired output and additional information you may need.

User prompt: I want to create a python application using FastAPI that mimics an existing Ruby on Rails application we have. The existing rails app makes use of ActiveRecord for many of the API endpoints for things like updating the records via "edit". The application is a simple "cart" of "products", each "product" is identified by an integer. The database has an "account" for each user, and each "account" can have multiple "products" associated with it. The user may login and logout as well as resetting their password.

It then asked me a series of questions which I would submit using the "assistant" and "user" additional parameters to keep the dialog flowing, until it generated a final prompt.

I then used this to feed to the first stage of smol-dev

Adding Additional Context

In my case, I had additional context that I wanted to give to GPT, but would cause the prompt to blow up. So I would "fake" additional dialog with GPT to give it additional context. I also tried telling smol-dev to ask for more details if it needed them to be able to generate the code.

For example, in file generation I might use:

System Prompt:

    You are a Python FastAPI code generation AI robot.  You specialize in writing one single file of the requested application.  You generate a complete, easily readable and maintainable source code file including type annotations and doc strings which will achieve the user's goals.
    Ask follow up questions to the user until you have confidence you can produce the perfect file list.  Start by asking the user for additional information you may need to generate the file list.  Once you are confident you understand the problem, begin generating the code for the file.

    Only write valid code for the given filepath and file type, and return only the code.
    Do not add any other explanation, only return valid code for that file type.

    Remember that you must obey 3 things:
       - you are generating code for the file "controllers.py"
       - do not stray from the names of the files and the shared dependencies we have decided on
       - MOST IMPORTANT OF ALL - the purpose of our app is specified by the user prompt - every line of code you generate must be valid code.

    The files we have decided to generate are:
    - [the list of files]
  
    And the shared dependencies are:
    -  [the list of dependencies]

User Prompt: [The prompt generated above]

Assistant response (faked): Can you tell me more about the data model for the listings and carts? For example, what are the attributes of a "listing", and how are listings associated with carts and accounts? How are they stored in the database?

User prompt: The database model looks like this: [Schema]

Assistant response (faked): To clarify, do you want the Python FastAPI application to have the same functionality as the existing Ruby on Rails app, including the use of ActiveRecord for the API endpoints? Are there any specific endpoints or features that are especially important to replicate?

User prompt: Here is a list of the endpoints I want to provide: [API endpoints]

It would then go on to generate code.

Thoughts?

[panuganti]

Did this work for you ?

[linsomniac]

It seemed to do a good job of the process, but I haven't yet had a chance to evaluate the code produced by it.

[swyxio]

haha thank you for the suggestions @linsomniac! i definitely feel like we need to build some tooling to evaluate the benefits of different prompts, and test for regressions. adding to todos...