developer icon indicating copy to clipboard operation
developer copied to clipboard

Having a ` character in the markdown prompt breaks smol

Open LukasMeine opened this issue 1 year ago • 6 comments

This is the error I get back in that case

 - `serverless.yml`   
SyntaxError: invalid syntax

LukasMeine avatar May 19 '23 22:05 LukasMeine

Same. For hacking, I replaced the ast.literal_eval line on 135 with

for line in filepaths_string.split('\n'):
    list_actual.append(line.split(' ')[1].replace('`',''))

terrafying avatar May 20 '23 16:05 terrafying

this is very strange. i definitely use backticks everywhere in my prompt: https://github.com/smol-ai/developer/blob/main/prompt.md

can you report the exact line of code that errors and what the input is? if like @terrafying is reporting, it is https://github.com/smol-ai/developer/blob/00a3366ac4411de102725d1115e045635b055394/main.py#L135 then i will try to work on a direct solution via smolai itself (i want it to just always return valid json)

swyxio avatar May 22 '23 00:05 swyxio

Same issue here. I'm using gpt-3.5-turbo

modal run main.py --prompt test-webapp/prompt.md --file package.json

prompt.md

a nodejs webapp that displays hello :name at the route /:name. include a package.json so I can run it with "npm start"

for the homepage, display a tutorial on how to use the webapp
I'd like an "npm run dev" command as well which reloads the server using nodemon.

## debugging notes

- sh: nodemon: command not found

output

modal run main.py --prompt test-webapp/prompt.md --file package.json
✓ Initialized. View app at https://modal.com/apps/ap-XXXXXXXXX
✓ Created objects.
├── 🔨 Created generate_response.
├── 🔨 Created mount /Users/jdan/Projects/smol-developer/main.py
└── 🔨 Created generate_file.
hi its me, 🐣the smol developer🐣! you said you wanted:
a nodejs webapp that displays hello :name at the route /:name. include a package.json so I can run it with "npm start"

for the homepage, display a tutorial on how to use the webapp
I'd like an "npm run dev" command as well which reloads the server using nodemon.

## debugging notes

- sh: nodemon: command not found
86 tokens in prompt: You are an AI developer who is trying to write a p
80 tokens in prompt: a nodejs webapp that displays hello :name at the r
- `./package.json`
- `./src/index.js`
- `./views/index.html`
- `./views/tutorial.html`

Here is the list of filepaths:

```
"./package.json"
"./src/index.js"
"./views/index.html"
"./views/tutorial.html"
```
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /opt/homebrew/Caskroom/miniforge/base/bin/modal:8 in <module>                                    │
│                                                                                                  │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(main())                                                                         │
│   9                                                                                              │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/modal/__main__.py:6 in main   │
│                                                                                                  │
│    5 def main():                                                                                 │
│ ❱  6 │   entrypoint_cli()                                                                        │
│    7                                                                                             │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/click/core.py:1130 in         │
│ __call__                                                                                         │
│                                                                                                  │
│   1129 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1130 │   │   return self.main(*args, **kwargs)                                                 │
│   1131                                                                                           │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/typer/core.py:778 in main     │
│                                                                                                  │
│   777 │   ) -> Any:                                                                              │
│ ❱ 778 │   │   return _main(                                                                      │
│   779 │   │   │   self,                                                                          │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/typer/core.py:216 in _main    │
│                                                                                                  │
│   215 │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                       │
│ ❱ 216 │   │   │   │   rv = self.invoke(ctx)                                                      │
│   217 │   │   │   │   if not standalone_mode:                                                    │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/click/core.py:1657 in invoke  │
│                                                                                                  │
│   1656 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1658                                                                                           │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/click/core.py:1657 in invoke  │
│                                                                                                  │
│   1656 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1658                                                                                           │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/click/core.py:1404 in invoke  │
│                                                                                                  │
│   1403 │   │   if self.callback is not None:                                                     │
│ ❱ 1404 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1405                                                                                           │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/click/core.py:760 in invoke   │
│                                                                                                  │
│    759 │   │   │   with ctx:                                                                     │
│ ❱  760 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    761                                                                                           │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/click/decorators.py:26 in     │
│ new_func                                                                                         │
│                                                                                                  │
│    25 │   def new_func(*args, **kwargs):  # type: ignore                                         │
│ ❱  26 │   │   return f(get_current_context(), *args, **kwargs)                                   │
│    27                                                                                            │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/site-packages/modal/cli/run.py:116 in f     │
│                                                                                                  │
│   115 │   │   │   else:                                                                          │
│ ❱ 116 │   │   │   │   func(*args, **kwargs)                                                      │
│   117 │   │   │   if app.function_invocations == 0:                                              │
│                                                                                                  │
│ /Users/jdan/Projects/smol-developer/main.py:135 in main                                          │
│                                                                                                  │
│   134 │   try:                                                                                   │
│ ❱ 135 │   │   list_actual = ast.literal_eval(filepaths_string)                                   │
│   136                                                                                            │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/ast.py:64 in literal_eval                   │
│                                                                                                  │
│     63 │   if isinstance(node_or_string, str):                                                   │
│ ❱   64 │   │   node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval')                 │
│     65 │   if isinstance(node_or_string, Expression):                                            │
│                                                                                                  │
│ /opt/homebrew/Caskroom/miniforge/base/lib/python3.10/ast.py:50 in parse                          │
│                                                                                                  │
│     49 │   # Else it should be an int giving the minor version for 3.x.                          │
│ ❱   50 │   return compile(source, filename, mode, flags,                                         │
│     51 │   │   │   │      _feature_version=feature_version)                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ - `./package.json`                                                                               │
│   ▲                                                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
SyntaxError: invalid syntax

jdan avatar May 29 '23 18:05 jdan

Printing filepaths_string shows:

- `./package.json`
- `./src/index.js`
- `./views/index.html`
- `./public/styles.css`

Here is the list of filepaths:

```
"./package.json",
"./src/index.js",
"./views/index.html",
"./public/styles.css"
```

jdan avatar May 29 '23 18:05 jdan

hey jdan! yeah ok this is another "doesnt return json" issue. short fix is to improve the prompt to not give us backticks like that. actual fix is to fix it at source... which i plan to do for smol.ai

swyxio avatar May 30 '23 04:05 swyxio

Suggest you just add the following to the prompt as another way to increase likelihood of proper output

Example output:
["index.html", "style.css", "script.js"]

clbarrell avatar Jun 11 '23 04:06 clbarrell