TypeChat
TypeChat copied to clipboard
Published
20 hours ago •
microsoft
microsoft
New (Python) example: simple drawings (rectangle, ellipse, arrow)
For me, this works best with openai model gpt-3.5-turbo; gpt-4 seems worse (?!).
Here's a sample session (no longer valid without history support)
PATH=/opt/homebrew/opt/ccache/libexec:/opt/homebrew/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Users/guido/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/bash:/Users/guido/TypeChat/python/.venv/bin:/Library/Frameworks/Python.framework/Versions/3.13/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/Users/guido/.nvm/versions/node/v19.8.1/bin:/opt/homebrew/opt/ccache/libexec:/opt/homebrew/bin
The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
(.venv) ~/TypeChat/python$ PYTHONPATH=src python examples/drawing/demo.py
~> draw three squares of side 50 in a diagonal
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n type: "Unknown";\n // The text that wasn\'t understood\n text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n type: "Box";\n // Top left corner coordinates\n x: number;\n y: number;\n // Size of the box\n width: number;\n height: number;\n // Text centered in the box\n text: string;\n // Box drawing style (optional)\n style: Style | null;\n}\n\ninterface Style {\n type: "Style";\n corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw three squares of side 50 in a diagonal\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}
retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
ReadTimeout('') raised from within internal TypeChat language model.
~> draw two squares of side 50 in a diagonal
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n type: "Unknown";\n // The text that wasn\'t understood\n text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n type: "Box";\n // Top left corner coordinates\n x: number;\n y: number;\n // Size of the box\n width: number;\n height: number;\n // Text centered in the box\n text: string;\n // Box drawing style (optional)\n style: Style | null;\n}\n\ninterface Style {\n type: "Style";\n corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}
{
"items": [
{
"type": "Box",
"x": 0,
"y": 0,
"width": 50,
"height": 50,
"text": "",
"style": null
},
{
"type": "Box",
"x": 50,
"y": 50,
"width": 50,
"height": 50,
"text": "",
"style": null
}
]
}
~> make it three squares
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n type: "Unknown";\n // The text that wasn\'t understood\n text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n type: "Box";\n // Top left corner coordinates\n x: number;\n y: number;\n // Size of the box\n width: number;\n height: number;\n // Text centered in the box\n text: string;\n // Box drawing style (optional)\n style: Style | null;\n}\n\ninterface Style {\n type: "Style";\n corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n "items": [\n {\n "type": "Box",\n "x": 0,\n "y": 0,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n },\n {\n "type": "Box",\n "x": 50,\n "y": 50,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n }\n ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}
retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
{
"items": [
{
"type": "Box",
"x": 0,
"y": 0,
"width": 50,
"height": 50,
"text": "",
"style": null
},
{
"type": "Box",
"x": 50,
"y": 50,
"width": 50,
"height": 50,
"text": "",
"style": null
},
{
"type": "Box",
"x": 100,
"y": 100,
"width": 50,
"height": 50,
"text": "",
"style": null
}
]
}
~> label them according to size
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n type: "Unknown";\n // The text that wasn\'t understood\n text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n type: "Box";\n // Top left corner coordinates\n x: number;\n y: number;\n // Size of the box\n width: number;\n height: number;\n // Text centered in the box\n text: string;\n // Box drawing style (optional)\n style: Style | null;\n}\n\ninterface Style {\n type: "Style";\n corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n "items": [\n {\n "type": "Box",\n "x": 0,\n "y": 0,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n },\n {\n "type": "Box",\n "x": 50,\n "y": 50,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n }\n ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n "items": [\n {\n "type": "Box",\n "x": 0,\n "y": 0,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n },\n {\n "type": "Box",\n "x": 50,\n "y": 50,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n },\n {\n "type": "Box",\n "x": 100,\n "y": 100,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n }\n ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nlabel them according to size\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}
{
"items": [
{
"type": "Box",
"x": 0,
"y": 0,
"width": 50,
"height": 50,
"text": "Small",
"style": null
},
{
"type": "Box",
"x": 50,
"y": 50,
"width": 50,
"height": 50,
"text": "Medium",
"style": null
},
{
"type": "Box",
"x": 100,
"y": 100,
"width": 50,
"height": 50,
"text": "Large",
"style": null
}
]
}
~> oops, label them according to position
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n type: "Unknown";\n // The text that wasn\'t understood\n text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n type: "Box";\n // Top left corner coordinates\n x: number;\n y: number;\n // Size of the box\n width: number;\n height: number;\n // Text centered in the box\n text: string;\n // Box drawing style (optional)\n style: Style | null;\n}\n\ninterface Style {\n type: "Style";\n corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n "items": [\n {\n "type": "Box",\n "x": 0,\n "y": 0,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n },\n {\n "type": "Box",\n "x": 50,\n "y": 50,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n }\n ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n "items": [\n {\n "type": "Box",\n "x": 0,\n "y": 0,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n },\n {\n "type": "Box",\n "x": 50,\n "y": 50,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n },\n {\n "type": "Box",\n "x": 100,\n "y": 100,\n "width": 50,\n "height": 50,\n "text": "",\n "style": null\n }\n ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nlabel them according to size\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n "items": [\n {\n "type": "Box",\n "x": 0,\n "y": 0,\n "width": 50,\n "height": 50,\n "text": "Small",\n "style": null\n },\n {\n "type": "Box",\n "x": 50,\n "y": 50,\n "width": 50,\n "height": 50,\n "text": "Medium",\n "style": null\n },\n {\n "type": "Box",\n "x": 100,\n "y": 100,\n "width": 50,\n "height": 50,\n "text": "Large",\n "style": null\n }\n ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\noops, label them according to position\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}
retrying (ReadTimeout('')) ...
{
"items": [
{
"type": "Box",
"x": 0,
"y": 0,
"width": 50,
"height": 50,
"text": "First",
"style": null
},
{
"type": "Box",
"x": 50,
"y": 50,
"width": 50,
"height": 50,
"text": "Second",
"style": null
},
{
"type": "Box",
"x": 100,
"y": 100,
"width": 50,
"height": 50,
"text": "Third",
"style": null
}
]
}
~>
There's some stuff here that makes intuitive sense, but I'll need to check in with others (CC @ahejlsberg)
I would think that the best format for incorporating chat history would be something like
- System: The assistant is a bot that responds in JSON according to a schema.
- ...: In-between messages alternating between user/assistant
- User: Some request
- System: Translate the prior request with JSON that satisfies Type in the following schema:
That gives background for the current convo, plus reinforces the current task at hand. This PR is pretty close to that.
I wonder how to proceed. Clearly this PR two things (adding history and adding a new example) and that's not great. Your fixes to the translator (#240) broke my PR and I would rather not try to merge it back. I propose to add my example without chat history and start a separate discussion on chat history -- okay?