stackwire-gpt
stackwire-gpt copied to clipboard
sisbell
A command line tool for running GPT commands. This tool supports prompt-batching and prompt-chaining.
If the user is expecting a text response, allow them to enter in a regex for validating. They may expect certain words or even a specific format in the response.
The developer should be able to configure max/min character/word and token count for a prompt response. If it falls outside of this range, this will register as a failure for...
Support running with a parent project. This will have - Main configuration - these can be overridden by the child project - modules for doing multi-runs of various project -...
There is a case where I have a prompt chain but I want the OpenAI configuration to be different. For example, the story prompt and the user-action prompt may be...
The plugin should be able to override the block configuration values. Currently configuration is per-blocked scope, which would require the dev to create a new block to modify parameters. ```...
Allow comments in prompt files for clarity. These need to be stripped out before creating the calculated prompt.
Add verbose flag - this will output configurations and information about the internal steps of the tool
The data file for a batch includes arrays of data to run against. Allow the dev to choose an from the command line. `air run -b product-1 --index 2` This...
Currently in CLI --blockId arg only supports a single ID. This means we can only run one block or all blocks. The tool should support a set of block IDs...
Create a plugin that will compare the performance of different runs. This will involve - Creation of box plots measuring request times - Measure relation of tokens to time taken...