arteme
ChatGPT support
I recently discovered that ChatGPT is really good at taking effect descriptions and dumping out parameters to match... like "create an underwater piano sound" or "create a crunchy slap bass effect".
I'd like to implement support that allows ChatGPT to easily update POD settings using your tool.
--Dev Effort--
- create an xml schema for the pod-ui data,
- implement file i/o for pod-ui to read in an xml file, then unmarshal into the effect params
- add file watch capabilities that automatically reloads the file and applies params
-- Use Case -- Me: "ChatGPT, here is my schema that defines parameters for my effects processor" Me: "ChatGPT, using that schema, create an effect that sounds like a creepy toy piano and save it to PodEffect.xml"
- ChatGPT creates the PodEffect.xml file and gives the download link
- I manually download the file
- Pod-UI notices that the file has changed and auto loads it
- Pod-UI applies the changes
- I test the effect
Me: "ChatGPT, the low end is overpowering the mix — dial it back a bit. Also, introduce some distortion and give the reverb more depth"
- ChatGPT updates the xml file and give the new download link
etc....
I'm considering taking this task on as a prototype, I think MVP should be fairly straight forward. Are you interested in collaborating and formally implementing a feature like this? or should i just run with it solo? (I'm a senior C++ dev over 25yrs of dev experience, but have never used Rust.)
I have a prototype working for the PODxt. its really fun to play with. I mostly use the POD to add effects to keyboard samples, but I wonder if you could say "Give me the Enter Sandman guitar effect" and it could get close. Let me know if you want me to demo it.
I have to admit, when the "ChatGPT support for pod-ui" message popped into my inbox, I had a number of competing thoughts:
- This must be a joke!
- ChatGPT must've generated a bogus ticket!
- Do we need AI everywhere?!
However, having read the issue, I can totally see your use-case for it. I have to admit, I'm hopeless with getting the right tone so I really see the value here. I'd love to collaborate on this.
While file saving and loading would be needed at some point, I really think this can be done with a simple networked API and/or CLI. I don't know how exactly you plan to integrate with ChatGPT, but it should be relatively straight-forward to dump parameters (https://github.com/arteme/pod-ui/blob/master/mod-xt/src/config.rs#L199) via a CLI that talks to a running pod-ui.
We have the following types of controls:
- range controls - the most typical controls such as "bass", "mid", "trebble" - which are just a value of 0..127;
- these can be simple percentage (such as "bass", etc) 0..127=0..100%;
- or a linear function (such as "compressor threshold")
v = 63/127*x - 63; - some sort of multi-step linear interpolation (like eq virtual controls);
- switch controls - for things like "wah enable", "stomp enable", "wah position" - there are just on/off switches;
- select controls - like "amp select", "effect select" - which are range controls with control-specific upper limit;
- virtual range controls - specialized range controls where the values and upper limit are control-specific;
Getting pure values from pod-ui should be rather straight-forward, modelling the data for ChatGPT - I can't imagine ;) Maybe I'm overthinking and you don't need the AI to know the difference between the stomp effects (https://github.com/arteme/pod-ui/blob/master/mod-xt/src/config.rs#L85) and their parameters.
I'm curious to hear your thought on this.
The cool thing is that ChatGPT is already trained on the various delay, stomp effects, params, etc. it already seems to know what its doing! We just need to formalize the data for it.
My solution is very straight forward, and very hacky. But, as a proof of concept, it works fine. I generated the attached PODxt json schema and added comments that ChatGPT uses for context. (i know I'm missing some of your data, and there are errors, but it seems fine for a proof of concept)
Then I implemented a filewatcher that watches a specific file ("patch.json") in my downloads folder. On startup, I spawn a filewatcher thread and leave it running. When the watched file is updated the thread parses the json, converts it to a map, then shoves it into your controller to be sent to my device. (I'm not a Rust dev, so cut me some slack, there is no attempt at thread safety here)
To use it.... I feed the schema to ChatGPT and prompt it with something like
"Create an effect for a church organ that makes it sound larger than life and will make the hairs stand up on your neck, slight distortion, but not much.
Use the schema provided. Output must include every parameter from the schema with valid values. Format the result as a downloadable patch.json."
I download the ChatGPT file and pod-ui auto updates my device accordingly.
Its super fun to play with.!
As a test, what i built works fine, but I agree that direct ChatGPT integration is a better formal solution.
Feel free to grab my solution to see how it works for you.
Just had a chat with ChatGPT regarding this project idea (which is awesome btw!) and I think it’s worth sharing here. Might be a kick-start and or a source of inspiration for the project.
Link to the ChatGPT convo: https://chatgpt.com/share/68a28131-ee6c-8001-a11a-7f268ca0232d
P.s., Finding your POD UI project has been such a blessing. Thank you for your amazing work!
