progrockdiffusion icon indicating copy to clipboard operation
progrockdiffusion copied to clipboard

Question: Prompts at different steps of creation in prdgui.py

Open WorldTeacher opened this issue 2 years ago • 14 comments

I have a question regarding the prompt-fields in the GUI.

Can I set them to apply at different steps, like eg.: 0: ['An oilpainting of a lighthouse in the arctic'] 100:['fog, clouds, shor at dawn'] etc?

or is this only possible in the cli?

This would be useful to "nudge" DD in a certain direction at specific points of the creation.

WorldTeacher avatar Aug 09 '22 10:08 WorldTeacher

At the moment no, that's only possible with the CLI. The goal (of sorts) with the GUI was to provide a way to access the most common / bare minimum settings since editing a settings file can be overwhelming, especially for new users. The tricky thing with switching prompts at a certain step is it requires the prompts to be provided in a slightly different format, which makes doing it in the GUI tricky as well. Not impossible, but I would want to make sure from a UI standpoint that it's still clean and clear how to do it. Perhaps @KnoBuddy can have a think about it, or we can brainstorm a way.

lowfuel avatar Aug 09 '22 12:08 lowfuel

I mean, it's certainly possible. The way the prompt is handled in the json, it's just another nested dictionary that can be indexed. So adjusting it's value isn't an issue, although it is quite tricky and finicky but it can be done.

My concern, as has been with all features, is fitting them into the GUI nicely. If I start making frames/panels that pop up and appear and disappear the code is going to explode. While at it, I probably should get some things broken out into classes, as well. It's not something I'm really interested in doing right now, I've got other projects I'm working on currently. Although I managed to find time for the last feature so I wouldn't count anything out completely.

The real issue is: at what point are panels and buttons disappearing and reappearing more complicated than just opening and editing a text file?

I don't want to make a GUI that is more difficult to use than the settings file it is supposed to replace.

KnoBuddy avatar Aug 09 '22 13:08 KnoBuddy

I can imagine a button to add "prompts" and a button to add new "prompt step".

The first button increases amount of the currently displayed prompts, the second button adds a new tab that allows you to input a step and as many new prompts that you want to. You could do this theoretically to infinity.

Sounds nice, we'll see if I find the motivation to code it lol.

KnoBuddy avatar Aug 09 '22 13:08 KnoBuddy

this is just me thinking a bit, as i don't know much about gui input detection and DD, but would it be hard to detect if the user inputs a number before the prompt like

0: prompt bla bla bla
50: prompt2 bla bla

or would it be easier to do it like the 'prompts' and 'prompt step' @KnoBuddy was thinking about?

WorldTeacher avatar Aug 09 '22 15:08 WorldTeacher

Doing that would mess with the current multi-prompt with weights. The suggestion I made with the prompts and prompt steps would be completely modular.

KnoBuddy avatar Aug 09 '22 16:08 KnoBuddy

true, didn't think about that, please disregard the previous comment

WorldTeacher avatar Aug 09 '22 20:08 WorldTeacher

Also I just tried a settings.json with prd.py and got this error:

Parsing settings.json
Parsing mysettings/settings.json
Failed to open or parse mysettings/settings.json - Check formatting.
<string>:11 Unexpected "c" at column 16

and this was the settings file I was trying to load, a copy of the normal settings file: see pastebin

will try this again tomorrow or so, when I have time, maybe I'll figure it out

WorldTeacher avatar Aug 09 '22 20:08 WorldTeacher

You have missing quotes on line 11.

mhop1 avatar Aug 09 '22 20:08 mhop1

Also, to change the prompt within an individual image you need to format it slightly differently (this is why doing it in the gui would require extra processing and UI). The way you have it right now is how you'd format it to use those prompts at image 250, 750, etc, in a batch. What you want is to do it like this:

    "text_prompts": {
        "0": {"0": ["A hyper realistic octane render of a massive aged tower reaching the sky, by greg rutkowski and jesper ejsing, detailed, dreadful, 8k resolution, trending on artstation"],
        "250":["alternate dimension, vivid colors, mandala"],
        "500":["lightning flashes around the tower"],
        "750":[clouds, stars flickering through the clouds"]},
    },

Note how it's nested in another set of {}, telling it these prompts are all for image 0. You can look at the change_prompt_sample.json in the settings folder for another example that includes multiple images in a batch.

lowfuel avatar Aug 10 '22 13:08 lowfuel

image

Well you got me started...

KnoBuddy avatar Aug 10 '22 22:08 KnoBuddy

Sorry for the late reply, some irl stuff hapened. Thanks for letting me know that there is some progress :+1:

WorldTeacher avatar Aug 12 '22 12:08 WorldTeacher

The way PRD handles these nested dictionaries, which contain lists at different levels of nesting, is driving me nuts. Making it work for both multiple prompts and multiple steps as well as handle the default multiple prompts and single steps, is quite the challenge, requiring me to recode entire portions for each functionality. It's in progress, but as I haven't ever used prompt scheduling myself, I don't know how worth while this effort is.

KnoBuddy avatar Aug 12 '22 16:08 KnoBuddy

Personally I agree. I think of it as an advanced technique and that goes against the spirit of the GUI, imho.

lowfuel avatar Aug 12 '22 16:08 lowfuel

Have moved PRD GUI to:

https://GitHub.com/KnoBuddy/VisualDiffusion

I do plan to implement more advanced features, but they will be done in a way that doesn't destroy the original GUI. Maybe even a second separate script for advanced features.

KnoBuddy avatar Aug 13 '22 15:08 KnoBuddy

closing this, since it is now working in VisualDiffusion

WorldTeacher avatar Sep 05 '22 07:09 WorldTeacher