Acly
Wildcard support
I would like to request support for wildcards in Krita AI Diffusion.
Wildcards are a very important part of my creative workflow. They allow me to quickly generate variety and randomness in prompts, pulling from curated lists of words or concepts without having to manually rewrite prompts each time. This functionality is especially useful for:
- Rapid exploration: cycling through design variations (characters, outfits, colors, backgrounds, etc.) without re-typing.
- Creative iteration: testing multiple prompt combinations to discover unexpected results.
- Efficiency: reducing the friction of repetitive prompt editing when working on a large number of generations.
Many users coming from ComfyUI, Forge, or AUTOMATIC1111 are accustomed to this workflow, where wildcards dramatically accelerate idea generation. Not having wildcard support in Krita makes it harder to integrate it into established pipelines.
Proposed Implementation:
- A syntax similar to {option1|option2|option3} or referencing external .txt lists (like other UIs support).
- Ideally, nested wildcard expansion and random selection could be supported.
- Optional controls for deterministic vs. random expansion (e.g., seed-based).
Benefit: Implementing wildcard support would significantly enhance creative flexibility in Krita AI Diffusion and make it much more attractive to users who rely on procedural, exploratory workflows. This small feature has an outsized impact on productivity and creative discovery.
i'd suggest working with the impact nodepack which (i think) most users use for wildcard usage https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/tutorial/ImpactWildcard.md
I would also like wildcards, using the ImpactPack wildcard nodes for processing in the format __wildcard__. Any prompts with a wildcard (or multiple wildcards) so formatted could be passed to ImpactWildcardProcessor to get the processed text prompt.
I made an effort toward this feature a few months back, but never really finished the concept. At first, I rolled my own dynamic prompt evaluation by following the expected syntax of other wildcard solutions and that worked for my purposes. Then I investigated using the dynamicprompts library to use a more established and well tested solution, but didn't follow through with it. I still want to revisit this feature at some point, but one of the struggles I had was how to display to the user the exact prompt that resulted from the evaluation. I can see where a user might want to see and copy one or both, the original dynamic prompt and the evaluated prompt that actually generated the image. The current UI is not conducive to showing and handling both of these.
A basic short-term solution could be to check if the entered prompt has wildcards (words with double underscores __ surrounding them), and if so then pass that prompt to a ImpactWildcardProcessor node to process the prompt, getting back the final evaluated prompt to send along the workflow. The Impact pack is a fairly popular pack of nodes many users will probably already have that already does all of this. It would be even better if after typing a double underscore __ a popup list of available wildcards could be shown with an autocomplete search-as-you-type list of wildcards, similar to how available loras are added to the prompt after typing <lora:.
As far as showing the final prompt to the user in the UI, I'm not sure how that could be done. Seems like it would have to be some kind of websocket output node that sends the evaluated prompt text back to the plugin in the same way the generated image is passed back. Or it could be extracted from the metadata of the returned image (since the png should include the workflow embedded?). The evaluated prompt would just be the layer name, as the UI already shows the prompt as the layer name.
You're right, that was pretty easy. It can't send information back to know how the prompt was evaluated and it also requires Impact Pack nodes installed, but it was as simple as:
Add new function to ComfyWorkflow class in comfy_workflow.py
def impact_wildcard_processor(
self,
wildcard_text: str | Output,
mode: str | None = None,
seed: int = -1,
):
inputs = {"wildcard_text": wildcard_text, "populated_text": ""}
if mode in ("populate", "fixed", "reproduce"):
inputs["mode"] = mode
else:
inputs["mode"] = "populate"
if isinstance(seed, int) and seed >= 0:
inputs["seed"] = seed
return self.add("ImpactWildcardProcessor", 1, **inputs)
Change one line on TextPrompt class encode function
# Wildcard expansion first, then CLIP encode
processed_text = w.impact_wildcard_processor(text, seed=generate_seed(), mode="populate")
self._output = w.clip_text_encode(clip.model, processed_text)
We are overthinking this. The Impact Pack Server API already has HTTP endpoints to deal with wildcard refresh request, wildcard list request, and wildcard text processing request.
I have verified the first two work, but not yet sure how to do the processing requests. This would simplify the problem by not requiring the custom tooling nodes and API to be updated to send back the evaluated prompts. Instead, the impact pack API could evaluate the text prompt sent to it by this plugin and return the resultant prompt that we propagate throughout this plugin's code and UI and eventually send back to ComfyUI as the job request prompt.
Another benefit of this method is evaluating the prompt before it gets to the ComfyUI workflow, so embedded dynamic lora additions in the wildcard text will be properly handled.
The refresh request could be useful by putting a refresh button somewhere in the UI. The list request would be useful for autocomplete.
Oh, yeah, that sound much easier. I didn't know the Impact Pack had an API.
It still sounds like overkill to me. Adding custom node pack, more dependencies and client-server roundtrips just to do a little string replacement.
IMO there's no need to blow this up with its own recursive macro-like prompting mini-language. Things to make it accessible, like documentation, auto-complete, settings tab to configure simple replacements and folders to pull files from, will make it useful to a much wider audience.
There is already quite a bit of prompt processing happening in workflow.prepare (merge with style prompt, remove comments, handle lora...). Adding another call there is fairly simple, and the processed prompt can be pulled from the WorkflowInput return value to display as metadata.
That was the path I went down to start with, but it turns out that it is several thousand lines of code to adhere to the numerous accepted standards of wildcard support these days. It is a lot of opportunity to add bugs and a lot of code to have to maintain going forward. Plus, it is another location users have to add and maintain their list of wildcards, instead of leveraging what already exists and is well tested in ComfyUI nodes.
It sounds like more than just myself actually does want the full power of the recursive replacement language and that necessitates a lot of code to support it. If you do want a full feature support built-in to the plugin, that sounds okay to me, but a simple replacement feature wouldn't cover all my use cases. I guess I was just looking for the shortest putt to get the full feature working to satisfy the numerous requests for it. I didn't consider the additional client/server round-trip delay, as I have always used a local install.
Either way, several UI changes would be necessary to give the user proper insight to the dynamic prompt evaluation and to handle autocomplete and other configuration. That has given me pause to investigate further.
No I explicitly don't suggest a lot of code to match capabilities of other libraries/nodes, but rather go for a simple solution that brings 95% of the benefit. Focus on usability and accesibilty first rather than esoteric syntax for web UI power users.
Basic stuff:
-
{a|b|c}-> randomly select one -
__file__-> randomly select one from txt file
That seems to fit the proposed use case here, is simple, lightweight, and already provides value.
If you do that limited functionality, existing wildcard files, especially the more modern yaml files will not be usable and people will have to manually modify their hundreds or thousands of files to remove recursion. You will get lots of questions about why their wildcard files aren't working. I also find immediate variable substitution very useful as well, to have a dynamic term be the same if used in multiple places in the prompt.
Many users probably already have Impact Pack installed and it provides a lot of wildcard features. Having ComfyUI backend allows you to not remake every feature, but reuse what's already been built.
I did not see this issue sooner, but some days ago I sent a pull request to ComfyUI-Impact-Pack to allow the krita-ai-diffusion main node seed output to be used as ImpactWilcardProcessor seed input. Otherwise it crashes because the Impact Pack node only accepts only very few node types as seed input.
I can now use wildcards in Krita, at least when using custom workflows. 🙂
Once (if?) the PR is accepted it will be easier to implement it in krita-ai-diffusion.
[edit] : I read the issue way to fast... of course it won't be that simple to implement in the main code, since a lot of things has to be parsed from the prompt. Maybe if the impact-pack node is the very least in the workflow before the conditioning ones. 🤔
I won't pull in a huge node pack with multiple dependencies just to add wildcard support. It's like solving 1 problem and introducing 10 new problems, especially with any feedback loop requiring http requests.
@Danamir I have a branch on my fork to test out impact pack wildcard/dynamic prompt integration without needing a custom workflow. It is called dynamic_prompts_impact. It has a few options and useful features, like wildcard autocomplete and separate prompt vs dynamic template prompt history. I don't plan to merge it back here, since Acly doesn't want all the features of wildcards. I will just try to keep it up-to-date with the main branch for a while.
@nolan778 Thanks ! I'll take a look.
But I don't really use the wildcards that often, so using a custom workflow for this purpose is no big deal. I also use a fork on the side, so eventually I'll merge your code.
v1.41.0 has basic wildcards.
It does the main work of adding choices to metadata, allowing to restore interpolated prompts, documentation, etc.
There's no file support, but (simple text) files could be added relatively easily. Probably should also add a UI to select the folder where they are stored though.