ComfyUI_NAIDGenerator icon indicating copy to clipboard operation
ComfyUI_NAIDGenerator copied to clipboard

Request for NAI Weight Converter

Open da2el-ai opened this issue 1 year ago • 2 comments

Thank you for creating such wonderful custom nodes! They're so useful that I use them every day.

I have a request, if I may. I'd like a custom node that converts NAI-style weights to SD-style weights. For example: {{foo}} > (foo:1.1)

The custom node "D2 Load Image" that I've published has a feature to extract prompts from images, and it's compatible with NovelAI images. If there were a custom node to convert NAI weights to SD weights, it would make prompt reuse much easier, which would be fantastic. https://github.com/da2el-ai/D2-nodes-ComfyUI

I'm not sure if it will be helpful, but here's the JavaScript source I created: https://github.com/da2el-ai/sd-d2-nai-prompt-convert/blob/4d3da2a0ff7ff4cda7d6726fee28b96f96233e2e/src/js/D2NPCPromptConvert.ts#L40

da2el-ai avatar Oct 08 '24 08:10 da2el-ai

Thank you for the request!

the difficult part of NAI weight to SD weight is that multiple brackets doesn't end at same time. for example, the below prompt is also valid. {{1girl, {black hair, chibi,}}} {{catgirl [[under]]}} a {{{{{cherry blossom tree https://docs.novelai.net/image/strengthening-weakening.html#note And converters I found did not works correctly at that condition.

However, I will try best to meet the requirement. Please wait for a while.

bedovyy avatar Oct 11 '24 02:10 bedovyy

I've noticed some additional complexity in NAI's weight handling that's worth considering:

  1. NAI processes bracket weight inheritance differently from SD's standard multipliers. For example: {{character, {red hair}}} in NAI doesn't translate directly to (character:1.1, (red hair:1.1):1.1) in SD - the weights handle differently.

  2. Alternative bracket types like [[ ]] and ( ) can mix with curly brackets in NAI, and their interactions aren't always straightforward. Example: {{wild [flower]}} vs {wild {flower}}

  3. The way NAI handles weights at phrase boundaries vs word boundaries differs:

{{red hair}} blonde -> affects 'red hair' as a unit
{{red}} hair blonde -> affects just 'red'

These patterns might need special handling in the converter to ensure accurate weight translation between the systems.

Mooshieblob1 avatar Dec 24 '24 05:12 Mooshieblob1