Ability for extensions to provide custom extra_data to prompt

[IARI]

Use case:

An extension allows the user to store some session-specific settings in the web-ui. A custom node needs access to that data.

See the following discussion https://github.com/comfyanonymous/ComfyUI/discussions/3476

Description of the changes

This PR adds

• js extension method provideExtraData which allows an extension developer to provide additional data which is sent with ui-requests to the /prompt endpoint.
• custom nodes may specify hidden inputs with type "EXTRA_DATA". the parameter name is looked up in the extra_data object from the prompt, and if there is any, that data is passed to the execution function of the custom node.

Example Code

js

app.registerExtension({
    name: "my-example-extension",
    async provideExtraData() {
        return {my_custom_data: { hello: "world"}}
    }
})

python

class MyExampleNode:
    @classmethod
    def INPUT_TYPES(s):
        return {
            "hidden": {
                "my_custom_data": "EXTRA_DATA"
            }
        }

    RETURN_TYPES = ("FLOAT",)
    CATEGORY = "example"
    FUNCTION = "extra_data_example"
    OUTPUT_NODE = False

    def extra_data_example(self, my_custom_data=None):
        if my_custom_data is None:
            print("extra_data my_custom_data is None")
        else:
            print("my_custom_data " + json.dumps(my_custom_data, indent=4)")

        return 1.0 # some fake output, just so the node can be executed

[RandomGitUser321]

How safe is something like this? Sounds like a good way for bad actors to potentially inject malicious exploits into workflows.

[IARI]

@RandomGitUser321 thats a good point. I was also thinking that maybe this could be exploitet, here's my assesment:

clientside

• What this PRs code does: add js data which will be stringified and attachted to the prompt.
• How you can already hack it right now: As far as I can tell, nothing prevents you from importing and overwriting any of the already existing js api code. particularly your extension could easily import api.js and override the api.queuePrompt method. For example ComfyUi Manager does this in these places with:
  • app.handleFile
  • app.loadGraphData
  • app.graphToPrompt

serverside

• What this PRs code does: It offers any custom node potentially access to all of the extra_data, where before it was only possible to access specific fields (namely the prompt, extra_pnginfo and unique_id)

• Situation right now: I think there is no way to directly access other than the already defined extra_data data from a custom node so far.

possible exploit

It is conceivable that an extension A provides sensitive data with extra_data. Another malicious extension could access extra_data from A with this PR.

Is there a point in trying to 'fix' something?

I could try that the extra data that extension A provides can only be read by nodes that also come from A. But then again that's nearly impossible to accomplish, because there is no security with the client side. Any extension can always overwrite methods from the api and thereby probably already intercept everything that ComfyUi and any other installed extension is doing clientside... I don't really see a point fixing a small hole in the window when there's no roof on top.

Am I missing something critical, like any real vulnerability that isn't already there?

[mcmonkey4eva]

frontend is replaced by https://github.com/Comfy-Org/ComfyUI_frontend now