comfyui-workspace-manager icon indicating copy to clipboard operation
comfyui-workspace-manager copied to clipboard

[backup workflow versions .json files in my_workflows/ folder]After backing up the workflow, versions history will be lost.

Open qh0814 opened this issue 2 years ago • 12 comments

Since the node supports saving version history, I save workflows as different history versions so that I can streamline the workflow list. But when I try to backup that workflow and reimport it, the workflow's history version is lost. How to back up a workflow as well as backing up the version history of that workflow? 截图 2024-03-02 18-11-03

qh0814 avatar Mar 02 '24 10:03 qh0814

But when I try to backup that workflow and reimport it,

hi how did you backup the workflow? for re-import I assume you clicked the "Import" button?

also did you change browser? or did you comfyui domain change? like from 127.0.0.1:8188 to somewhere else

Weixuanf avatar Mar 02 '24 11:03 Weixuanf

Thank you for your reply. I didn't change my browser or change the domain of comfyui. Here is what I did: I copied the json file for one of the workflows in the workspace save directory to somewhere else. Then I go back to comfyui and click the Import button and select the json file I just copied, at which point the workflow is automatically named "workflow 2". I switch to "workflow 2" and find nothing in the versions history.

qh0814 avatar Mar 02 '24 11:03 qh0814

ok I see. Yeah when you import a workflow, we are creating a brand new workflow with the same .json content with the same file name. That means, we don't copy the version history, gallery history attached with the original workflow.

How to back up a workflow as well as backing up the version history of that workflow?

I actually have been thinking about it, when people try to import an existing workflow, maybe we should prompt them with "Open existing", "Make a copy" or "Make a copy with version history" options. But the problem is that version history is stored in indexdb (browser storage), not in physical .json files like workflows, so even making copies, it's still stored in indexdb, not really making it safer

Screenshot 2024-03-02 at 9 08 04 PM

So right now we don't have good ways to backup your version history locally. you can try manually open each version and "Save As" to get physical .json files for backup. but you will see those versions as new workflows in your list.

But luckily we are testing a new feature to sync your workflow versions to cloud: Screenshot 2024-03-02 at 8 55 20 PM

Screenshot 2024-03-02 at 9 04 13 PM

let me know if you are interested for a test or have some suggestions! Thank you!

Weixuanf avatar Mar 02 '24 13:03 Weixuanf

I prefer to keep each history version locally managed rather than having to separate them with a browser, and I can even use git for version control if all versions of the workflow are locally managed. However, I am still looking forward to cloud synchronization feature, and hope to complete it soon.

qh0814 avatar Mar 02 '24 15:03 qh0814

I prefer to keep each history version locally managed rather than having to separate them with a browser,

I see, in that case, your best bet would be to use "Save As" to save each version with different names, it will create real files for you in disk

or maybe we can add a "Download all versions" button in Version History drawer, it will download a zip of all your versions .json files. Will that be helpful?

or if you have some better ideas, feel free to suggest it!

cheers 🍻

Weixuanf avatar Mar 02 '24 15:03 Weixuanf

or maybe we can add a "Download all versions" button in Version History drawer, it will download a zip of all your versions .json files. Will that be helpful?

Why not automatically save the current state of the workflow to local disk every time a historical version is created?The directory structure can be like this:

├── workspace save directory
    ├── workflow name
        ├── workflow version-0
        ├── workflow version-1

qh0814 avatar Mar 02 '24 16:03 qh0814

or maybe we can add a "Download all versions" button in Version History drawer, it will download a zip of all your versions .json files. Will that be helpful?

Why not automatically save the current state of the workflow to local disk every time a historical version is created?The directory structure can be like this:

├── workspace save directory
    ├── workflow name
        ├── workflow version-0
        ├── workflow version-1

I see, this is a good idea, but the only problem is that if saving this way, all these versions folder will appear in the files list, making the files list messy and crowded.

maybe we can do this, all workflow versions are saved inside __workflow_versions__ folder, this folder will not show up in your workflows list drawer

├── workspace save directory
    ├── __workflow_versions__
          ├── workflow name A
              ├── workflowA version-0
              ├── workflowA version-1
          ├── workflow name B
              ├── workflowB version-0
              ├── workflowB version-1

Thank you very much for your suggestion!

Weixuanf avatar Mar 02 '24 17:03 Weixuanf

I think the structure of the front-end UI display can be done without having the same file structure as the back-end, can we use json in the workflow save directory to manage these directories? The json file possible like this:

{
    "workflow list": [
      {
        "workflow_name": "Workflow name",
        "group": "group-1",
        "tags": ["tag-1,tag-2"],
        "description": "workflow description",
        "path": "workflow path",
        "versions": [
          {
            "version_name": "version name",
            "update_time": "2024-03-03 15:00:00",
            "tags": ["tag"],
            "description": "version description"
          },
          {
            "version_name": "version name",
            "update_time": "2024-03-03 15:01:00",
            "tags": ["tag-1,tag-2"],
            "description": "version description"
          }
        ]
      }
    ],
    "group list":["group-1","group-2"]
  }

qh0814 avatar Mar 03 '24 07:03 qh0814

Sorry for the late response. I have been busy working on v2.0 and now it's launched to beta branch, you are very welcome to check it out! We have two way sync, cloud sharing backup, etc.

I think the structure of the front-end UI display can be done without having the same file structure as the back-end

If we separate UI files structure from backend files structure, things will get very complicated because we need to support files moving, renaming, creation, deletion, and nested folders. It will be hard to sync files back and forth between UI and and backend files structure if they have different structure. And it may bring unexpected bugs. The most reliable way is to keep UI strictly reflects what's on your disk.

The .json you showed can work may work if we only support reading and listing your workflow files, but we need to support files moving, renaming, deletion, updating, etc. But we can use the structure you mentioned to only store versions data, for example,

{
    "workflow list": [
      {
        "workflow_name": "Workflow name",
        "id": "xhufrogtr2432",
        "path": "workflow path",
        "versions": [
          {
            "id": "hfruei26732"
            "version_name": "v1",
            "update_time": "2024-03-03 15:00:00",
            "json": "{workflow_json_string}"
          },
           {
            "id": "hfefruei77732",
            "version_name": "v12,
            "update_time": "2024-03-03 15:00:00",
            "json": "{workflow_json_string}"
          },
        ]
      }
    ],
  }

I think your main concern is how to backup or store the versions files in local disk instead of in browser. I think we can achieve that by having a separate hidden folder in /my_workflows folder called "interal_backup" or something, store the above mentioned structured versions .json there, or store 1 .json file for 1 version.

Weixuanf avatar Mar 05 '24 08:03 Weixuanf

Your consideration is reasonable, I prefer "store 1 .json file for 1 version", this way it can have better compatibility when sharing with others. Finally, I have a feature request, could you add a function to edit notes for workflows? Although comfyui currently comes with a note node, it must be accessed by entering the workflow and then moving to the specific location to view. If I could quickly view these contents in the workflow list, it would save time and also make the workflow look more aesthetically pleasing.

qh0814 avatar Mar 05 '24 11:03 qh0814

hi thanks for your understanding!

I prefer "store 1 .json file for 1 version", this way it can have better compatibility when sharing with others.

Yeah that makes sense. We can add a special folder inside /my_workflows to output/backup all your versions created there

If I could quickly view these contents in the workflow list, it would save time and also make the workflow look more aesthetically pleasing.

umm that sounds a good idea. I think we can extract the note node (if you have any) in the workflow and preview it on the workflow list. The only concern is that the current workflow list is already crowded and packed with many info, adding extra information may make the UI looks more messy.

Screenshot 2024-03-05 at 10 38 11 PM

The place i have in mind to display the note is ^^ under the workflow name title, at most 2 lines of preview of the note. If you have a better place in mind, you are very welcome to suggest it!

Weixuanf avatar Mar 05 '24 14:03 Weixuanf

Maybe when the mouse moves over the workflow title, a floating window will pop up on the right to display its note. Like this: float

qh0814 avatar Mar 05 '24 14:03 qh0814