Proposal: Storing Model Information in Workflow JSON

[robinjhuang]

Problem

The workflow json is the primary way ComfyUI workflows are shared online. When dragging in a workflow, it is sometimes difficult to know exactly which model was used in the workflow.

Knowing the exact model that was used can be crucial for reproducing the result in the workflow output.

Proposed Enhancement

Capture the model information directly in the workflow json.

{
	...workflow json,
	"models": [{
		"name": "sdxl",
		"url": "",
		"sha256": 75be28421e846fec54cea35951aadfb8c9617e0698481bfe5a5efe559f3a0497,
		"directory": "/checkpoints",
	}]
}

Saving the URL

Goal: Save download url for each model file.

Save a parallel model file under model-name.json . This idea is stolen from StableSwarm (thank you @mcmonkey4eva). The benefits of having a parallel file is that if a user moves/renames a model file, they will see the parallel file and move it as well.

{
   "url": "http://example.com/model",
   "sha256": "abc123..."
}

ComfyUI Manager / CLI

ComfyUI Manager and comfy-cli should create a parallel model file when they are used to download a model.

Getting the Hash

1. Civitai - Look up hash using API
2. huggingface - not sure?
3. Other model hosting places - not sure?

We can calculate it locally. Took me 30s for a 4.5GB model file on a m1pro macbook. We can prompt users if they want to include it or add an optional in settings to opt-in/out.

Approach 1: Calculate hash at import time

As soon as the model is moved into the models folder, calculate the hash. Calculating the hash is CPU-only, so should not impact workflow generation speeds.

Approach 2: Calculate hash at export time

Ask the user if they want to include model hash when exporting the workflow and then calculate them.

Exporting Workflow JSON

When exporting the workflow json, Comfy can reference the parallel model file to check the model metadata.

Unresolved Problems

1. If the user renames the file without renaming the parallel json file, it will be difficult to track the information. The hash is the only reliable identifier and can be used as a lookup id, but this can be expensive to recalculate. If comfy had a built in file explorer this problem could be improved upon but it is still not foolproof.

[robinjhuang]

This proposal came out of the node-standard discussions at the Comfy Leadership Summit last month. Let me know if there are better solutions for any of the approaches, or if this is a bad idea altogether.

[teward]

This has a prerequisite: that the model be somewhere for downloading, or that the model exists on a site where we can API-query to check the sha sums.

A number of models I play with are made privately and not shared everywhere, so these would be edge cases that wouldn't work.

Additionally, we have a Security concern here - if not carefully handled this leads to malware link sharing. Model metadata also needs to have an official URL in it for downloading, and not all model sharing sites/systems provide such metadata - while that doesnt reduce the Security risk, it adds some problems when trying to store workflow data if metadata exists.

I agree including model, lora, etc. hash sums is a benefit, but this has a lot of requisite steps in my opinion before this can even be implemented.

[ltdrdata]

Even for models that don't exist publicly, if they are available locally, you can obtain their hash. However, to properly utilize this information, a service like the Civitai API would be necessary.

I think including links is inappropriate, but tracking the hash is a good idea.
It would be best to record the information only for the checkpoints used within the workflow, storing it in an extra field.

[bigcat88]

I think including links is inappropriate, but tracking the hash is a good idea.

can we expect the ability to optionally store "hash" of models used in workflow during "Save (API format)" action in ComfyUI?

[ltdrdata]

I think including links is inappropriate, but tracking the hash is a good idea.

can we expect the ability to optionally store "hash" of models used in workflow during "Save (API format)" action in ComfyUI?

Request to here:
https://github.com/Comfy-Org/ComfyUI_frontend/issues