Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resize control if there is a tensor size mismatch #3917

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

asagi4
Copy link
Contributor

@asagi4 asagi4 commented Jul 1, 2024

Fixes PatchModelAddDownscale when used with controlnets, and possibly some other nodes doing similar things as well.

Fixes #3022

I'm not sure if there's a situation where the operation can fail after this, so I didn't remove the try/except

Fixes PatchModelAddDownscale when used with controlnets, and possibly some other
nodes doing similar things as well.

Fixes comfyanonymous#3022
@comfyanonymous
Copy link
Owner

Does this actually work and would it be better to use nearest instead of bicubic?

@asagi4
Copy link
Contributor Author

asagi4 commented Jul 1, 2024

I've used this locally for quite a while (never just got around to sending a PR) and it does at least let the controlnet do something, which is an improvement over things not working at all (Instead you just get a bunch of "warning control could not be applied torch.Size([2, 1280, 20, 16]) torch.Size([2, 1280, 40, 32])" ). You can also look at the linked issue where a couple people have reported using it.

I haven't tested different algorithms.

@mcmonkey4eva
Copy link
Contributor

"it does something", or "it does what it's supposed to do"?

Is this an attempt at hacking SD1 and SDXL controlnets to cross over, or...?

@asagi4
Copy link
Contributor Author

asagi4 commented Jul 3, 2024

Iit allows the controlnet to work instead of failing and doing nothing when PatchModelAddDownscale does its downscaling, by scaling the control tensor to match; this has nothing to do with mixing models.

@mcmonkey4eva
Copy link
Contributor

ooh the PatchModelAddDownscale (Kohya Deep Shrink) node, my bad, I skimmed past that the first look.

@mcmonkey4eva mcmonkey4eva added User Support A user needs help with something, probably not a bug. and removed User Support A user needs help with something, probably not a bug. labels Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PatchModelAddDownscale is not compatible with controlnet
3 participants