Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node for Blending Sigmas #5250

Open
recris opened this issue Oct 15, 2024 · 0 comments
Open

Node for Blending Sigmas #5250

recris opened this issue Oct 15, 2024 · 0 comments
Labels
Feature A new feature to add to ComfyUI.

Comments

@recris
Copy link

recris commented Oct 15, 2024

Feature Idea

Different schedulers have different strengths and weaknesses. In my experiments with photography/realism prompts using Flux.1 Dev I've noticed that beta scheduler has very good prompt adherence but the output tends to be less diverse (certain prompts have a very similar look and feel across different seeds). By contrast ddim_uniform is a bit worse when it comes to adherence but the output looks more varied and creative and photographic images also look more "natural".

The question is: can we combine the best of both schedules? I think having a way to blend the schedule sigmas using linear interpolation can work for this purpose (to some extent).

Existing Solutions

I've hacked together a modification of BasicScheduler node that implements the requested effect:

class BlendedScheduler:
    @classmethod
    def INPUT_TYPES(s):
        return {"required":
                    {"model": ("MODEL",),
                     "scheduler1": (comfy.samplers.SCHEDULER_NAMES, ),
                     "scheduler2": (comfy.samplers.SCHEDULER_NAMES, ),
                     "ratio": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
                     "steps": ("INT", {"default": 20, "min": 1, "max": 10000}),
                     "denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
                      }
               }
    RETURN_TYPES = ("SIGMAS",)
    CATEGORY = "sampling/custom_sampling/schedulers"

    FUNCTION = "get_sigmas"

    def get_sigmas(self, model, scheduler1, scheduler2, ratio, steps, denoise):
        total_steps = steps
        if denoise < 1.0:
            if denoise <= 0.0:
                return (torch.FloatTensor([]),)
            total_steps = int(steps/denoise)

        sigmas1 = comfy.samplers.calculate_sigmas(model.get_model_object("model_sampling"), scheduler1, total_steps).cpu()
        sigmas1 = sigmas1[-(steps + 1):]
        sigmas2 = comfy.samplers.calculate_sigmas(model.get_model_object("model_sampling"), scheduler2, total_steps).cpu()
        sigmas2 = sigmas2[-(steps + 1):]
        
        sigmas = torch.lerp(sigmas1, sigmas2, ratio)
        return (sigmas, )

It probably makes more sense to have something that takes the sigmas as input instead?

Other

No response

@recris recris added the Feature A new feature to add to ComfyUI. label Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature A new feature to add to ComfyUI.
Projects
None yet
Development

No branches or pull requests

1 participant