img2img upscale by tiles

[playlogitech]

is it possible in comfy? Every pro workflow rn is about details boost trough tile upscale

[WASasquatch]

Wouldn't you just upscale anyway and use tiled VAE?

[throwaway-mezzo-mix]

I think he probably means the thing you can do in automatic1111 where the actual generation is done in tiles. (i know the built-in img2img SD upscale script does this, probably some more things too)
tiled vae is great but it doesn't help much if keeping the desired latent resolution in memory is too much and causes an OOM.

[playlogitech]

https://2ch.hk/ai/src/191780/16801925155242.png

current tile upscale looks like a small factory in factorio game and this is just for 4 tiles, you can only imagine how it gonna look like with more tiles, possible but makes no sense

[WASasquatch]

Ah, you mean the GO BIG method I added to Easy Diffusion from ProgRockDiffusion. That would indeed be handy. I did find the results were hard to control, though I never tried A1111 version. It probably uses img2img color fix so they don't vary.

[GrimblyGorn]

Ya this is something I would like as well. I've tried a few similar setups to the one depicted and they seem very daunting to contemplate expanding. More so when considering how large I would need them to expand to. It does seem like something that could be built into 1 or 2 nodes that could work together and achieve the same results.

Along the same lines, an infinity outpaint node would be nice. Something that adds a pad buffer and grows the images but only processes those new chunks and does so on each individually, then it loops and grows the image again. Then you can give it the inputs and set the growth rate (padding size), set the desired final size for the image, and it can just sit there looping and expanding the image. Having it only process the new chunks individually should allow for growing these images as large as you want with no vram issues.

[WASasquatch]

Not sure an infinite canvas would be possible. The system is linear to a end point. So you'd have to manually set all that up to a final result, every run. Otherwise I'd image you'd need a canvas node, and then a layer node that layers the new result on the open canvas, meaning you'd have to still manually set up the location the result is applied to. And there is no backup, it's linear in one direction, so you'd need a node to slice up the canvas and feed into img2img to generate a new tile to then again, layer on to the canvas, but not sure if that is even possible without recursive issues

[playlogitech]

well i really tried, but my head is about to explode already, because on last stage i get high quality drop, probably due to attempt of too low resolution upscale, i simply cant go higher on my 4090, because there is always OOM error, comfy do not release the memory from previously made steps. Maybe you guys can have more success than me.

Here is my workflow:

https://drive.google.com/file/d/1fv77SX03mzUV1iID2xDjwHrHZ4IF4joz/view?usp=sharing

by any chance can you dig into it? @comfyanonymous

[WASasquatch]

Using latent space may cause artifacts though. Latent scaling is very lossy it seems. If the image tensors weren't on GPU this wouldn't be much an issue, as the images would be in RAM not hogging up the VRAM.

[Ferniclestix]

the trick is to add a sampler after a latent upscale, it is a lot more like trying to control an exited ferret than standard upscaling but you can get very high res images this way on lower spec hardware.

[WASasquatch]

the trick is to add a sampler after a latent upscale, it is a lot more like trying to control an exited ferret than standard upscaling but you can get very high res images this way on lower spec hardware.

Yeah the VAE encoding process is a beast. Seems very wasteful on resources.

[BlenderNeko]

for those that missed it. tiled Ksamplers have been around for a little while now.

[Ferniclestix]

Hah, no Im blind, thanks a bunch for making it, ill integrate it into my planz now >:)