Latent Upscale in Automatic1111 vs Other Latent Upscale vs LDSR

[Lute-4]

Hi, I am a noob.

What is the difference between latent upscale in automatic1111, this latent upscale spoken about here and LDSR?
Does LDSR upscale the image in latent space or the image in pixels?

Thank you.

[Lute-4]

What is this one?
https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler

And what is doing img2img at high resolution? Is the latent space and the noise bigger? Does it divide the image into tiles?

Thank you.

[Alcyon6]

I don't know what's happening under the hood but LDSR upscaling takes a huge amount of compute and gives me really nice upscales. It can take 10+ minutes on a 3090 to do a single image but it very closely follows the original image while "reimagining" it to greater detail. The SD upscale on the other hand often hallucinates new stuff into the picture like duplicate faces and I find it entirely unreliable.

[skein12]

latent (LDSR) upscaling is incredible it's my favorite but it can take some tweaking with noise and it's inconsistent.
there are two types of upscalers from my understanding.
passive ones that are using neural networks like SWINIR or ESRGAN to sharpen, add details and smooth out rough parts.
then there are diffusion based ones that take the prompt and render new details by using img2img with a lower noise strength that allows in the original image, like latent or the SD upscale img2img script (which uses tiling)

i find using a mix of upscalers the best method depending on the image i'm upscaling. there really is no magic setting. I find latent at 2.2x with 35 hires steps and 0.68 noise a great starting place but I often find myself making adjustments. sometimes i use it at 1.5x and then run it in LDSR or ESRGAN to get it the final size i want, other times it comes out great from latent upscaling.

[rafstahelin]

Hey i am getting a weird seam problem here. how do you fix it?

![image](https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/64715158/2cafc796-9203-4c57-8991-62e

1163ccd63)

[octopushugger]

Stop using scripts/extensions that tile the processing or use lower denoise strength. Ultimate SD upscale has some seam fixes you can try but none of them are great.

[Lute-4]

Basic latent upscale is very blurry as shown here, so the latent space has little detail:
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#upscalers

Img2Img using blurry image, little detail latent space, doesn't seem very useful whether high denoising or low denoising is used.

Is it a really bad option or is there something i missed for now?
If it is bad why does a lot of people use this low quality technique in hires fix and img2img (with just resize (latent upscale) selected)?
Using a good GAN upscaler seems way better.

Thank you for your help.

[alphaomegarandomname]

I have no idea what you are saying. People use stuff for reasons, from not knowing better to not caring to those things working out for them

How would anyone know why anyone uses anything? Ask THEM.

[Lute-4]

I have no idea what you are saying. People use stuff for reasons, from not knowing better to not caring to those things working out for them

How would anyone know why anyone uses anything? Ask THEM.

That's what I am doing right now. x)

[octopushugger]

I'm also wondering the differences and I'm trying to figure out how LDSR works. It has its own model and runs 100 steps of DDIM so it must be very similar to the way img2img latent upscale works. What denoise level does LDSR use? Why can't I use the LDSR model as a normal SD model? Couldn't LDSR be potentially even better if you could input a prompt to describe what is in the image?