Add support for Hires. fix values

[paulm]

I don't see if there's a way to define the values for the hires. fix like: Upscaler, Hires steps, Denosing strength.

What values are used today if I wanted to reproduce an exact output from dream-factory in a1111?

[rbbrdckybk]

Dream Factory uses the old/original Auto1111 hires fix method. You'll get the same results in Auto1111 if you enable the hires compatibility option in your Auto1111 settings (image below).

Adding an option for Dream Factory to use the newer hires fix method is on my list!

[paulm]

Good to know! thank you

[Koneko349]

What about the hires steps in Auto1111, how can those be set in dream factory?

I would like to take a 512x768 image apply a second hires pass with steps and scale up by 2x. I can do this with upscale in dream factory but it is only a blown up version of the 512 image where as auto1111 produces a slightly different/clearer image with the box mentioned unchecked.

[rbbrdckybk]

You can do that with a !MODE = process .prompts file currently. Take a look at example-process.prompts in the /prompts directory for an explanation of how they work. Note that you will need to have MAX_OUTPUT_SIZE set in your config.txt file for this to work. See config-default.txt for instructions on how to set it.

Basically, you want to create a .prompts file containing something like this and run it:

[config]

!MODE = process
#!NEXT_PROMPT_FILE = # uncomment and set to another valid .prompts file if you want to run it when this is done

!USE_UPSCALE = yes
!UPSCALE_MODEL = sd
!UPSCALE_SD_STRENGTH = 0.3

!FILENAME = <input-img>

!INPUT_IMAGE = [set your input path here- can be a single image or a folder of images]
!OUTPUT_DIR = [set your output path here]

[prompts]

go

[Koneko349]

hmmm, I must be doing something wrong... the resolution is being scaled up, however auto1111 produces a slightly different image still. Here are some screenshots so you can see what I mean. For hires fix, use width/height sliders is unchecked in auto1111.

Here is the image in auto1111

Here is what dream factory produces with the closest settings I can find. Setting highres_fix to yes creates a totally different image.

in dream factory I get the same image using upscale in the standard prompt or using it in the process prompt.

Here is my prompt file

[config]

!MODE = standard # standard mode; queue all possible combinations of [prompts] below
!DELIM = " " # delimiter to use between prompt sections, default is space
!REPEAT = no # repeat when all work finished (yes/no)?

!WIDTH = 512 # output image width
!HEIGHT = 768 # output image height
!HIGHRES_FIX = yes # fix for images significantly larger than 512x512, if enabled uses !STRENGTH setting
!STEPS = 35 # number of steps, more may improve image but increase generation time
!SAMPLER = DPM++ 2M Karras # sampler to use; press ctrl+h for reference
!SAMPLES = 1 # number of images to generate per prompt
!SCALE = 9 # guidance scale, increase for stricter prompt adherence
!INPUT_IMAGE = # specify an input image to use as a generation starting point
!STRENGTH = 0.5 # strength of input image influence (0-1, with 1 corresponding to least influence)
!CKPT_FILE = all # model to load, press ctrl+h for reference
!SEED = 1178461960
!CLIP_SKIP = 2

!USE_UPSCALE = yes # upscale output images?
!UPSCALE_AMOUNT = 2.0 # upscaling factor
!UPSCALE_SD_STRENGTH = 0.5
!UPSCALE_CODEFORMER_AMOUNT = 0.0 # how visible codeformer enhancement is, 0-1
!UPSCALE_GFPGAN_AMOUNT = 0.0 # how visible gfpgan enhancement is, 0-1
!UPSCALE_KEEP_ORG = no # keep the original non-upscaled image (yes/no)?

!NEG_PROMPT = blurry, out of focus, chromatic abberations, (low quality, worst quality:1.2), bad hands, poorly drawn face, watermark, text, poorly drawn hands, username, lowres, (EasyNegative), (bad_prompt_version2), (negative_hand), (verybadimagenegative_v1.3)

[prompts]

cute adorable girl, (3d disney pixar style:1.2), (arms above head:1.4), age 19, (anna of arendelle:1.1), (adorable female:1.2), green pleated skirt, braided hair, (skirt:1.4), perfect teeth, perfect human face, (perfect hands:1.2), (on a mountain:1.2), (best quality, masterpiece, 4k, sharp focus, 8k, high definition:1.2) lora:lit:0.6 lora:anna_of_arendelle_v22:0.75

[rbbrdckybk]

It looks like you're upscaling at generation time in DF, which will use ESRGAN by default.

DF uses the original hires fix method implemented in Auto1111, which doesn't allow specifying anything other than final output size. The best way to mimic the new Auto1111 hires method in DF currently is to generate small images initially (in your case 512x768). And then use a !MODE = process .prompts file like the example I gave above to upscale them using stable diffusion (which will add detail, unlike ESRGAN and other traditional upscalers).

You can also upscale directly from DF's built-in gallery by clicking the upscale icon (or pressing 'U') on the image you want to upscale. This requires that MAX_OUTPUT_SIZE be set in your config.txt file to work (like the process mode file solution I gave above). Once set, upscaling from the gallery will use the PF_UPSCALE_SD_STRENGTH value in your config.txt as the denoising strength (default is 0.3 which usually yields decent results), and will upscale to the size dictated by the MAX_OUTPUT_SIZE you supplied.

[Koneko349]

That works thank you! I didnt realize the difference between sd and ESRGAN.

Do you have any plans to implement auto1111's new method of upscaling?

[rbbrdckybk]

It's been a minute, but the newer hires fix/upscale method is supported in the latest version:

https://github.com/rbbrdckybk/dream-factory/blob/main/CHANGELOG.md#20230922

[paulm]

Thanks for this update. I'm trying to sort out the best value to insert in the config file. I continue to get this error when requesting an upscale from the viewer Error: SD upscale unable to find appropriate upscale size under MAX_OUTPUT_SIZE

Can you share a recommended example for using the latest/best model, settings, and process to upscaling a folder of images? I know every machine and intent is different, but curious if you have a general recommendation to help me get started.

[rbbrdckybk]

The upscale operation that I built into the gallery probably needs to be re-done based on newer methods. It currently uses SD Upscale, which provides good quality but is limited by the amount of GPU VRAM that you have. With 12GB VRAM, I'm barely able to get 3MP upscaled images, which isn't very large at all. That error indicates that the source image is already as large (or larger) than SD Upscale would be able to give you, based on the MAX_OUTPUT_SIZE that you specified in your config.txt.

Instead, I highly recommend going the Ultimate SD Upscale route for upscaling images in Dream Factory - you'll need to have the extension installed in Auto1111 first.

After that, copy & paste the script below into an empty .prompts file and run it after making a couple minor changes:

• Fill in !INPUT_IMAGE = and !OUTPUT_DIR = with your desired input folder of image(s) to upscale, and the output folder to save them in.
• Set !UPSCALE_AMOUNT = to your desired upscaling factory (2 = 2x the original image size). The advantage of Ultimate SD Upscale is that it breaks the image into smaller pieces to do the upscaling, so there shouldn't be any kind of limitation here due to GPU VRAM. I find values up to 2.5 or so work very well. If I want to go larger than that, I'll typically run multiple passes (e.g. upscale twice at 2x instead of trying to upscale all the way to 4x in one pass).
• !UPSCALE_SD_STRENGTH = controls the denoising strength (0-1); 0.25 is a good starting point. A higher number will create more detail in the final image but may also introduce unwanted artifacts. Values higher than 0.40 or so rarely work well.
• Verify that you have the 4x-UltraSharp upscaler installed to Auto1111. You can also set !UPSCALE_ULT_MODEL = esrgan if you'd rather not install a new upscaler, but 4x-UltraSharp gives noticeably better results.
• I typically use an excellent SD 1.5 model for upscales, even if the original image was created with SDXL. You can force the model that's used with !OVERRIDE_CKPT_FILE = cyberrealistic (this model, in this example). If you don't set this, then Dream Factory will attempt to re-use whatever model was used to create the original image by looking at the image metadata.

[config]

!MODE = process

# uncomment below and put the name of another .prompts file here if you want DF to automatically execute 
# it when the upscale processing is finished
#!NEXT_PROMPT_FILE = 

!USE_UPSCALE = yes
!UPSCALE_MODEL = ultimate
!UPSCALE_ULT_MODEL = 4x-UltraSharp
!UPSCALE_AMOUNT = 2
!UPSCALE_SD_STRENGTH = 0.25
!OVERRIDE_CKPT_FILE = cyberrealistic

!FILENAME = <input-img>-upscaled

!OUTPUT_DIR = C:\put_your_output_folder_here
!INPUT_IMAGE = C:\put_your_input_folder_here


[prompts]

go