FLUX.1-Krea-dev compatibility

[josemerinom]

The FLUX.1-Krea-dev model was recently released.

I tested (comfyui) my LoRA training (FLUX.1-dev) with the FLUX.1-Krea-dev model, and they are compatible.
I tried training using the flux1-krea-dev_fp8_scaled.safetensors model, and the training completed.
But, when generating images, it only showed "noise."

https://huggingface.co/black-forest-labs/FLUX.1-Krea-dev
https://huggingface.co/Comfy-Org/FLUX.1-Krea-dev_ComfyUI

Please consider making sd-scripts compatible with flux-krea :D

[elen07zz]

I have tried flux1 dev 8, Not fp8 scaled and it kinda works, But in comparison with flux dev, Training a character and real people, The likeness is far from good as dev (Using the same parameters and dataset). So I'm not sure if Krea needs more aggressive LR or if they are some parameters missings in the trainer to be able to LoRAs to learn better using Krea model
https://huggingface.co/boricuapab/flux1-krea-dev-fp8/tree/main

[rockerBOO]

Try without the scaled model as similar is for flux1.dev. It should work the same way but you may need to modify some settings that worked with flux1.dev as the weights are different.

[josemerinom]

Thanks, I'll try with fp8 (not scaled) (01-aug)

It works! (01-aug)

[josemerinom]

LoraFlux: was trained using the fp8 model FLUX.1-dev
LoraKrea: was trained using the fp8 model FLUX.1-Krea-dev

[C0nsumption]

Just ran a test training (Lora) on 21 images last night.
Wasn't on the FP8 but still figured I'd hop in on the conversation.

This was BF16 with the full Flux1-K dev weights.
At only 21 images/1680 steps, results are showing (although undertrained ).
So just noting it for sure works.

[sayantan-2]

LoraFlux: was trained using the fp8 model FLUX.1-dev LoraKrea: was trained using the fp8 model FLUX.1-Krea-dev

whats the config needed to train flux1-krea lora ? same as flux1-dev ? because i am not getting good results on training krea lora with same config as dev

[sayantan-2]

Just ran a test training (Lora) on 21 images last night. Wasn't on the FP8 but still figured I'd hop in on the conversation.

This was BF16 with the full Flux1-K dev weights. At only 21 images/1680 steps, results are showing (although undertrained ). So just noting it for sure works.

what was your training config?

[josemerinom]

¿Cuál es la configuración necesaria para entrenar Flux1-Krea Lora? ¿Es la misma que Flux1-Dev? No obtengo buenos resultados al entrenar Krea Lora con la misma configuración que Dev.

I did a few tests, and I was limited because I use Google Colab:

For me:

• It's better to use the Flux Dev base model (like when training SD 1.5 or SDXL).
• I trained using LR 1e-4/adam8bit optimizer/dim 8 alpha 8, and around 3000 steps, and I got minimally decent results.
• Then, I used the "came" optimizer suggested in comfyui. "CAME" is more aggressive, and with 1000 steps, I got good results.
• This suggests to me that for KREA you need higher LRs, especially if the alpha:dim ratio is less than 1. (Formula: alpha / dim)
• This is why I used alpha=dim (8/8=1)

https://colab.research.google.com/github/josemerinom/test/blob/master/lora_flux.ipynb

PD: I'm trying LR 8e-5 in came, to dampen/lower the aggressiveness in training and (I think I could use dim>alpha and get similar results), get better details (I could also use COSINE or warmup_steps)

[BotPC0]

The idea is to use the original base model, the scaled one does not work with Kohya.
flux1-krea-dev.safetensors

If you have low VRAM let the program do the transformation using --fp8_base as argument.

You can use AdamW8bit or Adam8bit but I normally use Adafactor, if you use Adafactor don't use warmup_steps.

In fact, if you train well, you should get with the same steps similar results at Krea and in Dev. It depends of what you are looking for, Krea has a more natural vibe to it, but with Dev you can get some sharp generations.