merging several LoRA

[Rudy34160]

Hi,

I would like to know if it is possible to merge several LoRAs, some of which are identical but with different weights, in order to create a single one.

I have done many tests on a LoRA that I am trying to refine, and I would like to keep the best of each test.

For example, in my prompt, I would like to transform

"lora:ejlor04_dream-000004:0.3 lora:ejlor04_dream-000008:0.2 lora:ejlor04_dream-000009:0.2 lora:ejlor09_dream-000008:0.2 lora:ejlor06_dream:0.1"

into a single LoRA, like "lora:ejlor_final:1".

I saw the possibility to do it in Kohya GUI, but it only works for 2 LoRA, and I can't specify the weights...

Ideas ? 🤔

[TomLucidor]

Similar to #1751
Mostly read this part and use the Command-Line https://github.com/kohya-ss/sd-scripts/blob/main/docs/train_network_README-ja.md#%E8%A4%87%E6%95%B0%E3%81%AElora%E3%81%AE%E3%83%A2%E3%83%87%E3%83%AB%E3%82%92%E3%83%9E%E3%83%BC%E3%82%B8%E3%81%99%E3%82%8B
Betting that if they have different dimensions use SVD version https://github.com/kohya-ss/sd-scripts/blob/main/docs/train_network_README-ja.md#%E8%A4%87%E6%95%B0%E3%81%AErank%E3%81%8C%E7%95%B0%E3%81%AA%E3%82%8Blora%E3%81%AE%E3%83%A2%E3%83%87%E3%83%AB%E3%82%92%E3%83%9E%E3%83%BC%E3%82%B8%E3%81%99%E3%82%8B

Weights wise apply the same logic with character blending (e.g. 0.6+0.4)