When you first come into contact with stable diffusion, you will hear a lot of professional terms, includingLoRAThe model will definitely be mentioned, so what is the LoRA model? What does it do?
The full name of the LoRA model is: Low-Rank Adaptation of Large Language Models. It can be understood as a plug-in in Stable-Diffusion. It is a model that can be trained with only a small amount of data. When generating images, the LoRA model will be used in conjunction with the large model to adjust the output image results.
Let's take an example that is easier to understand: the big model is like a person without makeup, and the LoRA model is like a person who has undergone makeup, plastic surgery or cosplay, but the inner part is still the foundation of the big model. Of course, the LoRA model is not limited to characters, scenes, animations, and styles all have corresponding LoRAs.
Let's take a look at the effect of the LoRA model with actual examples. The large model and LoRA model used this time are as follows. You can visit and download them for your own experience:
Large model: CamelliaMIx_2.5D_V2
LoRA model: Yae Miko | Realistic Genshin LORA
LoRA Model: Adventurers
LoRA model: Elegant hanfu ruqun style
1. Large model: CamelliaMIx_2.5D_V2 + LoRA model: Yae Miko | Realistic Genshin LORA comparison
Prompt: (masterpiece:1.2, best quality), (real picture, intricate details), 1girl, solo, upper body, casual, Small eyes, short hair, minimal makeup, natural fabrics, close-up face, smile, home, white cropped shirt
1-1. The effect of using only the large model (see the figure below)
1-2. Effect of using large model + LoRA model (see the figure below)
Summary: When the LoRA model is used and the prompt is not adjusted, the output effect changes significantly, making the final result more inclined to the LoRA style.
2. Large model: CamelliaMIx_2.5D_V2 + LoRA model: Adventurers comparison
There are three groups of comparison items in this graph: 1. Adding LoRA without new prompt words; 2. Adding new prompt words without LoRA; 3. Adding LoRA with new prompt words;
A new concept (Trigger Words) is introduced here. The function of Trigger Words is to activate specific content in the LoRA model and let LoRA know that we want to use this thing. Trigger Words are written into the model when the LoRA model is made.
The Trigger Words for LoRA model Adventurers are as follows:Trigger Words: ARMOR, KNIGHT, ROGUE, PALADIN, WARRIOR, BERSERKER, RANGER, CLERIC, MUSKETEER, VALKYRIE, PIRATE
2-1. Use the large model + LoRA model + prompt word without modification (see the figure below)
2-2. Using large models + prompt words has additional effects (see the figure below)
New prompts: ARMOR, KNIGHT, ROGUE, PALADIN, WARRIOR, BERSERKER, RANGER, CLERIC, MUSKETEER, VALKYRIE, PIRATE
2-3. Using the large model + LoRA model + prompt words has additional effects (see the figure below)
New prompts: ARMOR, KNIGHT, ROGUE, PALADIN, WARRIOR, BERSERKER, RANGER, CLERIC, MUSKETEER, VALKYRIE, PIRATE
Summary: When there is no LoRA model, adding new prompt words can show some of the effects of prompt words, but the texture of armor and other items is poor. After adding the LoRA model, the texture of armor and other items is obviously improved. This is the beautification effect of LoRA.
3. Large model: CamelliaMIx_2.5D_V2 + LoRA model: Elegant hanfu ruqun style comparison
The comparison items of this graph are still: 1. Adding LoRA without new prompt words; 2. No LoRA with new prompt words; 3. Adding LoRA with new prompt words;
The trigger words of the LoRA model Elegant hanfu ruqun style are as follows:ru_qun
3-1. Using the large model + LoRA model + prompt words has no additional effect (see the figure below)
3-2. Use large models + Prompt words have new effects(See the picture below)
New prompt words: (hanfu:0.9), (ru_qun:1.1)
3-3. Using the large model + LoRA model + prompt words has additional effects (see the figure below)
New prompt words: (hanfu:0.9), (ru_qun:1.1)
Summary: As shown in Case 2, the style of the output image cannot be completely changed by the prompt words alone. When combined with the LoRA model, the output image style will be greatly changed.