Recently, we have received a lot of questions from our users aboutAI Photoaspect of the message, I hope to write some articles about AI portraits. Individuals concerned about the AI photo for a period of time, the Internet has also seen a lot of articles written tutorials and cases, as an avid fan of AI painting, naturally, can not help but go to the operation of the practice of AI photo, I will write a series of articles as a topic later on, the actual operation of their own in the process of the bits and pieces of the record, I hope that can help you all.
The so-called AI portrait, simply put, is the use of AI tools to generate portraits of people, here we mainly use the AI tools areStable DiffusionAI RealizationArt PhotoThere are many ways to play, today we introduce AI for artistic portraits of one of the simplest way to realize: plug-in face-swapping method.
Plug-in face replacement method is to use plug-in tools to recognize the features of the characters' five senses, especially the facial features, and then generate a new photo, the new photo and the original picture of the face as much as possible to maintain the same.
The plugin used here isInstantID. For the basics of InstantID and how to install it, you can refer to the previous article [Stable Diffusion [ControlNet]:Achieving Persona Consistency with the InstantID Plugin]
I. InstantID realizes the production method of plug-in face swap
[Step 1]: Selection of large model
The big model supported by InstantID plugin is SDXL big model, so we should not choose the big model based on SD 1.5 training. Here we use LEOSAM HelloWorld|SDXL realism big model, the latest version is V5.0.
Here we choose "None" for our VAE model.
【Step 2】：Writing prompt words
Positive prompt words:
Prompt: high quality,masterpiece,rich details,realistic photography,8k,high-definition image quality.
1 girl,solo,brown eyes,brown hair,long hair,dress,earrings,white dress,fields,fields of flowers,looking at the audience.
outdoors, pink flowers.
Cue words: high quality, masterpiece, rich detail, photorealistic, 8k, HD quality.
A girl, solo, brown eyes, brown hair, long hair, dress, earrings, white dress.
Fields, flowers, fields of flowers, looking at the audience, outdoors, pink flowers.
Reverse prompt words:
easynegative,ng_deepnegative_v1_75t,(worst quality:2),(low quality:2),(normal quality:2),lowres,bad anatomy,bad hands,normal quality,((monochrome)),((grayscale)),(( watermark)),
Related parameter settings of Wensheng diagram
- Sampler: DPM++2M Karras
- Sampling iteration number: 30
- Image width and height: It is recommended to set it to around 1024.
- Cue Factor for Guidance (CFG): between 3 and 5
【Step 3】ControlNet plug-in InstantID model setting
Two ControlNet units are required here.
ControlNet Unit 0: Upload original image of a real person
Here we use photos of Fairy Sister to generate AI artistic photos.
Note that the image here should be around 1024 in size.
The relevant parameter settings are as follows:
- Control Type: Select "Instant_ID"
- Preprocessor: instant_id_face_embedding
- Model: ip-adpter_instanid_sdxl
- Control weight: 1
ControlNet unit 1: Upload face positioning photo
Upload any photo, its role is to control the final image of the character's face position. We can do the same as the first picture.
The relevant parameter settings are as follows:
- Control Type: Select "Instant_ID"
- Preprocessor: instant_id_face_keypoints
- Model: control_instant_id_sdxl
- Control weight: around 0.5 (0.45-0.5)
【Step 4】Image generation
Click the [Generate] button and let’s take a look at the final generated image effect.
II. Changing the background of a character's clothing
We can change the character's clothes, background and other elements in the picture by modifying the positive cue word.
Prompt 1: Christmas Portraits
Prompt:high quality,masterpiece,rich details,realistic photography,8k,high-definition image quality,
1 girl,solo,exquisite hairstyle,brown hair,long hair,Christmas clothing,Christmas dress,bow,Christmas hat,Christmas,festive atmosphere. indoor,upper body,christmas lights,tree,gift box,roses,ribbons,Christmas decorations,lighting,fireplace,warm.
Tips:High quality, masterpiece, rich in detail, photorealistic, 8k, HD quality,...
1 girl, solo, delicate hairstyle, brown hair, long hair, Christmas costume, Christmas dress, bow, Christmas hat, Christmas, festive atmosphere, indoor, upper body, Christmas lights, Christmas tree, gift box, roses, ribbons, Christmas decorations, lights, fireplace, heater,
Tip #2: Wedding Dresses
Prompt:high quality,masterpiece,rich details,realistic photography,8k,high-definition image quality,
1 girl,solo,exquisite hairstyle,crown,headscarf,hair bun.
White wedding dress,long dress,jewelry,looking at the audience,
Tips:High quality, masterpiece, rich in detail, photorealistic, 8k, HD quality,...
1 girl, solo, elaborate hairstyle, crown, headscarf, chignon.
White wedding dress, long skirt, jewelry, looking at the audience.
3. Change the character's posture
We can change the character's action pose with ControlNet's unit 1. Just replace the uploaded character picture of ControlNet's cell 1.
The generated image effect is as follows.
IV. Summary
Plug-in face changing method to achieve AI art portraits, the advantage is that the operation is very simple, the face effect of maintaining consistency is still relatively excellent. But there are also some shortcomings.
(1) The InstantID plugin is only used for SDXL large models, limiting the use of many LORA models because many LORA models are trained on SD 1.5 base models.
(2) the overall lack of realism in the generated picture, especially the color rendering feeling is not very in place, it is not clear whether this is the reason for the big model or the plug-in itself. If you have a better big model or parameter settings can leave a message to communicate with the discussion of learning together oh.
Okay, that’s all for today’s sharing. I hope that what I shared today will be helpful to you.