CatVTONIt is a simple and efficient virtual try-on diffusion model open sourced by Sun Yat-sen University. The effect is quite good. Today I will teach you how to deploy it locally and build a workflow.
Local deployment
First in ComfyUI Search in the manager: ComfyUI_CatVTON_Wrapper, then install and restart
Or if the network is not good, install it manually
Project address: https://github.com/chflame163/ComfyUI_CatVTON_Wrapper
Download the model
After installing the node, we need to install the model. If you have a good network connection, it is recommended to install it using Google Cloud Drive.
Address: https://drive.google.com/drive/folders/1TJNNql7UfDPVgHJuItDDjowycN5jpC5o?usp=sharing
If you have a Baidu member, you can use Baidu Netdisk to install it.
Address: https://pan.baidu.com/s/1xbL5evB6iiav3QozsL0fJQ?pwd=06nv
We download all the compressed packages and unzip them to a directory.
ComtyUI/models/CatVTON
CatVTON
You need to create this folder yourself.
Building a workflow
The core node is the CatVTON_Wrapper node we just installed, which requires a model image, clothing reference image and mask. I add a mask node SegmentAnythingUltra that can automatically identify clothing. You can also replace it with other mask nodes that can identify objects, or apply the mask manually.
Automatic recognition:
Manual application:
Node parameter introduction
- Image: Input image.
- mask: input the mask of the image. The clothes within the mask range will be redrawn.
- refer_image: The reference image for redrawing the clothing.
- mask_grow: How much the mask expands. If the redrawn clothing style is not correct, try adjusting this value.
- hybrid_precision: You can choose fp32, fp16, or bf16, and the model will run under the selected data type.
- seed: The random seed for the sampler.
- steps: The number of steps of the sampler.
- cfg: cfg strength for the diffusion model.
The workflow is still very simple.
Test results
It still fits the figure very well, and the pattern is well restored, although it is still a little distorted.
The pants are also interchangeable.
If you want to change to other types of clothes, it is recommended to apply the mask manually. For example, my model here is wearing short sleeves, but I want to change to long sleeves, so you can apply the mask to the upper body of the character.
Summarize
This dressing tool is simple to install and easy to use. Although it does not reach commercial level, it is still a good choice for ordinary people to use as a cyber fitting room.