Today, I'm installing what I believe to be one of the strongestAI image restorationModelDiffBIRDiffBIR is released and open-sourced by Shenzhen Institutes of Advanced Technology (SIAT) of Chinese Academy of Sciences (CAS), Shanghai Laboratory of Artificial Intelligence (SLAI), and The Chinese University of Hong Kong (CUHK), which utilizes Generative Diffusion Prior (GDP) to progressively repair blurry images, and its core role is to be able to Solve the problem of image blurring and loss of detail in old images.
Let's take a look at the official demo renderings first.
Very strong restoration of portrait pictures.
DiffBIR Architecture
DiffBIR decomposes the blind image recovery problem into two stages:
Degradation Removal: in the first stage, the model initially corrects the blurred regions and noise by removing content that is not related to the image. This process uses uses the Swin Transformer model to accomplish the basic denoising, and the recovery module removes the degradation and obtains high fidelity recovery results.
INFORMATION REGENERATION: In the second stage, the model generates the lost image content and further refines the details of the image. The research team proposed IRControlNet, which utilizes the generative power of the classical Stable Diffusion model to generate realistic details and employs the classifier gudance technique to perform a balance between realism and diversity.IRControlNet is trained based on specially crafted conditional images without distracting noisy content to achieve stable generation performance. The goal of this phase is to recover the high quality features of the image, making the final output clearer and more realistic.
Comparison with mainstream image restoration models
Numerous experiments show that DiffBIR outperforms state-of-the-art methods in blind image super-resolution, blind face repair, and blind image denoising tasks. The official results are shown in comparison with StableSR, PASD, RealESR-GAN, SwinIR, CodeFormer and other models.
Currently there are many image restoration models, the official list of comparison models are only a small part of them, and still the weakest several. B station has a called "deepfree3" upload owner, compared the current mainstream image restoration models, including DiffBIR, Flux Upascler, SUPIR, CCSR, TTPlanet v2, PMRF, InstantIR, StableSR eight model effect comparison video. Those who are interested can check it out.
From my experience, DiffBIR is better at restoring blurred portrait photos, especially blind face restoration, the main feature is the high fidelity restoration of the original image, after restoration, it can keep the same information with the original image very well, unlike some image restoration models, after the portrait restoration, it is not quite the same. Maybe other image restoration models are better than DiffBIR in high-definition detail restoration, but I think DiffBIR is the strongest in face fidelity restoration.
Usage and Deployment
The official thoughtful in replicate on the open use of the page, do not have to deploy directly free to use.
ComfyUI also has a plugin for DiffBIR that can be used in ComfyUI. It's just that ComfyUI-DiffBIR is using the DiffBIR v2 model, and the v2.1 model, which was officially released a while ago, is not supported.
After looking around on the Internet, there is no god to do the latest DiffBIR v2.1 integration package, only do it yourself. Because I am also a programmer, bumping around for a few days to do a good job of this Windows one-click integration package. Here is the use of one-click integration package, mainly part of the parameter settings.
After downloading and unzipping, click the "start.bat" file, after the startup is completed, open the browser and type "http://127.0.0.1:7860/" to open the interface. But the official introduction said that LLaVA is integrated to generate image prompt words, although I downloaded the LLaVA model, but I can't try it successfully, so I can't use this function.
Task in the parameter is mainly to choose sr blind map repair or face face repair.Upsample factor to choose the magnification times. Tiled options can be checked Tiled diffusion, tile size = 512. prompt word can be filled in or not, not much impact.
In addition to Steps generation step parameter can be adjusted, the other can be the default, if you want to generate more detailed, the number of steps can be adjusted a little larger, of course, the generation time spent more time.
Take a look at my photo restoration effect. I couldn't see it clearly in the big picture, I cropped it to see the details of the face, and the restored face kept a high degree of consistency, and the zoomed in skin texture is also very good, unlike the PMRF restored model, which has a good face consistency as well, but the restored skin texture is not at all visible when zoomed in.
In addition to real photographs, DiffBIR does a good job of restoring antique paintings.
Officially, DiffBIR v2.1 can run on graphics cards with 8GB of VRAM. Of course, this is theoretical, if you repair larger images, or magnification is too high, it will still cost more video memory. Finally, the biggest problem of DiffBIR is its slowness. Among all the image restoration models, DiffBIR's slowness can be ranked in the top three.
Well, today is introduced here, pay attention to my private letter "DiffBIR" to get a key integration package quark network disk download link, do not ask me why there is no Baidu network disk, I hate Baidu network disk, locked me uploaded and saved more than 10G e-books, the loss of my charge for many years of VIP. look at my hard work a few days to do integration package The first thing you need to do is to get your hands on a new one, and you'll be able to do it on your own.
Project page:
github.com/XPixelGroup/DiffBIR
Free use of the web page:
replicate.com/zsxkib/diffbir
Comfyui plugin for DiffBIR v2:
github.com/jtscmw01/ComfyUI-DiffBIR
One-click integration package Quark.com:
https://pan.quark.cn/s/1e90834438e3
