In this section we will dive into ControlNet's depth map constraints and normal map constraints.
In this section, you will learn the basic concepts, roles, processors and their applications of depth and normal maps.
I. Depth map constraints: exploring depth information in 3D scenes
1. What is a depth chart?
A depth map is a two-dimensional image representing the depth information of a three-dimensional scene, where the value of each pixel represents the distance between a point in the corresponding scene and the camera. In theStable DiffusionIn this, white represents the near distance and black represents the far distance.
II. Role of the depth chart
Depth maps are mainly used to specify the front and back relationships and outlines of elements. With a depth map, we can tell Stable Diffusion which parts are foreground and which are background, and their relative distances.
1、Application cases of depth map
We show a case study on how to use depth maps to generate images of characters with specific actions. By adjusting the parameters and selecting appropriate models, we were able to generate character actions and backgrounds that match the depth map.
2. Differences in preprocessors
Depth maps have four preprocessors, which extract depth information to different degrees.
depth_midas: extracting depth information with minimal effort, can simply realize the separation of front and rear view, not reflecting the rear view outline information
depth_zoe:
depth_leres:
depth_leres++:
The MIDS preprocessor extracts the least effort, while the LERES depth map estimation provides more detailed depth information.
Some of the preprocessors also allow the adjustment of parameters to freely select the desired depth range of the depth map.
III. Normal map constraints: modeling surface details
1. What is a normalized chart?
A normal map is a map that preserves information about the bumpiness of an object's surface and is used to simulate surface details and enhance 3D graphics. It simulates complex surfaces through special texture mapping techniques to make the model look more realistic and detailed.
2. The role of normal maps
The normal map retains the bump information on the surface of the object and generates a feature map, which simulates the bump details of the surface using three spatial directions, namely, horizontal, vertical, and depth, represented by the red, green, and blue colors, respectively.
3. Examples of the application of normal charts
We show a case study on how to use normal maps to generate an image with a specific bumpiness. Normal maps play a huge role in the generation process, especially in images that require a high degree of surface bumpiness.
IV. Conclusion
In this lesson, we have learned the basic concepts and applications of depth map constraints and normal map constraints in ControlNet. This knowledge will help us to control depth and detail more precisely in image generation.
I hope this article helps you better understand ControlNet's depth map constraints and normal map constraints. If you have any questions or would like to discuss this further, feel free to leave them in the comments section. We'll see you in the next course!
