Joy CaptionIntroduction
In previous articles have described the current excellentimage inversionPrompt model: Joy Caption.Joy CaptionThe backpropagation cue word produces detailed and unambiguous descriptive information about the image and supports theSFWandNSFWimage descriptions that are getting a lot of attention. This is a program that combines Googlesiglip-so400m-patch14-384models and image-cue oriented fine-tuning of theMeta-Llama-3.1-8B-bnb-4bitmodel composition. Running locally theJoy CaptionThe model takes up about7GB video memory.
Joy Caption Installation Guide
It is first necessary to utilize theComfyUIPlugin Manager SearchComfyui_CXH_joy_captionplugin and click Install this plugin and restart. Then you also need to download the corresponding model, which is also a step in the local complex installation:
- Note that the local runtime environment needs to confirm thattransformers>=4.44.2 Version.Plug-in address:: https://github.com/StartHua/Comfyui_CXH_joy_caption
- Download the modelgoogle/siglip-so400m-patch14-384and place the catalog /ComfyUI/models/clip/siglip-so400m-patch14-384. Here you need to downloadEntire project document, modeled at https://huggingface.co/google/siglip-so400m-patch14-384
- Download the model unsloth/Meta-Llama-3.1-8B-bnb-4bit and place the catalog /ComfyUI/models/LLM. DownloadEntire project document, modeled at https://huggingface.co/unsloth/Meta-Llama-3.1-8B-bnb-4bit
- Download the model fancyfeast/joy-caption-pre-alpha and place the catalog /ComfyUI/models/Joy_caption. Here you need to download it manuallyEntire project document, modeled at https://huggingface.co/spaces/fancyfeast/joy-caption-pre-alpha/tree/main/wpkklhc6
If you don't want to download the file manually, you can try to use git to download it, navigate to the corresponding directory in CMD and then use git to download the directory, e.g.google/siglip-so400m-patch14-384 The model is as follows:
git lfs install
# Replacement of corresponding model git address
git lfs clone https://huggingface.co/google/siglip-so400m-patch14-384
Basic Flux ComfyUI Workflow
About Flux model native ComfyUI workflow experience see the previous article: FLUX [Continued]: 12B parameters 23G largest open source Vincennes diagram model, Dev version straight out of the stunning beauty to appreciate .
The ComfyUI workflow and models mentioned in this article can be downloaded on LIBLIBAI or run online for experience:
- FLUX.1 is available online - Dark Forest Studio: https://www.liblib.art/modelinfo/488cd9d58cd4421b9e8000373d7da123
- Workflow - Flux text | picture + LORA + CN + prompt reverse push one-click switch workflow: https://www.liblib.art/modelinfo/782aacd70f604da39e83368c696a02a8
Joy Caption Backpropagation Workflow
To increaseJoyCaptionBackpropagation prompts workflows by simply adding the Flux base workflow to theJoyCaption model loading and backpropagationTwo nodes.Workflow Download AddressFor: https://www.liblib.art/modelinfo/cc112e6f18bf46049b680ec4b42c511a .
Notice: This paper uses theFlux Detail Texture Enhancement LORA Model,Enhances overall image quality. About the introduction of LORA see the previous article: [ComfyUI] Flux: it's awesome! Detailed texture enhancement, character drop oil light realistic, rich movie light, rich picture elements.
01. Drying of clothes
The woman is facing away from the camera, gazing towards the horizon with a simple, white, short-sleeved T-shirt and light blue denim shorts. She has long, straight black hair tied into a high ponytail, and is wearing a simple, white, short-sleeved T-shirt and light blue denim shorts. The woman is facing away from the camera, gazing towards the horizon with a serene expression. The woman is facing away from the camera, gazing towards the horizon with a serene expression. She holds a wooden clothespin in her right hand, which is holding a white T-shirt on a clothesline strung horizontally across the rooftop. The clothesline is made of thin, yellow string, and the clothespin is positioned near the sleeve of the shirt. In the background, there is a clear blue sky with a few scattered clouds, and the clothespin is gazing towards the horizon with a serene expression. blue sky with a few scattered clouds, and a view of a cityscape featuring multiple high-rise apartment buildings with balconies. The rooftop surface is concrete, with a few small plants adding to the surface. The rooftop surface is concrete, with a few small plants adding some greenery. The overall scene conveys a sense of tranquility and simplicity, with the bright sunlight casting The overall scene conveys a sense of tranquility and simplicity, with the bright sunlight casting soft, natural shadows.
02. Together
octane rendering,UE5,Maya,blender, . This is a digital photograph featuring two fingers, one on top of the other, with the tips touching. Each finger is drawn with black marker to resemble a person. The top finger is a girl, depicted with closed eyes and a small smile, suggesting happiness. She has a pink heart above her head, and her arms are bent at the elbow, with her hands in the air. She has a pink heart above her head, and her arms are bent at the elbow, with her hands clasped together. The bottom finger is a boy, with closed eyes and a small smile, also suggesting happiness. The bottom finger is a boy, with closed eyes and a small smile, also suggesting happiness. The background is a soft, pale yellow color, providing a neutral and soothing backdrop that enhances the warm, affectionate theme of the image. The background is a soft, pale yellow color, providing a neutral and soothing backdrop that enhances the warm, affectionate theme of the image. Text is written in black, playful, hand-like font, with the words "Together Forever" above the girl's finger, and "I love you..." below the boy's finger. The text is surrounded by small pink hearts, adding a whimsical touch. The overall mood is one of love and affection, conveyed through the simple yet charming depiction of love and affection. The overall mood is one of love and affection, conveyed through the simple yet charming depiction of the fingers and the accompanying text.
03. Selling piglets
octane rendering,UE5,Maya,blender, Slung over his shoulder was a stick with two caged piglets on either side, of a photorealistic CGI (computer- generated imagery) artwork. This digital artwork depicts a chubby, adorable baby with dark hair and large, round eyes, dressed in a sleeveless, light, pink dress with a subtle polka dot. This digital artwork depicts a chubby, adorable baby with dark hair and large, round eyes, dressed in a sleeveless, light pink dress with a subtle polka dot pattern. The baby is holding two woven baskets on either side of its body, balanced on its shoulders. The baby's expression is one of contentment and innocence. The background features a rural setting with a paved path leading into the distance, a small piglet with pink skin and short snouts. The background features a rural setting with a paved path leading into the distance, flanked by lush green foliage and a wooden building on the left. The lighting is soft and natural, creating a serene atmosphere. The textures are meticulously detailed, from the smoothness of the baby's skin to the coarse texture of the woven baskets and the soft fur of the piglets. This CGI artwork combines photorealism with a whimsical, almost surrealistic touch, enhancing the charm and cuteness of the subject.
04. Cucumber costume show
octane rendering,UE5,Maya,blender, . This is a highly detailed, high-resolution photograph featuring a life-sized, stylized human figure crafted entirely from cucumber slices. The figure stands against a plain white background, emphasizing its vivid green hue. stands against a plain white background, emphasizing its vivid green hue. The person, with a serene expression, wears an elegant, sleeveless gown made The person, with a serene expression, wears an elegant, sleeveless gown made from cucumber slices arranged in a layered, petal-like fashion. The gown's neckline is V-shaped, and the slices form a series of overlapping, scalloped edges that resemble a flower's petals. The figure's head is adorned with a crown of cucumber leaves, adding to the botanical theme. The texture of the cucumber slices is smooth and glossy, with the light reflecting off the wet surface, giving it a fresh, vibrant appearance. The overall effect is both surreal and artistic, blending elements of nature and human craftsmanship. The photograph captures the intricate details of the cucumber slices, emphasizing their natural patterns and the delicate nature of the slices. The photograph captures the intricate details of the cucumber slices, emphasizing their natural patterns and the delicate arrangement that creates the gown. The figure's skin is a pale, almost translucent white, contrasting starkly with the green. The figure's skin is a pale, almost translucent white, contrasting starkly with the green of the cucumber slices, enhancing the surreal nature of the image.
