AI open source projects

Diffutoon is an AI framework for converting videos into cartoon-style animations, launched by researchers from Alibaba and East China Normal University. The editable cartoon shading technology based on the diffusion model can convert realistic videos into cartoon-style animations. The technology achieves high resolution and long-term rendering of videos by decomposing it into subtasks such as stylization, consistency enhancement, structure guidance, and coloring. Diffutoon also has a content editing function that can adjust video details based on text prompts, maintaining a high degree of visual effect and consistency when processing videos, and achieving efficient and high-quality processing of video animations…

ProPainter is an advanced video restoration tool that uses AI technology to remove specific objects and watermarks from videos. Through the loop flow completion network and Transformer technology, ProPainter can intelligently detect and remove moving objects in videos, repair damaged areas, and restore the integrity of videos. Whether it is removing watermarks or restoring videos, ProPainter can provide high-quality solutions. ProPainter features Remove moving objects/people: Using advanced E2FGV1 technology, ProPaint…

ChatTTS is a speech generation model designed for conversational scenarios. It supports Chinese and English. After large-scale data training, it can generate high-quality and natural speech synthesis. The product is designed to support applications such as conversational tasks of large language model assistants, generating conversational speech, video introductions, and speech synthesis for education and training content. ChatTTS features multi-language support: supports Chinese and English, suitable for multi-language environments. Large-scale data training: trained with about 100,000 hours of Chinese and English data to ensure high-quality and natural speech synthesis. Conversational task compatibility…

StoryDiffusion is an innovative AI tool developed by the HVision team at Nankai University. Its core function is to generate coherent image and video stories, especially good at comics. The tool uses advanced consistent self-attention technology to generate thematically consistent image sequences without additional training. These images are very suitable for storytelling or as a basis for further content creation. StoryDiffusion is a joint venture between ByteDance and Nankai University…

IDM-VTON is a novel diffusion model for image-based virtual try-on tasks, which generates virtual try-on images with a high degree of realism and detail by combining high-level semantics of visual coders and UNet networks as well as low-level features. The technique enhances the realism of the generated images by providing detailed textual cues and further improves the fidelity and realism in real-world scenarios through customization methods. IDM-VTON is an advanced virtual try-on technique that generates high-quality virtual try-on images by combining a visual coder and a UNet model, and can be customized to...

Rope is a GUI-focused AI face swapping tool that combines insightface's inswapper_128 model to provide a feature-rich GUI. The highlight of this tool is its fast face swapping speed, image upscaling, similarity adjuster, and orientation management. In addition, Rope supports face swapping for images and videos, and has advanced features such as automatic save file name generation, docking/undocking of video players, real-time playback, image setting markers for specific frames, etc. Rope Features AI Face Swapping: Leveraging the most advanced…