AI Video ToolsThe track is getting more and more exciting, and the various upgrades are overwhelming, especially Gen-2,Pika1.0,Stable Video Diffusion,andMagic AnimateThey are becoming the most commonly used video generation products for users, and are supported by powerful AI models.
Although the current video model is not as powerful as "generating a movie by describing a story in words", it can already create a vivid video through a series of prompts, and has also developedFigure videoAnd other powerful multimodal capabilities.
Metaverse Daily has tested four major AI video tools, hoping to help you quickly get started. It should be noted that when entering prompt words for all tools,mostUsing English, the generation effect will be better than the Chinese prompt. Here, we also hope that domestic AI video generation tools can catch up quickly and create good products suitable for Chinese users.
Runway Gen-2
Gen-2 isFirstA publicly available text-to-video tool developed by Runway Research. Its related functions include text/image to video conversion, video stylization, image expansion, one-click video background removal, erasing specified video elements, training specific AI models, etc. It can be said that it is currently the most widely used text-to-video tool.StrongestAI video generation/editing tool.
Gen-2's text-to-video function has been greatly improved compared to Gen-1. Here you can see the generated effect of inputting "raccoon play snow ball fight in sunny snow Christmas playground". It can be said that Gen-2 is excellent in both image quality and composition. However, there will be problems with missing keywords, such as the effects of "Christmas" and "snowball fight" are not reflected in the picture.
Just a few days ago, Runway launched a new feature called "Motion Brush". We can turn a static image into dynamic content by simply painting an area on the image. The "Motion Brush" feature is very user-friendly. Select an image, use the brush to paint the area you want to animate, and then adjust the approximate direction of movement. The static image can be animated as planned.
Let's take a look at the effect:
However, the "motion brush" function has some shortcomings. For example, it is only suitable for slow-moving images, and cannot generate fast-moving images such as vehicles driving at high speed. In addition, if the motion brush is used, the area outside the application remains almost still, and it is impossible to fine-tune the movement trajectory of multiple objects.
Currently, Runway free accounts can only generate 4-second videos, consuming 5 credits per second, and can generate up to 31 videos, and the watermark cannot be removed. If you want higher resolution, no watermark, and longer videos, you will need to pay to upgrade your account.
In addition, if you want to learn AI videos, you can try Runway TV, a TV channel launched by Runway, which plays videos produced by AI in a loop 24 hours a day. Through these AI videos, you may also be able to find some creative inspiration.
Website positioning:
https://app.runwayml.com/video-tools/teams/wuxiaohui557/ai-tools/gen-2
Pika1.0
Pika1.0 is released by Pika LabsFirstThe first official product of this lab is an AI technology startup founded by a Chinese team. Pika1.0 can not only generate 3D animations, anime, cartoons and movies, but also can achieve style conversion, curtain extension, video editing and other heavyweight capabilities. Pika1.0 is very good at making anime-style pictures and can generate short videos with movie effects.
The most popular gadget in Pika 1.0 is the "AI Magic Wand", or partial modification function. A few months ago, this was a capability that AI painting had only just acquired. Now, "partial modification" can modify all background and subject local features in the video, and it is also very convenient to implement, with only three steps: upload the dynamic video; select the area to be modified in the Pika console; enter the prompt word to tell Pika what you want to replace it with.
In addition to the "local modification" function, Pika 1.0 brings the "image expansion" function of the text image tool Midjourney to the video industry, which is a new feature of the video AI generation tool.firstUnlike the "AI image expansion" that was ruined on TikTok, Pika1.0's video expansion is quite reliable. Not only are the pictures natural, but they are also very logical.
Currently, Pika1.0 supports free trial for users, but users need to apply for a trial quota. If you are still waiting in line, you can log in to Discord on the official website. Similar to Midjourney, users need to create in the cloud in Discord, and can experience the two major functions of text-video and picture-video.
After entering the Pika1.0 Discord server, click any channel in Generat, enter "/", select "Create", and enter the prompt word in the pop-up prompt text box.
Compared with Gen-2, Pika1.0 understands the prompt words better, but the picture quality is not as good as Gen-2. This is probably because of the cloud creation. Let's take a look at the effect:
If you want to generate a video using a picture, enter "/", select "animate", upload a picture, and enter the prompt word description.
The image-video effect of Pika1.0 is comparable to that of Gen-2. See the effect below:
Website positioning:
https://pika.art/waitlist
Stable Video Diffusion
On November 22, Stability AI released an open source project for AI-generated videos: Stable Video Diffusion (SVD). According to the official blog of Stability AI, the new SVD supports text-to-video and image-to-video generation, and also supports the transformation of objects from single perspective to multi-perspective, that is, 3D synthesis. The generation effect is no less than Runway Gen2 and Pika1.0.
There are currently two online ways to use it, one is the official trial demo released on replicate, and the other is a newly released online website, both of which are free.
We testedFirstBecause it supports parameter adjustment, the operation is relatively convenient: upload the picture, adjust the frame rate, aspect ratio, overall motion and other parameters. But the only drawback is that the picture generation effect is relatively random, and it needs to be constantly debugged to achieve the desired effect.
See the effect:
Stable Video Diffusion is currently just a basic model and has not yet been productized, but the official revealed that "it is planning to continue expanding and establish an ecosystem similar to Stable Diffusion" and plans to continuously improve the model based on user feedback on security and quality.
Website positioning: demo version and online version
- https://replicate.com/stability-ai/stable-video-diffusion
- https://stable-video-diffusion.com/
Magic Animate
MagicAnimate is a portrait animation generation method based on a diffusion model, which aims to enhance temporal consistency, maintain the authenticity of reference images, and improve animation fidelity. It was jointly launched by the National University of Singapore Show Lab and ByteDance.
In simple terms, given a reference image and a pose sequence (video), it can generate an animated video that follows the pose and maintains the identity of the reference image. The operation is also very simple, and only requires three steps: upload a static person photo; upload the action demo video you want to generate; adjust the parameters.
MagicAnimate also provides a local experience in GitHub. Interested friends can try it!
Website positioning:
https://github.com/magic-research/magic-animate