OpenAI has delivered an early Christmas present.
officially released its video generation model on December 10, 2024Sora.
This much-anticipated product of the outside world is called Sora Turbo, which supports generating videos from images and text, and adds a variety of advanced features such as style remixing, time extension, storyboarding, etc. The industry commented that this is a phenomenal product after ChatGPT, which will open a new paradigm for AI video creation. The industry evaluates that this is another phenomenal product after ChatGPT, and that AI video creation will open up a new paradigm after a year of continued buzz.
Summarize its innovations:
Sora is currently a web product independent of ChatGPT at sora.com, with basic generation capabilities such as Text to Video/Image to Video.
Inspiration Community. It has an Explore square to showcase other people's work for you to discover inspiration and remix (which is a powerful, powerful second creation feature).
The groundbreaking Storyboard. It's a superbly easy-to-use tool that combines the linguistic power of LLMs with the simplicity of video editing, allowing you to quickly organize your video stories.Sora's input box also focuses on a range of video-generation capabilities such as recut/trim/remix, as well as support for creating workspaces, file clip management, and much more.
Recall that the sora craze started on February 16, 2024, when OpenAI announced the launch of "Sora", a natural language video generation model.
Text-only commands output up to 60 seconds of video with highly detailed backgrounds, complex multi-angle shots, and emotional characters.
OpenAI says that Sora is the foundation for models that can understand and simulate the real world, and that this capability will be an important milestone in the realization of AGI (Artificial Intelligence for General Purpose).
The industry evaluation said "faster than expected", said "we really see the new industrial revolution coming", sora completely broken circle.
How hot is sora? Even though the product hadn't been released yet, there were people selling classes immediately.
Li Yi Zhou: No buddy, you're this...
Subsequently, various domestic and foreign model manufacturers have laid outAI VideoMarket.
Why is everyone so focused on AI video?
The multimodal large model centered on AI video generation is now a key node in the development of the AI industry. Video content integrates multi-dimensional information such as text, speech, and images, and its generation technology is backed by the quantity and quality of video data, the optimization of algorithmic architecture, and the grasp of physical regularity.
And in practical application, AI video can dozens of times reduce production costs and improve efficiency.
In reducing live-action shooting, lowering production costs and improving creative efficiency. For example, in the field of educational courseware, feature films, science fiction movies and so on;
AI content quality is also more stable, can be standardized, some aspects of AI creation level beyond 95% manual, suitable for large-scale enterprise use, to achieve stable and high output;
Additionally AI technology allows for the production of various abstract and unreal video clips, reducing issues such as prop waste.
The AI video market is in a rapid growth phase, with sales and production of video A products showing growth, and increasingly fierce competition as domestic and international tech giants and startups lay out their plans.
How do we master AI video?
For the average person, the key to getting started with AI video is understanding the tool itself.
AI video creation is centered on three things:
Scriptwriting: i.e. AI raw text. Includes plot design, hooks, dialog, etc.
Split Shot: i.e. AI drawing. Translates the script into visual elements, involving cue words, and the use of drawing tools.
Video generation: that is, AI video content generation. Includes graph-generated video, voice, soundtrack, etc.
AI Scriptwriting
What the content of the script actually elicits is emotional resonance.
AI scriptwriting has the following uses:
1️⃣ Inspiration Triggers
Creative sparks often flash by, but with AI Script Master, capturing inspiration is a breeze. Whether it's plot ideas or episode planning, AI can provide you with unlimited inspiration.
2️⃣ Characterization
Every story needs vivid characters.AI Script Master can quickly build character biographies based on your ideas and bring each character to life.
3️⃣ Plot Building
How to build the skeleton of a story?AI Script Guru helps you to sort out the outline of the plot, from the ups and downs to the climax, to build the framework of a compelling story.
4️⃣ Script Filler
When the characters and outline are ready, AI Script Master will help you fill in the body of the script, whether it's dialog or scene descriptions, which can be generated with a single click, so that your creation will be smooth and free.
⑤ Script revision/correction
Use AI to revise/polish the script with a more logical plot, more natural and fluent language, or to add drama, conflict, point of interest settings, etc.
The amount of pre-training data for the big models is data rich, far more than any one person's life experience, and requires only the right guidance to produce any content you want.
So how do we write great content with AI?
How to pick an authoring tool (model)? : There is a huge difference in capabilities between models, which greatly affects the quality of the content you create.
Writing style and language ability (Ai flavor): If you do not teach the AI, directly produce a piece of content, full of "AI flavor", you can not use directly, you need to be taught to overcome as far as possible, and lead to the content style you need.
Knowledge base (cases): for example, for short drama creation, upload the required knowledge files of script outline examples, hook setup, plot setup, character customization, subplot, plot, and dialogue, so that the intelligentsia can learn the script outline plot and pacing characteristics. Story type, male and female identities, number of episodes (1 for short films, 70 to 80 episodes for short drama scripts in general), duration (1 to 2 minutes per episode), plot tempo (10 30 50 episodes), story tempo (the more detailed the better the result, see templates), and proofing for the intelligent body.
Getting rid of mediocrity (inspiration): without special constraints, model-generated content tends to lean towards the greatest common denominator of human preferences, which means that the generated content is particularly mediocre and uniform, so the core is constraints:by controlling the information given to the AI, it is possible to set aside more possibilities.
Character customization: using AI screenwriting assistant to create character mini-biography, we can set character information in character card:character's name, character traits, life experience and other information to make the character more three-dimensional and specific details.
The Hero's Journey (story template): In fact, Hollywood and other masterpieces are almost always a formula, the most classic writing paradigm is the hero's journey, Joseph Campbell found that the world's myths follow a specific stage and pattern, which is the "hero's journey" theory. However, stories written in a fixed structure are often too restricted by templates or frameworks, so they require more subtle design in practice.
Short play writing (terps): there are a lot of very specific know how in short plays writing, such as jamming, pacing; there are tons of very specific internet terps in short plays, such as marriage before love, wife chasing crematorium; short play scripts require a fairly specific format.
With a lot of practice, we got a script and finally got to the picture part.
split-screen lens
Subplotting is the first step in translating the script into visual elements, encompassing the visual composition of each scene, including camera angles, character positions, and movement. At the heart of this is AI Raw Graphics.
[Recommendation tool]
Midjourney:https://www.1ai.net/10463.html
StableDiffusion:https://www.1ai.net/10461.html
I.e., dreams:https://www.1ai.net/10005.html (That is, Dream has free points that can be used every day, and there are more features and points if you open a VIP)
Build a comfyui locally, free with the flux model, and the effect is close to Midjourney
[Workflow]
1. material to learn: in Midjourney and other sites user community search for relevant keywords, drawing on the raw ingredients into the mirror image. midjourney search related images need to pay, the principle is the same.
2. Split-screen assistant: Generate pictures in Midjourney after using AI to generate picture cue words.
3. Figure to figure turn drawing: in StableDiffusion, with controlnet control the line or edge, and then use the animation model to generate animation pictures according to the line can be, such as an aerial map of the city, the cultural and tourism promotion of the film inside the ai I China's effect.
[Image Processing]
Post-processing front, deal with the picture and then generate video, figure born video, the better the quality of the picture, the better the effect of the generated video. Better results, lower cost and easier to solve.
Video generation and processing
Video generation is the process of transforming scripts and subplots into actual visual content and is the final step in AI video creation.
The process varies from tool to tool, but most of them require you to know some basic movie and TV vocabulary, which you can get in this lexicon out there.
Ai Video Cue Word Book｜Applicable to Sora, Pika, MJ
https://kh18s6uvdi.feishu.cn/wiki/PP0XwDxoZijp8gkB16DcDVCNnb3
In order to make the picture more controllable, so we directly choose the graphic raw video, although I do not have access to sora now, but most of the tools are similar in operation, the following provides some keywords:
i.e. Dream Tucson video keywords: subject + description + movement
Clear Shadow Tusheng video keywords: subject + description + movement + lens
LUMA graphic born video keywords: subject + description + movement + lens
Of course there are very many tips, implementation steps, pointers, styles, etc. about AI videos that you can get in this continuously updated list:
AI Video Tips|Adapt Sora, Pika, runway, i.e. Dreams
wrap-up
This is a brief primer on this sora. Of course there's a lot to talk about regarding AI videos, so if you like it, we'll continue to update with AIGC related instructional dry runs.