A few months ago.AI RestoreHistorical FiguresimageThis project has become an extremely popular and explosive AI side project, which had attracted widespread attention.The development of AI technology has not only demonstrated the excellent ability of AI in the field of image processing, but also opened up a new chapter in the reproduction of history.
In those days, AI finely restored the ancient people's appearance through original black and white pictures, or hand-drawn pictures, or mutilated picture information. In this process, it is both a challenge to AI technology and a tribute to history and culture.
AI technology discovery to date, this project is an even more important step - it not only restores the images, but also makes theseHistorical Figures "Speak". It became version 2.0 of the program.
With the power of natural language processing technology, AI is able to simulate the language style and thinking logic of historical figures, allowing them to recite their own poems and express their insights on the current situation, as if coming through time and space to engage in a conversation with us that spans thousands of years.
This experience, undoubtedly, brings new possibilities and far-reaching effects to the fields of historical research, cultural heritage, and edutainment.
Next, I'll give you an example of how AI can restore images of historical figures and make them say their own famous quotes or poems and songs.
Let's take Cai Wenji, one of the Four Great Beauties, as an example and have her recite the words she wrote, "Poem of Sorrow and Anger".
Cai Wenji (蔡文姬) was a talented woman in the late Eastern Han Dynasty, whose name was Diem (琰), her character Wenji (some accounts believe her character Zhaoji (昭姬)), and she was a native of Chenliu County (陈留縣圉县), and the daughter of the literary scholar Cai Yong.
Cai Wenji was a learned woman who specialized in literature, music and calligraphy, and was an outstanding female writer of her time. Her life experience was quite rough, she was first married to Wei Zhongdao and returned home after her husband's death.
At the end of the Eastern Han Dynasty, when the Central Plains were in great turmoil and the vassals were in power, the Southern Xiong Nu, which had originally surrendered to the Han Dynasty, took advantage of the opportunity to rebel, and Cai Wen Ji was taken captive by the Xiong Nu's Zuo Xian Wang, and gave birth to two children.
Later, after Cao Cao unified the north, he spent a great deal of money to redeem her and have her married to Dong Si.
The Poems of Sorrow and Anger are two five-character poems and a poem in the form of a sousetta written by Cai Wenji after her return to the Han Dynasty. The pentameter poem focuses on "sentimental disarray" and is the first autobiographical narrative poem written by a literati in the history of Chinese poetry.
It details Cai Wenji's misfortune during the war and her complex feelings after returning to Han, with deep emotions and vivid narrative.
Let's take the following excerpts from the Poem of Sorrow and Anger as an example:
"Han Ji lost his authority, and Dong Zhuo messed up the heavenly norms.
He wanted to usurp and kill, but first he killed all the virtuous men and women.
He was forced to move the old state and embraced the Lord for self-strengthening.
The sea rises up with righteous divisions, wanting to fight together against the ominous.
Zuozhong came to the east, and his golden armor shone in the sunlight.
The people of Pingtu are fragile, and the incoming soldiers are all Huqiang.
The hunting field surrounds the cities and towns, and they are all destroyed.
There is no one left to cut off, and the corpses support each other and reject each other."
I. AI restoration of Cai Wenji images:
1. Use AI tools to make the original image clear and remove unnecessary elements
Baidu search for "Cai Wen Ji historical figures" images, select the image with the front, such as the following picture:
The original image is very blurry and we need to make it clear with AI tools. Open the "Baidu AI Photo Assistant", upload this image and click on the right side of the page "clearer”.
Re-generate and crop the image to 3:4 as follows:
2, AI restore image:
The AI tool used is Comfyui Workflow:"Ancient Portraits Reduced to Real Life Photos Comfyui Workflow", link:
https://www.liblib.art/modelinfo/58268736255d4b749750f83b090ee8f4
After opening that link, click "Run workflows online"Go to the creation page:
In "Load Image"Here, upload the edited image from the previous step:
At this time, if you click "Start Chart Generation" in the upper left corner, the large model and Controlnet model will be the model selected by the workflow author by default. Because it is an XL model, you need to pay to use it.
So we need to change the model:
In "Checkpoint LoaderIn "MAGNIFICENT MODEL", select "MAGNIFICENT MODEL" (other realistic models are also possible):
in the first"Controlnet model"The canny SD15 model was selected in the intensity 0.9:
In the second "Controlnet model" select Tile SD15, strength 0.5:
Because Cai Wenji is a woman, therefore, to be in "ClIP Text Encoder"Replace "man" with "woman" in the default prompt."
When you're done, click on the top left corner "Beginning of graphic production". That workflow is right in the middle of generating the image for us, wait a moment and the generated image is as follows:
II. Generate a video of Cai Wenji reciting the Poem of Sorrow and Anger (some excerpts):
1、Tu Sang Video
OpenDream AI(https://klingai.kuaishou.com/), click "AI Video" to enter the video generation page. Choose "Picture Video", upload the picture generated in the previous step, choose "Moderate" for the motion speed, 12 seconds for the generation duration, and other defaults.
Click "Generate Now". Dream generated a 12-second video for us. Since the verse I chose was longer than 12 seconds, we need to generate two more 12-second videos.
Note: How can I know how long video I need to generate? You can fill in the text in the "Video Generation"-"Matchmaking" of Dream AI, choose the tone, adjust the speed of the words, and then click audition, the system will automatically give you the length of the video.
To better make the transition from video to video natural, you need to use the article I wrote yesterday:Example demonstration of how to create consistent video sequences with consistent characters using AI.
According to this method, generate 3 videos, after combining them together with clipping (note that you don't need to add anything), export the video. Note, that Dream AI generates a video lip-sync once can not be longer than 30 seconds, and the synthesized video has 36 seconds, so we need to crop to 30 seconds.
2. Counterpoint:
Open AI, click "AI Video"-"Video Generation", enter the video generation page, and select "Lip Synthesis". Upload the video synthesized in the previous step at "Role"; paste some excerpts of "Sorrowful Poem" into the text box of "Text Reading"; choose the tone as "Sad Female Voice"; Generate the effect "Standard":
III. Post-editing
We can also import the generated video to edit in Cutscene. AddBackground music, beginning and end, coverAnd so on. I'm not going to do this step, so if you're interested, you can try to do it.