January 4, 2011 - Technology media outlet The Decoder published a blog post yesterday (January 3) reporting thatGoogle DeepMind In conjunction with researchers at Columbia University and the University of California, San Diego, a program called CAT4D AI systems.The ability to transform ordinary video into dynamic 3D scenes lowers the threshold for 3D content creation and opens up new possibilities for multiple industries.
The CAT4D system utilizes a diffusion model to convert a single-view video into a multi-view view and builds it into a dynamic 3D scene, allowing the user to view the subject of the video from different angles as if they were in it. The attached demo is shown below:
Previously, multiple cameras were required to record the same scene at the same time to achieve similar effects, but CAT4D simplifies the process by requiring only common video footage, a technology that is expected to revolutionize game development, filmmaking, and augmented reality, among other fields.
In training the AI, the Google DeepMind team found that there wasn't much existing data, and to solve this problem, the team mixed real-world footage with computer-generated content.The training data consisted of multi-view images of static scenes, single-view videos, and synthesized 4D data, which was learned through a diffusion model that creates an image from a specific angle at a specific moment in time.
The 3D scenes generated by the system at this stage are shorter than the original footage, but the quality of CAT4D's imaging is already superior to comparable systems.CAT4D technology has a wide range of applications. Game developers can use it to create virtual environments, and filmmakers and AR developers can integrate it into their workflow.