March 30,FamousTechnology Mediaexclusiveinformation,Microsoft,OpenAIA data center project is being developed, including aStargate”superThe computer will be equipped with millions of AI-specific chips, mainly providing services for OpenAI's research and development and products.
According to a person who has spoken with OpenAI co-founder and CEO Sam Altman and reviewed Microsoft's cost estimates for the project, the total investment is as high as $100 billion.
As early as May 19, 2020, Microsoft officially announced thatA server with 10,000 GPUs and 285,000 CPUs was built specifically for OpenAI in the Azure cloud service.superThis computer was one of the five fastest computers in the world at the time., which made great contributions to the later development of products such as ChatGPT and DALL·E3.
This Stargate will be more powerful and faster than the one in 2020, and will help OpenAI develop a large model to achieve AGI.
A brief introduction to Stargate
According to the information,The data center project is divided into five phases, and Microsoft and OpenAI are currently in the middle of the third phase.Phase 4 is that Microsoft will build a satellite that is slightly smaller than Stargate.superThe computer is expected to be put into use around 2026.
Phase 5 is StargatesuperComputers will be equipped with millions of AI-specific chips and are expected to be put into use around 2028. One of the main tasks of the fourth and fifth stages is to purchase a large number of AI chips.
The project will cover an area of more than 400 acres and require about 5 gigawatts of electricity, which will be a power-consuming beast, so nuclear power may be used.
The investment in this project is currently the majorityadvancedThe main reason why Microsoft spends so much money is that OpenAI has been troubled by computing power and cannot unleash its full product and innovation capabilities.
For example, in 2023, OpenAI was originally going to launch an AI product called "Arrakis", but gave up due to limitations in AI computing power.
Microsoft once built a dedicatedsupercomputer
Spend $100 billion to build asuperThe data center is not operated by Microsoft alone, but it has been successfully done once before.
As early as May 2020, at the "Build" developer conference, Microsoft said it had built a server equipped with 10,000 GPUs and 285,000 CPUs specifically for OpenAI in its Azure cloud service.superComputers, each GPU server has a network connection speed of 400GB per second, dedicated to training AI models.
It can be said that without the help of this supercomputer, there might not have been ChatGPT, which later became popular all over the world.
Microsoft's investment this time is 100 times that of 2020. This is mainly because OpenAI's technology is iterating rapidly and has developed epoch-making cultural video models such as Sora.
Compared with ChatGPT text, the pre-training and fine-tuning of the video model requires higher computing power and processing difficulty because video data is high-dimensional data containing width, height, time, and color channels.
Therefore, more complex data preprocessing steps are required, including video decoding, frame extraction, frame resampling, and resizing. Text data is relatively low-dimensional, and mainly processes sequence data, including word segmentation, tokenization, word embedding, and other processes.
In terms of computing power, due to the high-dimensional nature of video data, the AI computing power of video models is much higher than that of text models during training and reasoning. Video processing usually needs to capture temporal changes while maintaining spatial details, which requires a lot of AI computing power support.
Mira Murati, CTO of OpenAI, once saidIt takes about several minutes for Sora to generate a 20-second 720P video.
If 100,000 people use Sora to generate different types of videos together, the AI computing power consumed will be astronomical.
Therefore, AI computing power, like data, has become the water, electricity, and coal infrastructure in the field of generative AI. Any product's technological innovation and iteration cannot be separated from it.