April 9, U.S. local time.IntelHosted an Intel Vision 2024 industry innovation conference for customers and partners, making several major announcements, including the new Gaudi 3 AI gas pedal, including the new Xeon 6 brand, as well as full-stack solutions covering new open, scalable systems, next-generation products and a range of strategic partnerships.
Data shows thatThe global semiconductor market is projected to reach $1 trillion by 2030, with AI being a major driver, although in 2023, only 10% of companies will succeed in bringing theirAIGCProject Productization.
Intel's latest solution is expected to help enterprises address the challenges they face when promoting AI projects and accelerate the commercialization of AIGC on the ground.
Intel's existing Gaudi 2, born in May 2022 and officially introduced to China in July 2023, boasts extremely high deep learning performance, efficiency, and a great price/performance ratio.
It is manufactured using TSMC's 7nm process, integrating 24 programmable Tenor Tensor Cores (TPCs), 48MB SRAM cache, 21 100,000 Gigabit internal interconnect Ethernet interfaces (ROCEv2 RDMA), 96GB of HBM2E high-bandwidth memory (with a total bandwidth of 2.4TB/s), multimedia engine, etc., and supports PCIe 4.0 x16, with a maximum power consumption of 800W, which can meet the strong computing power demand of large-scale language models and generative AI models.
The new generation of Gaudi 3 is geared towards AI training and inference, upgraded to TSMC's 5nm process, bringing 2x FP8 AI power, 4x BF16 AI power, 2x network bandwidth, and 1.5x memory bandwidth.
Compared to NVIDIA H100, it has a 50% inference performance lead and 40% faster training time on popular LLMs.
Gaudi 3 is expected to significantly reduce the training time for the 7 and 13 billion parameter Llama2 models, and the 175 billion parameter GPT-3 model.
Gaudi 3's inference throughput and energy efficiency are also excellent on the Llama 7/70 billion parameter, and Falcon 180 billion parameter large language models.
Gaudi 3 offers a wide range of flexible forms, includingOAM-compatible mezzanine cards, universal substrates, PCIe expansion cardsThe company's products are designed to meet the needs of different applications.
Gaudi 3 provides open, community-based software and industry-standard Ethernet networking.Flexibility to scale from a single node to clusters with thousands of nodes, super-clusters and mega-clusters to support large-scale inference, fine-tuning and training.
Gaudi 3 AI Accelerator is high-performance, cost-effective, energy-efficient, and rapidly deployable to fully meet the needs of AI applications such as complexity, cost-effectiveness, fragmentation, data reliability, and compliance.
The Gaudi 3 will ship in the second quarter of 2024 for OEMs, including Dell, Wise and Wise, Lenovo, Ultraviolet and others.
Currently, Intel Gaudi Accelerator's industry customers and partners include NAVER, Bosch, IBM, Ola/Krutrim, NielsenIQ, Seekr, IFF, CtrlS Group, Bharti Airtel, Landing AI, Roboflow, Infosys. etc.
also,Intel also announced that it has joined with partners such as Anyscale, DataStax, Domino, Hugging Face, KX Systems, MariaDB, MinIO, Qdrant, RedHat, Redis, SAP, SAS, VMware, Yellowbrick, Zilliz and others to create an open platform to help organizations drive AI innovation.
The program aims to develop an open, multi-vendor AIGC system that provides best-in-class ease of deployment, performance and value through RAG (Retrieval Augmented Generation) technology.
Initially, Intel will utilize the Xeon processors, Gaudi gas pedals, launch a reference implementation of the AIGC pipeline, release a technical conceptual framework, and continue to enhance the capabilities of the Intel Tiber Developer Cloud Platform infrastructure.