reported for artificial intelligence (AI) and high-performance computing (HPC) applications.Nvidia H100 GPU Lead times have been dramatically shortened, from 8-11 months previously to just 3-4 months. This has led to some stockpiling companies trying to sell their surplus H100 80GB processors, as it's now easier to rent chips from large companies like Amazon Cloud Services, Google Cloud and Microsoft Azure.
According to The Information, some companies are reselling their H100 GPUs or reducing orders due to reduced scarcity and the high cost of maintaining unused inventory, in contrast to last year's scramble for NVIDIA Hopper GPUs. Despite improved chip availability and significantly shorter lead times, the report says that the demand for the AI ChipsDemand continues to outstrip supply, especially from companies that train large-scale language models (LLMs), such as OpenAI.
Cloud service providers (CSPs) such as Amazon AWS offer on-demand rental of H100 GPUs, alleviating some of the demand pressure and reducing wait times.
However, because it takes thousands of GPUs to train large language models, such companies still face supply bottlenecks and sometimes have to wait months to get the processing power they need. As a result, the price of processors such as NVIDIA's H100 has not fallen, and the company has maintained high profit margins.
As chip supply bottlenecks have eased, buyer behavior has shifted.Organizations are becoming more price and procurement/leasing conscious, preferring smaller GPU clusters and focusing on the economic viability of their business.
The AI space is likely to see a more balanced market landscape due to the increasing performance and software support of alternatives (e.g., processors from AMD or AWS), coupled with more prudent spending on AI processors.