AMD recently published a series of benchmarks claiming that its Riptide Mobile 7040 Phoenix series and 8040 series processors are capable of running theLarge Language Models (LLMs), with a performance lead of up toIntelThe latest Core Ultra Meteor Lake CPU reaches 79%.
This test compares the AMD Renegade 7 7840U and Intel Core Ultra 7 155H processors, both of which are equipped with hardware neural network processing units (NPUs.) AMD showed multiple slides comparing the performance of these two processors for large language models such as Mistral 7b, Llama v2, and Mistral Instruct 7B. Performance. In the Llama v2 dialog test, the AMD processor outperformed Core Ultra 7 by 141 TP3T using Q4 bitwidth, while in the Mistral Instruct test, the AMD processor was 171 TP3T faster using the same bitwidth.
In the same large language modeling tests, AMD processors also showed an advantage in responsiveness. For example, the AMD processor outperformed the Core Ultra 7 by 791 TP3T in the "Generate First Token for Example Prompt" speed test of the Llama v2 dialog test, and by 411 TP3T in the Mistral Instruct test.
AMD also showed another set of test results covering the Llama v2 7B dialog test at various bitwidths, block sizes and quality levels. Overall, the Rui Long 7 7840U outperforms the Intel competition by an average of 551 TP3T, with performance gains of even 701 TP3T at the Q8 bitwidth setting, although AMD also said that in real-world applications running a large language model, a better balance can be achieved with 4-bit K M quantization, while tasks such as coding that require extreme precision can be performed with 5-bit K M quantization. settings.
As it stands, AMD is temporarily ahead of Intel in terms of AI performance. While the Riptide 7040 series architectures are on par with Meteor Lake in terms of theoretical performance (TOPS), Tom'sHardware found late last year that AMD usually beats Meteor Lake when it comes to AI loads.This seems to be more of a software optimization issue than a hardware or driver issue.
This performance gap shouldn't last long, though. After all, the development of neural network processing units is just beginning. If more apps don't adopt the OpenVINO framework, expect Intel to change its strategy and look for an optimization that's easier for developers to adopt. Intel is also planning to release its next-generation Lunar Lake mobile CPU architecture later this year, which will reportedly deliver 3x the AI performance of Meteor Lake (as well as a significant increase in IPC performance in the core part of the CPU).
All in all, AMD currently has the edge in neural network processing unit performance, especially with the RuiLong 8040 series processors that have stronger NPU performance. However, with the release of Intel's Lunar Lake architecture at the end of the year and its AI optimization plans, the tide could turn.