Sundry Photography
Lately, whenever a major tech industry company hosts a large event, it almost inevitably ends up discussing their strategy and latest products focused on generative AI. Such was the case with major semiconductor provider AMD (NASDAQ:AMD) as they held their Advancing AI event in San Jose. The company officially unveiled their previously announced Instinct MI300 line of GPU-based AI accelerators for the data center, discussed the growing software ecosystem for the products, laid out their roadmap for AI-accelerated PC silicon, and sprinkled in several intriguing technological advancements along the way.
In truth, there was a relative scarcity of “truly new” news, and yet, you couldn’t help but walk away from the event feeling impressed. AMD told a solid, comprehensive product and technology story, highlighted a large (perhaps even too large?) number of clients/partners, and demonstrated the scrappy, competitive ethos of the company under CEO Lisa Su. On a practical level, I also walked away even more certain that the company is going to be a serious competitor to Nvidia (NVDA) on the AI training and inference front, an ongoing leader in supercomputing and other high-performance computing (HPC) applications, and an increasingly capable competitor in the upcoming AI PC market. Not bad for a 2-hour keynote.
Not surprisingly, most of the event’s focus was on the new Instinct MI300X, which is clearly positioned as a competitor to Nvidia’s market-dominating GPU-based AI accelerators, such as their H100. While much of the tech world has become infatuated with the GenAI performance that the combination of Nvidia’s hardware and CUDA software have enabled, there’s also a rapidly growing recognition that their utter dominance of the market isn’t healthy for the long term. As a result, there’s been a lot of pressure for AMD to come up with something that’s a reasonable alternative, particularly because AMD is generally seen as the only serious competitor to Nvidia on the GPU front.
Thankfully, the MI300X undoubtedly triggered enormous sighs of relief heard around the world, as initial benchmarks suggest that AMD achieved exactly what many were hoping for. Specifically, AMD touted that they could match the performance of Nvidia’s H100 on AI Model Training and offered up to a 60% improvement on AI inference workloads. In addition, AMD touted that combining eight MI300X cards into a system would enable the fastest Generative AI computer in the world and offer access to significantly more high-speed memory than the current Nvidia alternative. To be fair, Nvidia has already announced the GH200 (codenamed “Grace Hopper”) that will inevitably offer even better performance, but as is almost inevitably the case in the semiconductor world, this is bound to be a game of performance leapfrog for many years to come. Regardless of how people select to adopt or challenge the benchmarks, the key point here is that AMD is now ready to play the game.
Given that level of performance, it wasn’t terribly surprising to see AMD parade a long list of partners across the stage. From major cloud providers admire Microsoft Azure (MSFT), Oracle Cloud (ORCL) and Meta (META) to enterprise server partners admire Dell Technologies (DELL), Lenovo (OTCPK:LNVGY) and Super Micro (SMCI), there was nothing but praise and excitement from these partners. Of course, it’s easy to grasp given that these are companies that are eager for an alternative and additional supplier to help them confront the staggering demand they now have for GenAI-optimized systems.
In addition to the MI300X, AMD also discussed their Instinct MI300A, which is the company’s first APU designed for the data center. The MI300A leverages the same type of GPU XCD (Accelerator Complex Die) elements as the MI300X, but includes six instead of eight and uses the additional die space to blend eight Zen4 CPU cores. In addition, through the use of AMD’s Infinity Fabric chip-to-chip interconnect technology, it provides shared and simultaneous access to a large, shared set of high-bandwidth memory, or HBM, for the entire system. (One of the interesting technological sidenotes from the event was that AMD announced plans to open up the previously proprietary Infinity Fabric technology to a limited set of partners. While no details are known just yet, it could conceivably direct to some interesting new multi-vendor chiplet designs in the future.)
This simultaneous CPU and GPU memory access is essential for HPC-type applications, and that capability is apparently one of the reasons that Lawrence Livermore National Labs chose the MI300A to be at the core of its new El Capitan supercomputer being built in conjunction with HPE. El Capitan is expected to be both the fastest and one of the most power-efficient supercomputers in the world.
On the software side of things, AMD also made numerous announcements around its ROCm software platform for GenAI, which has now been upgraded to version 6. As with the new hardware, they discussed several key partnerships that build on previous news (such as with open-source model provider Hugging Face and the PyTorch AI development platform) as well as debuting some key new ones. Most notable was that OpenAI said it was going to bring native uphold for AMD’s latest hardware to version 3.0 of its own Triton development platform. This will make it trivial for the many programmers and organizations eager to jump on the OpenAI bandwagon to leverage AMD’s latest (and gives them an alternative to the Nvidia-only choices they’ve had up until now).
The final portion of AMD’s announcements covered their advancements in AI PCs. Though the company doesn’t get much credit or recognition for it, they were actually the first to blend a dedicated NPU into a PC chip with last year’s launch of the Ryzen 7040. The XDNA AI acceleration block it includes leverages technology that AMD acquired through its Xilinx purchase. At this year’s event, the company announced the new Ryzen 8040, which includes an upgraded NPU with 60% better AI performance. Interestingly, they also previewed their subsequent generation codenamed “Strix Point,” which isn’t expected until the end of 2024. The XDNA2 architecture it will include is expected to offer an impressive 3x improvement versus the 7040. Given that the company still needs to sell 8040-based systems in the meantime, you could argue that the “teaser” of the new chip was a bit unusual. However, what I think AMD wanted to do – and what I believe they achieved – in making the preview was to hammer home the point that this is a incredibly fast-moving market and they’re ready to contend. (Of course, it was also a shot across the competitive bow to both Intel (INTC) and Qualcomm (QCOM), both of whom will present NPU-accelerated PC chips over the next few months.)
Once again, in addition to the hardware, AMD discussed some intriguing AI software advancements for the PC, including the official release of Ryzen AI 1.0 software for easing the use of and accelerating the performance GenAI-based models and applications on PCs. AMD also brought Microsoft’s new Windows leader Pavan Davuluri onstage to talk about their work to supply native uphold for AMD’s XDNA accelerators in future version of Windows as well as converse the growing topic of hybrid AI, where companies expect to be able to split certain types of AI workloads between the cloud and client PCs. There’s much more to be done here – and across the world of AI PCs – but it’s definitely going to be an interesting area to watch in 2024.
All told, the AMD AI story was an impressive one, and it was undoubtedly told with a great deal of enthusiasm. From an industry perspective, it’s great to see additional competition, as it will inevitably direct to even faster developments in this exciting new space (if that’s even possible!). In order to really make a difference, however, AMD needs to continue executing well to its vision. I’m certainly confident it’s possible, but there’s a lot of work still ahead of them.
Disclaimer: Some of the author’s clients are vendors in the tech industry.
Disclosure: None.
Source: Author
Editor’s Note: The summary bullets for this article were chosen by Seeking Alpha editors.
