Intel Arc GPU performance momentum continues — 2.7X boost in AI-driven Stable Diffusion, largely thanks to Microsoft’s Olive

Intel’s latest Arc Alchemist drivers feature a performance boost of 2.7X in AI image generator Stable Diffusion. Although some of that boost was thanks to good old-fashioned optimization, which the Intel driver team is well known for, most of the uplift was thanks to Microsoft Olive. Microsoft’s machine learning optimization toolchain doubled Arc GPU performance in Stable Diffusion on its own, and it’s not even the first time we’ve seen it happen in GPU drivers.

Microsoft Olive is a piece of software that basically takes an AI or machine learning model and then finds all the ways different hardware can accelerate it. While this isn’t impossible without Olive, the AI-focused toolchain makes it so much easier. There is a large amount of AI hardware on the market made by different companies, and being able to skip a big step in the optimization process is really handy.

Intel Arc Alchemist GPUs are equipped with Xe Matrix Extensions (or XMX) cores, which are essentially Intel’s version of Nvidia’s Tensor cores. These cores can accelerate AI workloads like Stable Diffusion. However, previous drivers didn’t fully take advantage of the XMX cores, and Olive on its own boosted performance by two times when applied to Intel’s existing software.

The remaining performance uplift was accomplished by Intel optimizing the model that Microsoft Olive created, and the end result was a 2.7X boost in Stable Diffusion.

This isn’t the first time we’ve seen such a large performance gain in an AI app, and not even the first time we’ve seen it happen specifically with a GPU and Microsoft Olive. Nvidia used the exact same toolchain to optimize its drivers for Stable Diffusion and also achieved double the performance. 

AMD doesn’t have dedicated AI hardware in its GPUs but the newer RDNA 3 graphics cards are notably better than RDNA 2 GPUs at things like Stable Diffusion. Although AMD hasn’t used Microsoft Olive to optimize its drivers as far as we know, the company is delivering more optimized drivers for AI workloads. Perhaps AMD might want to look into Olive next to see if it can do anything for RX 7000 series cards.