TECHnalysis Research Blogs
TECHnalysis Research president Bob O'Donnell publishes commentary on current tech industry trends every week at LinkedIn.com in the TECHnalysis Research Insights Newsletter and those blog entries are reposted here as well. In addition, those columns are also reprinted on Techspot and SeekingAlpha.

He also writes a regular column in the Tech section of USAToday.com and those columns are posted here. Some of the USAToday columns are also published on partner sites, such as MSN.

He also writes a 5G-focused column for Forbes that can be found here and that is archived here.

In addition, he also occasionally writes guest columns in various publications, including RCR Wireless, Fast Company and engadget. Those columns are reprinted here.

July 18, 2024
The Rapidly Evolving World of AI PCs

By Bob O'Donnell

Even though it’s only been two months since they were first announced and just a month since they’ve shipped, Copilot+ PCs are starting to have a noticeable impact on the PC market. Major PC OEMs like Dell, HP, Lenovo and Microsoft’s Surface group have already shipped slick new NPU-accelerated, AI-powered PCs and other vendors such as Samsung, Acer and Asus have jumped in with impressive new devices of their own.

Most of the early reviews and my own personal experience with four of the Qualcomm Snapdragon X Elite-powered machines highlights the fact that even though most of the AI software is still in its infancy, the extended battery life of these new PCs is proving to be breath of fresh air.

What’s particularly interesting is there’s a sense that these new Arm architecture-based machines are creating a new point of reference against which other PCs are starting to be compared. That’s a huge win for both Qualcomm and Microsoft as it shows that their long, hard-fought efforts in trying to redefine the Windows PC experience have already started to pay off.

Of course, the most important part of that redefinition was supposed to be AI-related and, particularly with the delay of the highly anticipated Recall feature from the first version of Windows for these PCs, that vision has definitely not yet come to pass. Add in the fact that much of the early Copilot+ software experiences and applications are of fairly limited use for most people, and you could argue that the AI side of the Copilot+ story has been a bit of a bust. At least, so far.

But in truth, it was relatively easy to predict from even the earliest hints of these devices that the AI story was going to need to develop over time. As with most major hardware innovations, you have to have the hardware in place for a while before you’ll really start to see the kind of impressive innovations and revolutionary experiences that we’re all hoping AI will bring to the PC.

In the case of Recall, it will certainly be interesting to see how well it works in real life, how rapidly (or not) people adapt to it, and what, if any, security- or privacy-related concerns arise as the result of its use. Beyond Recall, though, I’m most excited about AI software opportunities like personalized versions of RAG (Retrieval Augmented Generation) applications, on-device small language models, and intelligent agents that can finally start to do what the early digital assistants were supposed to do (but never did). In all cases, they involve creating more intelligence and insights about my own data, documents, calendar, etc., and making my PC experience easier, more rewarding, and more intelligent. Simple versions of these applications and capabilities are on their way, but I fully expect it to be a year or more before we get the kind of experiences that really start changing people’s minds about what AI on a PC can do.

Even without these types of advanced AI capabilities, however, the performance-related experience of the early Snapdragon X Elite machines is proving to be better than many expected. Straight performance benchmarks to existing x86 machines as well as M3-powered Macs have been as good as if not even a bit better than the lofty expectations that Qualcomm and Microsoft had created.

Even more impressive is that application compatibility—with the notable exception of gaming titles—has also been better than many had hoped. I tried several obscure music-related applications, including VCV Rack 2, which emulates modular synthesizers in software, and they worked with no issue. There are certainly a few remaining holes for some important titles from Adobe and others that need to be updated to Arm compatibility, but the overall situation is much better than what many feared it could be. Plus, it turns out developers need to specifically block Microsoft’s Prism emulator from running in order for it not to try and work. In other words, software vendors must go out of their way to make an effort to prevent emulation of x86 apps on the Qualcomm Arm-based chips. In a weird way, I’d argue that’s a good thing because most app developers are unlikely to make that effort so most apps will likely take advantage of the emulation and should just run.

One of the other interesting fallouts from the new reference point that these first Qualcomm-powered Copilot+ PCs have created, however, is a new target against which competitors can compare themselves. This is quickly becoming very relevant in the world of Copilot+ PCs because both AMD (see “Computex Chronicles Part 2: AMD Leaps into Copilot+ PCs and Outlines Infrastructure GPU Roadmap”) and Intel (see “Computex Chronicles Part 5: Intel Strikes Back”) have announced that their new Copilot+-compatible chips are coming shortly.

In fact, the first PC using AMD’s Ryzen AI 300 SOC was just announced last week by HP. The new Omnibook Ultra, which is expected to ship in early August, features a new Zen5-based CPU core and a 3rd generation NPU that offers 55 TOPs of performance—10 more than the Snapdragon X Elite. AMD and HP are also claiming up to 21 hours of battery life, which is higher than existing x86-based notebooks and theoretically puts it within the range of the Arm-based Qualcomm machines. Real-world testing will be needed to verify those claims and make those comparisons, but this is particularly important given how much attention the battery life has been getting on the Snapdragon X Elite platforms.

For its part, Intel is expected to make additional details about the performance and availability of systems using its Copilot+-compatible Lunar Lake platform at the upcoming IFA show in Berlin in early September. (Qualcomm is rumored to have plans for PC-related announcements of its own at IFA as well.) One early challenge for initial Ryzen AI 300 and Lunar Lake-based PCs is that they will initially be Copilot+-Ready, meaning the software bits from Microsoft won’t be ready and the AI-accelerated NPU-powered apps and services in Windows won’t work until an update is shipped. While there’s no firm date for when that Windows update will appear, it’s widely expected to be before the end of the calendar year.

It’s important to note that it’s not just the x86-side of the code, it turns out much of the labor required is to port the NPU software to each of the company’s own architectures. One of the little understood challenges of writing initial AI software for PCs is that it has to be adapted individually to the Qualcomm, AMD and Intel NPUs, all of which are architected in unique ways and function differently. Once the baseline support for each of these architectures is embedded into core Windows elements such as DirectML and ONNX Runtime the process should be much easier, but there’s a fair amount of work to make that happen.

Not to be left out the equation, Nvidia is also working with Microsoft to bring AI apps built with the PyTorch framework onto PCs with RTX-based Nvidia GPUs. The interesting twist here is that GPUs like the RTX 4090 can offer over 600 TOPs of performance on notebooks and over 1,300 TOPS (!) on desktop systems. Of course, the power draw for discrete GPUs is significantly higher than NPUs or even TPUs which means using them won’t be realistic for many of the existing Copilot+ Windows tasks and apps, but for AI applications that require quick bursts of performance (think photo, video, and audio editing), they could be a great solution.

The bottom-line takeaway is that we’re already seeing a number of very interesting shifts and redefined comparisons in the Copilot+ PC era and given the pace of innovation that’s already on tap, I expect to see significantly more in the months and years to come.

Here’s a link to the original column: https://www.linkedin.com/pulse/rapidly-evolving-world-ai-pcs-bob-o-donnell-fe7cc

Bob O’Donnell is the president and chief analyst of TECHnalysis Research, LLC a market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on LinkedIn at Bob O’Donnell or on Twitter @bobodtech.