October 29, 2024
By Bob O'Donnell
While most people think about Cisco as a company that can link infrastructure elements together in data centers and the cloud, it is definitely not the first company that comes to mind when you mention GenAI. At its recent Partner Summit event, however, the company made several announcements targeted at trying to change that perception.
Specifically, Cisco debuted several new Nvidia GPU and AMD CPU-equipped servers that are specifically targeted for AI workloads, a new high-speed network switch optimized for interconnecting multiple AI-focused servers, and several preconfigured PODs of compute and network infrastructure designed for specific types of applications.
On the server side, Cisco’s new UCS C885A M8 Server packages up to eight Nvidia H100 or H200 GPUs and AMD Epyc CPUs into a compact size rack server that can do everything from model training to fine tuning. Configured with both Nvidia Ethernet cards and DPUs, the system can function on its own or be networked with other servers into a more powerful system. The new Nexus 9364E-SG2 switch, based on Cisco’s latest G200 custom silicon, offers 800G speeds and large memory buffers to enable high speed, low latency connections across multiple servers.
The most interesting new additions come in the form of AI PODs, which are Cisco Validated Designs (CVDs) that incorporate a combination of CPU and GPU compute, storage and networking along with Nvidia’s AI Enterprise platform software. Basically, they’re completely preconfigured infrastructure systems that provide a much easier plug-and-play type of solution for organizations to get started with their AI deployments, which is something that a lot of companies beginning their GenAI efforts really need.
Cisco is offering a range of different AI PODs that are customized for different industries and applications, helping organizations take some of the guesswork out of the process of selecting the infrastructure they need for their specific requirements. Also, because they come with Nvidia’s software stack, there are a number of industry-specific applications and software building blocks (e.g., NIMs) that organizations can use to build from. Initially, the PODs are targeted more towards AI inferencing than training, but over time, Cisco plans to offer more powerful PODs that can handle AI model training as well.
Another key aspect of the new Cisco offerings is a link to its Intersight management and automation platform, allowing companies the ability to better manage the devices and to more easily integrate them into their existing infrastructure environments.
The net result is that it provides Cisco and its sales partners with a new set of tools that they can sell to their long-established base of enterprise customers. In reality, Cisco’s new server and compute offerings wouldn’t likely be attractive to the big cloud customers that have been the early purchasers of this type of infrastructure. (Cisco’s switches and routers, on the other hand, are key components for hyperscalers.) However, it’s becoming increasingly clear that enterprises want to start building up their own AI-capable infrastructure as their own GenAI journeys move forward. While there’s little doubt that many (perhaps even most) AI application workloads will continue to exist in the cloud, many companies are realizing that there is going to be a very real need to do some of this work on prem.
In particular, because good AI applications need to be trained or fine-tuned on a company’s best (and most likely most sensitive) data, many organizations aren’t comfortable having that data (and models based on it) running in the cloud. In that light, even though Cisco is a bit late in bringing certain elements of its AI-focused infrastructure to market, the timing for its most likely audience could prove to be just right. As Cisco’s Jeetu Patel commented during the Day 2 keynote for the event, “Data centers are cool again.” This point that was also reinforced by the recent TECHnalysis Research survey report, “The Intelligent Path Forward: GenAI in the Enterprise”, which found that 80% of companies doing GenAI work had an interest in running some of those applications on prem.
Ultimately, the expected market growth for on-site data centers does offer some intriguing new possibilities for Cisco and other traditional enterprise hardware suppliers. Whether it’s because of data gravity, privacy, governance or other issues, it seems clear now that while the move to hybrid cloud took nearly a decade, the transition to hybrid AI models that leverage the cloud and on-prem resources (not to mention on-device AI applications for PCs and smartphones) will be significantly faster. How the market reacts to that rapid evolution will be very interesting to see.
Here’s a link to the original column: https://www.linkedin.com/pulse/cisco-expands-ai-infrastructure-offerings-bob-o-donnell-65vcc
Bob O’Donnell is the president and chief analyst of TECHnalysis Research, LLC a market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on LinkedIn at Bob O’Donnell or on Twitter @bobodtech. |
