When smartphones emerged nearly two decades ago, they changed mobile communications forever.
Today, SmartNICs are fomenting a similar kind of revolution in the data center.
What Are SmartNICs?
A SmartNIC, or smart network interface card (NIC), is a programmable accelerator that makes data center networking, security and storage efficient and flexible.
SmartNICs offload from server CPUs an expanding array of jobs required to manage modern distributed applications.
Under the hood, they pack a variety of connected, often configurable units that put the smarts in SmartNICs. These silicon blocks act like a committee of experts that make decisions on how to process and route packets of data as they flow through the data center.
How Does a SmartNIC Work?
Most of these blocks are highly specialized hardware units called accelerators that run communications jobs more efficiently than CPUs. Some are flexible units that users can program to handle their changing needs and keep up with network protocols as they evolve.
This combination of accelerators and programmable cores help SmartNICs deliver both performance and flexibility with outstanding price performance.
Many Schools for SmartNICs
Every vendor has its own list of SmartNIC features. They can be long, filled with acronyms and hard to compare.
A simple test of which SmartNIC is best for you is to see which one has the list that contains the features your data center needs.
Every vendor also has its own ideas for how to make its SmartNICs smart.
Some are using FPGAs which promise flexibility. However, FPGAs are difficult to program, expensive and don’t deliver the same performance as integrated accelerators.
Others are using dedicated ASICs, which typically offer the best price performance and some flexibility. This is the approach the NVIDIA ConnectX family takes.
Others use system-on-chip designs that blend dedicated hardware accelerators with programmable processors. These SmartNICs tend to have the highest performance and maximum flexibility, so they cost more.
To help sort through the options, we wrote an article on finding the best SmartNIC for your situation.
Putting the NIC in SmartNIC
In 1973, Robert Metcalfe and colleagues at Xerox PARC (Palo Alto Research Center) prototyped one of the first NICs for technology he dubbed Ethernet.
The name stuck. It became an industry standard in 1985, and for many people it’s now a synonym for computer networking.
Connecting multiple personal computers in a local-area network was considered state of the art at the time.
In 1979, Metcalfe co-founded 3Com to build commercial NICs for the early PC market of the day. Since then, dozens of companies have made NICs in many flavors, speeds and sizes.
Today we call these Foundational NICs. They enable a basic network connection between two or more computers, but they lack the smarts to support more than a few value-added features.
To get an insider’s view of the early history of NICs, listen to a recording of NVIDIA CEO Jensen Huang interviewing Metcalfe in 2020, the 40th anniversary of Ethernet.
Data Center Networks Go Global
Today’s computers are both smaller and vastly larger than those that used the first NICs. Smartphones are more widely used than PCs and cloud-service providers have scaled data centers to global networks of warehouse-sized systems.
“The data center is the new unit of computing,” said Huang in a keynote at GTC in April.
But fueled by techniques such as virtualization and microservices, the growing stack of tasks data center CPUs are expected to handle has become unmanageable.
“A third of the roughly 30 million servers shipped each year are consumed by running the data center’s software-defined networking stack,” said Huang in the keynote.
“This workload is increasing much faster than Moore’s law, so unless we offload and accelerate this workload the data center will have fewer and fewer CPUs to run applications,” he added.
Putting the Smarts in SmartNICs
Seeing this trend, companies like Mellanox, now part of NVIDIA, had been packing more smarts in their NICs to offload work from overburdened CPUs in the data center.
NVIDIA’s latest ConnectX-7 SmartNICs handle a laundry list of networking, security and storage jobs. For example, they can:
- accelerate popular and custom software-defined networking stacks
- provide security with in-line encryption/decryption
- enable firewalls and a hardware root-of-trust for a secure boot
- run popular security protocols such as Transport Layer Security, IP Security, and MAC Security
- handle storage and data-access protocols such as RoCE, GPUDirect Storage, NVM Express and Transmission Control Protocol
- support virtualized data centers with Single-Root I/O Virtualization and virtual switching and routing
They do all this with low latency while providing anywhere from 25 to 400 Gbits/s of data throughput. As a result, data centers are more responsive and efficient.
Unlike Foundational NICs, SmartNICs have the spare muscle and flexibility to handle new tasks as they emerge, too. Network protocols are constantly changing, and new concepts for moving data, called network overlays, seem to come along on a regular cadence.
Users can configure SmartNICs to grab the latest software libraries that implement these changes.
Home in the Smart Mainstream
That’s why SmartNICs aren’t just for the world’s largest cloud service providers.
SmartNICs are used by telecommunications providers and media companies. They run in storage servers, database clusters and data warehouses.
Financial analysts use low latency SmartNICs for high frequency trading. Machine learning specialists use them to speed big data analysis.
In short, SmartNICs are going mainstream. NICs with no or low smarts are finding their way to the remainder bin.
New Horizons in Networking
SmartNICs are the Goldilocks of networking. They have more capability than foundational NICs, but they are not the big bear of wired communications.
The top spot today is reserved for data processing units (DPUs). These premium processors are fully programmable in user friendly languages like C, typically on banks of power-efficient Arm cores.
If your data center is among the largest or most demanding, you probably need DPUs to run their communications. But that’s another story.
To find out more on NVIDIA BlueField DPUs, watch Jensen Huang’s April 2021 keynote below.
Learn more about SmartNICs on the NVIDIA Technical Blog.