Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix NowHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
FREEDOM251

Freedom251 › Blog › AI infrastructure

What RDMA, SmartNICs and Computational Storage Do in AI Infrastructure

By Freedom251 Team · · 9 min read

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

RDMA, SmartNICs and computational storage improve different parts of an AI data path. RDMA moves data between machines or accelerators while reducing host-CPU copying. SmartNICs and DPUs offload networking, security and storage services from that CPU. Computational-storage devices run selected filtering, transformation or inference operations near the data, so fewer bytes have to travel to a host or accelerator.

They are complementary architectural options, not three names for the same product and not a package every AI server must use. The right choice depends on whether your bottleneck is network transfer, infrastructure processing, storage traffic or the application itself.

As an Amazon Associate I earn from qualifying purchases.

Start with the AI data path

An AI training or inference job repeatedly moves data through several stages:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  1. Datasets, checkpoints and logs reside on local or shared storage.
  2. Storage services deliver batches to a server.
  3. The server’s network adapter moves data between hosts, storage systems and accelerators.
  4. The host CPU coordinates protocols, security, memory copies and application logic.
  5. GPUs or other accelerators consume the data and return results.

A conventional path can require multiple copies: a device places data in host memory, the CPU processes or copies it, and a driver then transfers it to accelerator memory. CPU cycles, memory bandwidth, network latency and storage throughput can all become limiting factors. The three technologies in this article intervene at different points in that path.

#1 Best Overall
Sale
TP-Link 10/100/1000Mbps Gigabit Ethernet PCI Express Network Card, Win10/11 | PCIE Network Adapter, Network Card, PCIe Ethernet Card for PC (TG-3468)
  • Ultra-Fast: 10/100/1000Mbps PCIe Adapter upgrade your Ethernet speed to Gigabit
  • Automation: Wake-on-LAN supporting Auto-Negotiation and Auto MDI/MDIX
  • Supports: IEEE802.3x Flow Control for Full-duplex Mode and backpressure for Half-duplex Mode; 4k Bytes Port: 1x 10/100/1000Mbps RJ45 Network Media
  • Compatibility: Windows 11, 10, 8.1, 8, 7, Vista, XP
  • Dual Bracket: Low profile and standard profile bracket inside works with both mini and standard size PCs.
  • RDMA changes how data is transferred across a network or fabric.
  • SmartNICs and DPUs perform infrastructure work on or beside the network adapter.
  • Computational storage performs selected work beside the storage media.

Removing a copy or an interrupt does not automatically make an end-to-end application faster. Protocol setup, topology, drivers, queueing, data preparation and the workload’s compute-to-I/O ratio still determine the result.

What RDMA does for AI systems

RDMA in plain language

Remote direct memory access (RDMA) lets a capable device transfer data to or from memory on another machine without routing every byte through the receiving host CPU’s ordinary copy path. The network adapter and software stack establish the connection, register memory and move payloads directly between permitted memory regions.

For distributed AI, that can reduce host-CPU involvement and intermediate memory copies when one server sends batches, gradients, parameters or activations to another. It can also help storage-to-accelerator paths when the storage and accelerator stack supports a direct data path.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

RDMA is a capability, not a complete product stack

RDMA still requires compatible adapters, switches or fabrics, firmware, drivers, memory-registration support and application libraries. A direct transfer does not remove the need to configure addressing, congestion control, permissions, failure handling and monitoring. If any component falls back to a conventional path, the expected benefit may disappear.

RDMA can run over purpose-built fabrics or Ethernet-based RoCE implementations. AMD’s Embedded RDMA Enabled NIC: Applications, version 4.2 (released December 19, 2024), lists sensor-data transfer, video and image transfer, and remote-storage nodes over RoCE v2. Those examples show that RDMA is useful beyond one GPU vendor’s software stack; they are not a claim that every deployment is AI-specific.

RDMA for GPUs: keep the terminology precise

NVIDIA’s GPUDirect family includes technologies that allow network adapters and storage devices to read or write GPU memory under supported hardware and software configurations. GPUDirect RDMA describes direct communication between GPUs in remote systems and the network path. It is an NVIDIA implementation and ecosystem built on RDMA concepts, not a synonym for every RDMA deployment.

GPUDirect Storage addresses a different leg: moving data from supported storage devices toward GPU memory while avoiding unnecessary host-memory staging. A system can use ordinary RDMA without GPU memory, or use a GPU-direct path only where the complete compatibility matrix supports it.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #2
Ubit WiFi Card 6E 5400Mbps for Intel (6GHz&5Ghz&2.4GHz) PCIe WiFi Card, BT 5.3, PCIe Network Card with OFDMA,Ultra-Low Latency, Supports Windows 10/11 (64bit)
  • WiFi 6e up to 5400Mbps (6 GHz/2400 Mbps + 5 GHz/2400 Mbps + 2.4 GHz/600 Mbps) - Supports Windows 10/11 (64bit) - Driver installation required via Website or included CD.
  • Wi-Fi 6E technology powered by intel ax210, The latest Wi-Fi 6E standard gives you incredible speed ultra-low latency and uninterrupted connectivity with OFDMA and MU-MIMO technology Utilizing next-generation Wi-Fi 6E technology Ubit AX210S is designed to excel in even the busiest of networks for next-level entertainment.
  • Minized lag for your pc, revolutionary OFDMA reduces lag so you can enjoy ultra-responsive real-time gaming or an immersive VR experience.
  • Bluetooth 5.3 for greater speeds and conerage Ubit AX210S is equipped with the latest Bluetooth 5.3 technology achieving 2x faster speeds and 4x broader coverage than Bluetooth 4.2 connect as many devices as you want including game controllers headphones and keyboards for the ultimate setup.
  • Support windows 10/11 64-bit OS. compatible with all PCIe standard. standard and Low-Profile brackets to adapt to your different computer sizes. Please note: To utilize 6GHz Wi-Fi, the user might use Windows 11 or upgrade to Windows 10 Dev version.

Where RDMA helps—and where it does not

  • Potential fit: distributed training with frequent host-to-host exchanges, remote accelerator pools, high-throughput storage fabrics and inference services that exchange large tensors.
  • Not a guarantee: small messages, CPU-bound preprocessing, an oversubscribed switch, poor congestion control or an application that cannot keep the transfer queues full.
  • Operational requirement: test the entire path, including NIC firmware, switch configuration, kernel or runtime drivers, accelerator support and the framework’s communication library.

RDMA changes the transfer path; it does not make network capacity, GPU compatibility or storage performance irrelevant.

What SmartNICs and DPUs do

From a conventional NIC to an infrastructure processor

A conventional network interface controller connects a server to a network and handles packet transmission and reception. A SmartNIC adds hardware acceleration and often programmability for tasks such as virtual switching, telemetry, encryption, storage protocols or traffic shaping.

A data processing unit (DPU) is a programmable processor class intended to offload, accelerate or isolate data-centric services from the server’s main CPU. Depending on the product, a DPU can run networking, security and storage functions on its own cores and expose controlled resources to host workloads.

Names are not universal. One vendor’s “SmartNIC,” “DPU” or “SuperNIC” may differ in processor cores, memory, isolation model, accelerators and software support from another vendor’s device. Evaluate the documented functions rather than selecting by label alone.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Typical offloads

  • Networking: virtual switching, packet processing, RDMA transport and traffic steering.
  • Security: encryption, firewalling, tenant isolation, key handling and policy enforcement.
  • Storage: NVMe-over-fabrics services, data movement and storage virtualization.
  • Infrastructure control: telemetry, quality-of-service policies and management-plane functions.

Offloading these jobs can reserve host CPU cycles for model serving, preprocessing and orchestration. Isolation can also make a shared infrastructure service less dependent on the operating system that runs the application.

Examples and stated capability limits

NVIDIA identifies ConnectX SmartNICs and BlueField DPUs in its AI and GPU data-path material. Its DOCA software platform supplies libraries, services and management components for BlueField and ConnectX systems. The DOCA overview advertises connectivity of up to 800 gigabits per second (Gb/s) for the described platform. That is a vendor capability statement, not a universal SmartNIC speed or an independent workload benchmark; the actual rate depends on the specific adapter, port configuration, fabric and software.

What computational storage does

Compute near the data

Computational storage places selected processing functions in or near a storage device. Instead of sending every byte to the host CPU, the device may filter records, convert formats, decompress data, execute a narrow analytic operation or run a supported inference step before returning results.

Rank #3
WAVLINK Gigabit Ethernet PCIe Network Card, 10/100/1000Mbps RJ45 LAN Adapter for Desktop PC, Windows 11/10/Linux Supported 1G PCIe Network Card for PC
  • 1G High-Speed Transmission: This 1G network card ushers in a new era of network transmission speed, offering speeds up to 10 times faster than traditional 100Mbps rates. It provides an exceptional online experience, ensuring swift file transfers, seamless HD video streaming, and lag-free online gaming.
  • PCIe to 1G RJ45: This pcie ethernet card accommodates various PCI Express slot configurations, such as x1, x4, x8, and x16, enabling seamless installation and interchangeability across different motherboards to meet diverse hardware layouts and needs. Note: Compatible with PCI-E slots only, not for PCI slots.
  • Seamless OS Compatibility: Compatible with Windows 11, Windows 10, and Linux systems, this ethernet card guarantees seamless compatibility. This pcie network card eradicates worries about system compatibility and grants you the complete convenience of high-speed networking.
  • Flexible Bracket Versatility: Engineered with low-profile and standard bracket, this pcie nic provides heightened flexibility for diverse computer arrangements. Irrespective of your computer's specifications, effortless installation is ensured, optimizing your hardware setup.
  • Robust Metal Shielding: This ethernet pcie card minimizes electromagnetic interference, enhancing stability and reliability with every connection. (Tip: Please remove the plastic film from the PCI-e interface before connecting to the PC's PCI-e slot.) Excellent heat dissipation can reduce the temperature quickly and maintain the stability of network transmission.

The goal is to reduce unnecessary data movement and host processing. For example, a device could discard records that do not match a query before they cross the storage bus, or transform a stored representation into the format expected by a downstream accelerator.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

It is not an entire AI model inside an SSD

“Computational storage” describes an architecture and programming model, not a promise that a full training job or large language model runs inside a drive. The useful operation must be supported by the device’s processor, memory, firmware and API, and the operation must be worth performing there. Large, irregular or rapidly changing model workloads may still belong on host CPUs or GPUs.

Standards and interoperability

The Storage Networking Industry Association (SNIA) publishes a Computational Storage Architecture and Programming Model. The current page lists version 1.2, published April 21, 2026, and says: “The Computational Storage Architecture and Programming Model defines recommended behavior for hardware and software that supports Computational Storage.” SNIA’s Computational Storage Technical Work Group, updated April 2025, focuses on interoperability and common interfaces.

A standard improves the vocabulary and expected interfaces; it does not mean every computational-storage device supports the same instructions, data formats or performance. Verify the device API, supported operations, host drivers, orchestration tools and upgrade path before designing an application around it.

How the three technologies can work together

Consider a distributed inference service receiving images from remote storage:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  1. Near-storage reduction: a computational-storage device filters corrupted files, extracts metadata or converts images to the model’s input format.
  2. Infrastructure offload: a SmartNIC or DPU handles network virtualization, encryption, storage services and traffic policy without consuming the model server’s main CPU.
  3. Direct transfer: an RDMA-capable path moves the remaining payload toward the target host or accelerator with fewer host-memory copies.
  4. Accelerator execution: the GPU runs inference and returns results through the network or storage service.

This is an architectural synthesis, not a required three-part design. A deployment may use RDMA without computational storage, a DPU without GPU-direct transfers, or computational storage in a server whose network remains conventional.

Comparison by layer and responsibility

Technology Where it acts Work it can bypass or offload Primary requirements Main uncertainty
RDMA Network or fabric data transfer Host CPU copies and intermediate memory staging RDMA-capable adapters, fabric, drivers, registered memory and application support End-to-end compatibility, congestion and workload behavior
SmartNIC Network interface Packet processing, virtual switching, selected security or storage functions Supported hardware accelerators, firmware, host integration and vendor software Capabilities vary widely by model and vendor
DPU Programmable infrastructure processor beside the host Networking, security, storage and isolation services DPU cores, memory, control-plane software and deployment tooling Programming model, operational complexity and isolation boundaries
Computational storage Storage device or storage appliance Selected filtering, transformation or inference before host transfer Supported device functions, APIs, drivers and orchestration Interoperability and limited operation set; not every drive is computational
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Choose based on the actual bottleneck

When network movement dominates

Investigate RDMA when profiling shows that distributed workers spend substantial time exchanging large tensors or waiting on host-mediated copies. Confirm that the framework, accelerator runtime and network fabric can use the intended path. Measure application throughput and tail latency, not only link speed.

Rank #4
EDUP® Gigabit Ethernet PCIE Network Card 10/100/1000Mbps Network Adapter
  • With Gigabit Ethernet card, you can add a 10/100/1000Mbps Ethernet port to any PC through a PCI-E slot. Fits any PCI Express x1, x2, x4, x8 or x16 socket; additional 8cm low-profile bracket provides the option for installation in small size HTPC case.
  • EDUP pci-based 10/100/1000 Mbps Gigabit Ethernet network adapter card provides a simple connection to a Gigabit Ethernet network, and is fully compatible with 802. 1Q virtual LAN (VLAN) tag. And Ethernet card which Is compliant to IEEE802. 3, IEEE802. 3U & IEEE802. 3ab.
  • Gigabit PCI network adapter Suitable for any pci-enabled client, server or workstation, The Gigabit Ethernet network adapter card Supports high Performance Dual channel networking and maximum data transfer speeds of 1000Mbps in each direction (2000 Mbps total) - up to ten times faster than 10/100 Ethernet.
  • Gigabit network card compatible with Windows 11,10, 8/8. 1, 2000, XP, Vista, 7. Please kindly note: This product does not support Linux or Mac OS.
  • The high quality PCI Gigabit NIC with 12cm profile bracket and additional 8cm profile bracket. Support: 2-year and free 24/7 technical support.

When infrastructure consumes host CPU

Consider a SmartNIC or DPU when packet processing, encryption, virtual switching, storage services or tenant isolation compete with model execution. Define which services move to the device, how they are updated, and how operators will observe failures. Offloading a function that is rarely active may add complexity without meaningful relief.

When storage sends too much irrelevant data

Computational storage is most plausible when a simple, selective operation can eliminate substantial data movement: filtering, projection, decompression, format conversion or a narrowly defined inference step. Compare the device’s processing cost and capacity with the host CPU and network cost it replaces.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

When no single bottleneck is proven

Profile first. Collect storage queue depth, network utilization, host CPU time by service, memory-copy volume, accelerator idle time and end-to-end request latency. A faster network will not fix slow decoding; a DPU will not fix insufficient storage media; near-storage filtering will not help when the model itself dominates runtime.

Deployment requirements and failure modes

Compatibility is a matrix, not a checkbox

  • Match adapter, accelerator, storage and switch capabilities.
  • Check firmware, kernel, runtime, driver and communication-library versions together.
  • Verify topology: PCIe placement, NUMA locality, switch paths and link rates.
  • Confirm memory-registration, protection and tenant-isolation behavior.
  • Document fallback behavior when a direct path is unavailable.

Common surprises

  • CPU bypass with CPU work elsewhere: connection setup, protocol handling or preprocessing can remain expensive.
  • Hidden copies: an unsupported buffer type or driver can silently stage data through host memory.
  • Congestion: RDMA reduces copying, but an oversubscribed fabric still queues packets.
  • Incompatible operations: computational storage may support filtering but not the exact format conversion or model operator required.
  • Observability gaps: moving services onto a DPU changes where logs, metrics and security events are generated.
  • Upgrade coupling: a new accelerator driver or firmware release can alter direct-path support.

Use controlled benchmarks that represent production message sizes, concurrency, failures and security policies. Report the complete configuration and workload; do not generalize from a headline link rate.

What the current evidence establishes

The cited vendor and standards material explains mechanisms, interfaces and product capabilities. It does not provide a neutral, directly comparable benchmark of RDMA, SmartNICs and computational storage together, nor an independent percentage improvement in performance, energy use, adoption or cost. NVIDIA’s GPUDirect page makes a “10X better performance” claim for GPUDirect RDMA, but without benchmark conditions alongside that claim it should be treated as a vendor statement rather than a general result.

The defensible conclusion is narrower and more useful: RDMA can reduce host involvement in transfers; SmartNICs and DPUs can move infrastructure services away from the main CPU; and computational storage can reduce data movement by processing selected data near storage. Their value depends on the workload, implementation and supported end-to-end stack.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

AI infrastructurecomputational storagedata movementDPUsGPUDirectRDMASmartNICs

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Crashes, No Sound, or Screen Glitches?Free driver scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.