If a pod can reach devices on its own SR-IOV rail but not destinations on another, first check the pod’s interfaces, addresses, and route table. A missing route is only one possible cause: the interface or address may not have attached, the configured gateway may be unreachable, or the external network may lack a return path. Don’t add a second default route until you know which traffic should use it.
Why same-rail traffic can work while cross-rail traffic fails
Multus connects a pod to additional networks, but attaching a secondary SR-IOV interface does not by itself guarantee that the pod has a route for every destination reachable through that interface. The pod’s route table determines which next hop and interface it uses for an outbound destination; the network beyond the pod must also be able to deliver replies.
As an Amazon Associate I earn from qualifying purchases.
Multus documentation notes that, typically, “the default route for a pod will route traffic over the eth0 and therefore over the cluster-wide default network.” A secondary interface can be present and working on its local rail while traffic for another rail still follows a route through eth0 or another unintended interface. Multus CNI documentation
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →The components have separate jobs: the SR-IOV device plugin exposes virtual functions (VFs) as node resources, the SR-IOV CNI configures an assigned VF, and Multus coordinates attaching the additional network. A successful pod start alone does not prove that the intended IPAM route was installed. Oracle’s OKE SR-IOV tutorial describes this component flow in an OKE-specific deployment; its example is not a universal route design.
#1 Best Overall
- Equipped with Intel’s X710 Ethernet Controller
- Dual 10GbE (10G/5G/2.5G/1G/100M) ports allows connecting to multiple high speed networking devices
- PCIe Gen 3 x4 (compatible with PCIe x4, x1, up to x4 slots are recommended)
- Supports Port Trunking to combine both ports to achieve up to 20 Gbps transfer speeds for accelerating file sharing and intensive data transfer
- Supports SR-IOV and iSCSI to greatly boosts network efficiency and is ideal for I/O-intensive and latency-sensitive virtualization applications and data centers
Diagnose the failure in order
1. Check that the intended interface and address exist
Run these read-only checks, adapting the namespace, pod name, node name, and Multus labels to your cluster:
kubectl describe pod <pod> -n <namespace>
kubectl logs -l app=multus -n kube-system
kubectl describe node <node>
kubectl exec -n <namespace> <pod> -- ip link show
kubectl exec -n <namespace> <pod> -- ip addr showLook for the intended secondary interface and its assigned address. If either is absent, investigate pod events, Multus logs, node allocatable SR-IOV resources, and the NetworkAttachmentDefinition (NAD) before changing routes. Operator troubleshooting guidance identifies these as useful attachment checks; log labels and namespaces vary by installation. SR-IOV Network Operator troubleshooting guide
2. Compare the route table with the failing destination
From the pod, run ip route. Record the destination prefix, next hop, and interface selected for the failing destination, then compare them with the source and destination addresses of the flow. If the expected destination route is missing, inspect the NAD’s .spec.config and its ipam section. Check that the selected IPAM plugin supports the configured route and gateway fields and that they are placed where that plugin expects them. The SR-IOV CNI documentation’s example puts both routes and gateway inside ipam. SR-IOV CNI documentation
Free tools Windows power users keep installed
One-click scans. No signup required.
Rank #2
- 【Controller】: 25GbE PCI-E NIC with Original Mellanox ConnectX-4 Lx controller, which provide true hardware-based I/O isolation with unmatched scalability and efficiency, achieving the most cost-effective and flexible solution for Web 2.0, cloud, data analytics, database, and storage platforms.
- 【Data Rate】:Dual SFP28 Ports(1GbE/10GbE/25GbE) let you connect to network cable for meeting the demands of data center environments.PCIe v3.0 (8.0GT/s) x8(Compatible with 2.0/1.1); X8/X16 Lane.
- 【Technical Support】:iPXE, DPDK, iSCSI, TCP/IP, UDP/IP, Jumbo Frames, RDMA（RoCE v1, RoCE V2）,ASAP², VMDq, SR-IOV, RSS, IPsec.
- 【Supported Operating Systems】:Windows; Windows Server; Linux Stable Kernel version; Ubuntu; Vmware ESXi; Citrix XenServer; Deepin; RHEL/CENTOS; Freebsd; OFED AND WINOF-2; Mikrotik; Debian; BCLINUX; ALIOS; Euler; KYLIN; etc.
- 【I/O virtualization, multi-VM support】：SR-IOV technology enables efficient management of I/O resources of virtual machines by sharing physical resources. And Infiniband technology fully meets the needs of high bandwidth and low latency in big data, its aggregation on virtual I/O and flat network architecture provide a huge pipeline that can be dynamically distributed on demand to improve availability and load balancing.
3. If a route exists, test the next hop and return path
A route in the pod selects an outbound path; it cannot establish that the gateway is reachable or that replies can get back. Confirm with the network team that the gateway is reachable on the intended rail, the VF and switch configuration use the expected VLAN, and routers have a return path to the pod subnet. Check any applicable network policy as well. If the route looks right but traffic still fails, these external-path checks matter as much as the pod configuration.
Change the narrowest confirmed cause
- Interface or address missing: investigate VF allocation and node resources, pod events, Multus logs, and NAD attachment and IPAM configuration.
- Interface and address present, destination route absent: correct the relevant route and gateway entries in the NAD, using syntax supported by the deployed IPAM plugin.
- Route present, traffic still failing: verify next-hop reachability, VLAN and rail connectivity, applicable policy, and external return routing rather than adding another route blindly.
Restart or recreate workloads only as required by the cluster’s CNI or operator workflow. The cited documentation does not establish a single safe rollout procedure for every Kubernetes distribution.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What the SR-IOV CNI route example does—and does not—tell you
The SR-IOV CNI project publishes this illustrative configuration fragment:
Rank #3
- 【Controller】:10GbE PCI-E NIC with Original Intel ELX550AT2 controller, which supports single-root I/O virtualization and improves server stability.
- 【Data Rate】:Dual copper RJ45 ports(100MbE/1GbE/2.5GbE/5GbE/10GbE) let you connect to network cable for meeting the demands of data center environments.PCIe v3.0 (8.0GT/s) x4; (Compatible with 1.1/2.0), X4/X8/X16 Lane.⭐If the X550 NIC cannot negotiate to 2.5G/5G automatically, please try configuring it to 2.5G/5G manually, or seek assistance from customer support.⭐
- 【Technical Support】:On-chip QoS and Traffic management; FPP; Load balancing on multiple CPUs; VMDq; PCI-SIG* SR-IOV; Intel Data Directl/O Technology; TCP checksum offloading capabilities; iSCSI,FCoE,NFS; Jumbo Frames;PXE;DPDK;DCB;Auto-MDIX.
- 【Supported OS Online NVM Firmware Update】：Equipped with Intel official NVM Update Utility, this X550-T2 card enables in-system firmware refresh under Windows, Linux, VMware ESXi without entering BIOS or bootable USB drive. You can batch upgrade multiple adapters remotely, minimize business downtime and cut manual maintenance workload for data center servers.
- 【Supported Operating Systems】: Windows, Windows Server, Linux*RHEL, SUSE, Ubuntu, FreeBSD, Vmware ESX/ESXi, UEFI, WinPE, etc.
{
"type": "sriov",
"cniVersion": "0.3.1",
"name": "sriov-network",
"ipam": {
"type": "host-local",
"subnet": "10.56.217.0/24",
"routes": [{ "dst": "0.0.0.0/0" }],
"gateway": "10.56.217.1"
}
}These values demonstrate documented syntax; they are not production values or a prescribed multi-rail configuration. In particular, the sample’s 0.0.0.0/0 is a default route, not a destination-specific cross-rail route to copy into every NAD. Multiple defaults, overlapping prefixes, or mismatched forward and return paths can cause new failures. Choose route prefixes and gateways from the actual address plan, and confirm the active IPAM plugin’s supported configuration. SR-IOV CNI documentation Multus CNI documentation
Information needed to choose a route safely
There is no universal cross-rail route to prescribe from the symptom alone. Before changing the NAD, collect:
- Pod interface and address output, plus its route table.
- The NAD configuration and IPAM plugin type.
- The CIDRs for each rail, gateway addresses, and the intended cross-rail destination prefixes.
- Kubernetes and CNI versions, and confirmation that the external routers have return paths.
- The desired behavior for cluster services and other traffic currently using the primary network.
When more than one route design is plausible, compare destination-prefix coverage and overlap, gateway reachability on each rail, the desired default-network behavior, symmetric forward and return paths, and support in the deployed IPAM and CNI versions. The actual choice depends on the cluster and network topology. A reader discussion describes the symptom as same-rail traffic working while cross-rail traffic does not and mentions trying IPAM routes, but it is anecdotal and does not establish a correct configuration or fix. Reader discussion
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
