VXLAN Performance¶

EgressGateway utilizes VXLAN tunnels, which typically introduce an approximate 10% performance loss. If you observe that EgressGateway performance is below expectations, follow these steps for troubleshooting:

1. Verify Node-to-Node Throughput¶

The network speed between host nodes should be as expected. VXLAN performance depends on the host NIC offload settings.

NIC Offloading: The offload settings of the host network interface impact VXLAN performance slightly (in 10G network tests, the difference is about 0.5 Gbits/sec).
Enable Offloading: Run the following command on the host to enable offloading:
```
ethtool --offload <host-interface-name> rx on tx on
```

2. Verify VXLAN Offload Settings¶

VXLAN interface offload settings can significantly impact performance.

In 10G network tests, VXLAN throughput was:
Without checksum offload: 2.5 Gbits/sec
With checksum offload: 8.9 Gbits/sec
Check VXLAN checksum offload status with:
```
ethtool -k egress.vxlan
```
If offload is disabled, enable it by setting the following Helm values:
```
feature:
  vxlan:
    disableChecksumOffload: false
```

Performance Benchmark¶

Physical Server Benchmark¶

Test Environment:

Name	CPU	Memory	Network Interface
Node 1	Intel(R) Xeon(R) E5-2680 v4 @ 2.40GHz	128GB	10G Mellanox
Node 2	Intel(R) Xeon(R) E5-2680 v4 @ 2.40GHz	128GB	10G Mellanox
Target	Intel(R) Xeon(R) E5-2680 v4 @ 2.40GHz	128GB	10G Mellanox

Test Case	Scenario	Throughput
Case 1	Node → Node	9.44 Gbits/sec (send) - 9.41 Gbits/sec (receive)
Case 2	Egress VXLAN → Egress VXLAN	9.11 Gbits/sec (send) - 9.09 Gbits/sec (receive)
Case 3	Pod → Egress Node → Target	9.01 Gbits/sec (send) - 8.98 Gbits/sec (receive)

egress-check

Virtual Machine Benchmark (VMware)¶

Test Environment:

VMware virtual machines with 4 vCPUs and 8GB RAM

Name	CPU	Memory	Network Interface
Node 1	Intel(R) Xeon(R) Gold 5118 @ 2.30GHz (4C)	8GB	VMXNET3
Node 2	Intel(R) Xeon(R) Gold 5118 @ 2.30GHz (4C)	8GB	VMXNET3
Target	Intel(R) Xeon(R) Gold 5118 @ 2.30GHz (4C)	8GB	VMXNET3

Test Case	Scenario	Throughput
Case 1	Node → Node	2.99 Gbits/sec (send) - 2.99 Gbits/sec (receive)
Case 2	Egress VXLAN → Egress VXLAN	1.73 Gbits/sec (send) - 1.71 Gbits/sec (receive)
Case 3	Pod → Egress Node → Target	1.23 Gbits/sec (send) - 1.22 Gbits/sec (receive)

Key Observations¶

Physical servers provide near 90%+ efficiency even with VXLAN.
VMware VMs show significant performance degradation in VXLAN due to virtualization overhead.
Enabling checksum offload improves VXLAN performance significantly.

For best performance:

Ensure host NIC offload settings are optimized
Enable VXLAN checksum offload via Helm values
Use physical servers where high throughput is required