Skip to content

Sriov-network-operator

Sriov-network-operator is an open-source project that aims to simplify the usage of SR-IOV technology by integrating the sriov-cni and sriov-device-plugin projects. It uses Custom Resource Definitions (CRDs) to configure and manage SR-IOV, making it easier for administrators to use.

Components

[root@controller1 ~]# kubectl  get po -n sriov-network-operator -o wide
NAME                                                              READY   STATUS      RESTARTS   AGE    IP               NODE          NOMINATED NODE   READINESS GATES
sriov-device-plugin-kw8rc                                         1/1     Running     0          62s    10.5.212.132     worker1       <none>           <none>
sriov-network-config-daemon-nhmws                                 3/3     Running     0          76m    10.5.212.132     worker1       <none>           <none>
sriov-network-operator-6955b75d8c-gmpcc                           1/1     Running     0          67s    10.233.73.233    controller1   <none>           <none>

The Sriov-network-operator consists of the following components:

  • sriov-operator: This controller monitors changes in CRs and installs/configures the sriov-cni and sriov-device-plugin components.
  • sriov-network-config-daemon: This component interacts with the nodes to enable SR-IOV on the network interfaces and configure Virtual Functions (VFs). It contains the sriov-cni binary and copies it to the /opt/cni/bin directory on the host.
  • sriov-device-plugin: This component discovers the VFs available on the host and exposes them to the kubelet.

Custom Resource Definitions (CRDs)

The Sriov-network-operator introduces two CRDs:

SriovNetworkNodeState: This CRD discovers network interfaces on the host that support SR-IOV and writes them into the status field.

Example:

apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodeState
metadata:
  creationTimestamp: "2023-06-25T07:01:04Z"
  generation: 4
  name: worker1
  namespace: sriov-network-operator
  ownerReferences:
  - apiVersion: sriovnetwork.openshift.io/v1
    blockOwnerDeletion: true
    controller: true
    kind: SriovNetworkNodePolicy
    name: default
    uid: 111e692f-cc3c-40da-aa28-de3a7f8f7c0e
  resourceVersion: "11353566"
  uid: d1bef95a-82c5-4a5c-8eb1-0ff7744eff0f
spec:
  dpConfigVersion: "11351926"
status:
  interfaces:
  - deviceID: "1017"
    driver: mlx5_core
    linkSpeed: 10000 Mb/s
    linkType: ETH
    mac: 04:3f:72:d0:d2:86
    mtu: 1500
    name: enp4s0f0np0
    pciAddress: "0000:04:00.0"
    totalvfs: 8
    vendor: 15b3
  - deviceID: "1017"
    driver: mlx5_core
    linkSpeed: 10000 Mb/s
    linkType: ETH
    mac: 04:3f:72:d0:d2:87
    mtu: 1500
    name: enp4s0f1np1
    pciAddress: "0000:04:00.1"
    totalvfs: 8
    vendor: 15b3
  syncStatus: Succeeded

The above example shows that the interfaces enp4s0f0np0 and enp4s0f1np1 on worker1 have SR-IOV capability and can be used to configure VFs for Pods.

SriovNetworkNodePolicy: This CRD is used to configure the number of VFs and install the sriov-device-plugin component.

Example:

apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"sriovnetwork.openshift.io/v1","kind":"SriovNetworkNodePolicy","metadata":{"annotations":{},"name":"policy1","namespace":"sriov-network-operator"},"spec":{"deviceType":"netdevice","nicSelector":{"pfNames":["enp4s0f0np0"],"vendor":"15b3"},"nodeSelector":{"kubernetes.io/os":"linux"},"numVfs":4,"resourceName":"sriov_netdevice"}}
  creationTimestamp: "2023-06-25T07:01:28Z"
  generation: 3
  name: policy1
  namespace: sriov-network-operator
  resourceVersion: "11350025"
  uid: b0513a9c-8c64-421d-97cc-d780fd7e8cec
spec:
  deviceType: netdevice
  nicSelector:
    pfNames:
    - enp4s0f0np0
  nodeSelector:
    kubernetes.io/hostname: 10-20-1-240  # Only effect to the node 10-20-1-240
  numVfs: 4 # Desired VFs quantity
  resourceName: sriov_netdevice
  • spec.nicSelector.pfNames: The list of PFs where the specified number of VFs will be created.
  • spec.nodeSelector: Specifies the nodes where this policy should be applied. Note: The sriov-device-plugin component will be installed on the specified nodes.

Comments