Deploy Second Scheduler scheduler-plugins in a Cluster¶

This page describes how to deploy a second scheduler-plugins in a cluster.

Why do we need scheduler-plugins?¶

The cluster created through the platform will install the native K8s scheduler-plugin, but the native scheduler-plugin has many limitations:

The native scheduler-plugin cannot meet scheduling requirements, so you can use either CoScheduling, CapacityScheduling or other types of scheduler-plugins.
In special scenarios, a new scheduler-plugin is needed to complete scheduling tasks without affecting the process of the native scheduler-plugin.
Distinguish scheduler-plugins with different functionalities and achieve different scheduling scenarios by switching scheduler-plugin names.

This page takes the scenario of using the vgpu scheduler-plugin while combining the coscheduling plugin capability of scheduler-plugins as an example to introduce how to install and use scheduler-plugins.

Installing scheduler-plugins¶

Prerequisites¶

kubean is a new feature introduced in v0.13.0, please ensure that your version is v0.13.0 or higher.
The installation version of scheduler-plugins is v0.27.8, please ensure that the cluster version is compatible with it. Refer to the document Compatibility Matrix.

Installation Process¶

Add the scheduler-plugins parameter in Create Cluster -> Advanced Settings -> Custom Parameters.
```
scheduler_plugins_enabled:true
scheduler_plugins_plugin_config:
  - name: Coscheduling
    args:
      permitWaitingTimeSeconds: 10 # default is 60
```
Parameters:
- scheduler_plugins_enabled Set to true to enable the scheduler-plugins capability.
- You can enable or disable certain plugins by setting the scheduler_plugins_enabled_plugins or scheduler_plugins_disabled_plugins options. See K8s Official Plugin Names for reference.
- If you need to set parameters for custom plugins, please configure scheduler_plugins_plugin_config, for example: set the permitWaitingTimeoutSeconds parameter for coscheduling. See K8s Official Plugin Configuration for reference.
After successful cluster creation, the system will automatically install the scheduler-plugins and controller component loads. You can check the workload status in the proper cluster's deployment.

Using scheduler-plugins¶

Here is an example of how to use scheduler-plugins by demonstrating a scenario where the vgpu scheduler is used in combination with the coscheduling plugin capability of scheduler-plugins.

Install vgpu in the Helm Charts and set the values.yaml parameters.
- schedulerName: scheduler-plugins-scheduler: This is the scheduler name for scheduler-plugins installed by kubean, and currently cannot be modified.
- scheduler.kubeScheduler.enabled: false: Do not install kube-scheduler and use vgpu-scheduler as a separate extender.

Extend vgpu-scheduler on scheduler-plugins.

[root@master01 charts]# kubectl get cm -n scheduler-plugins scheduler-config -ojsonpath="{.data.scheduler-config\.yaml}"

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
leaderElection:
  leaderElect: false
profiles:
  # Compose all plugins in one profile
  - schedulerName: scheduler-plugins-scheduler
    plugins:
      multiPoint:
        enabled:
          - name: Coscheduling
          - name: CapacityScheduling
          - name: NodeResourceTopologyMatch
          - name: NodeResourcesAllocatable
        disabled:
          - name: PrioritySort
pluginConfig:
  - args:
      permitWaitingTimeSeconds: 10
    name: Coscheduling

Modify configmap of scheduler-config for scheduler-plugins:

[root@master01 charts]# kubectl get cm -n scheduler-plugins scheduler-config -ojsonpath="{.data.scheduler-config\.yaml}"

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
leaderElection:
  leaderElect: false
profiles:
  # Compose all plugins in one profile
  - schedulerName: scheduler-plugins-scheduler
    plugins:
      multiPoint:
        enabled:
          - name: Coscheduling
          - name: CapacityScheduling
          - name: NodeResourceTopologyMatch
          - name: NodeResourcesAllocatable
        disabled:
          - name: PrioritySort
pluginConfig:
  - args:
      permitWaitingTimeSeconds: 10
    name: Coscheduling
extenders:
  - urlPrefix: "${urlPrefix}"
    filterVerb: filter
    bindVerb: bind
    nodeCacheCapable: true
    ignorable: true
    httpTimeout: 30s
    weight: 1
    enableHTTPS: true
    tlsConfig:
      insecure: true
    managedResources:
      - name: nvidia.com/vgpu
        ignoredByScheduler: true
      - name: nvidia.com/gpumem
        ignoredByScheduler: true
      - name: nvidia.com/gpucores
        ignoredByScheduler: true
      - name: nvidia.com/gpumem-percentage
        ignoredByScheduler: true
      - name: nvidia.com/priority
        ignoredByScheduler: true
      - name: cambricon.com/mlunum
        ignoredByScheduler: true

After installing vgpu-scheduler, the system will automatically create a service (svc), and the urlPrefix specifies the URL of the svc.
Note
- The svc refers to the pod service load. You can use the following command in the namespace where the nvidia-vgpu plugin is installed to get the external access information for port 443.
```
kubectl get svc -n ${namespace}
```
- The urlPrefix format is https://${ip address}:${port}
Restart the scheduler pod of scheduler-plugins to load the new configuration file.

Note

When creating a vgpu application, you do not need to specify the name of a scheduler-plugin. The vgpu-scheduler webhook will automatically change the scheduler's name to "scheduler-plugins-scheduler" without manual specification.