Skip to content

Deploying a New Model

Publish the selected model on a target cluster as a callable model service, filling in deployment parameters such as resources, runtime, and instances as needed. You can initiate a deployment from the Model Gallery, Model Management, or the left navigation Model Serving - Deploy New Model. The following sections describe the steps and the meaning of each parameter.

deploy

Prerequisites

  • The target cluster and namespace are available.
  • To deploy a custom model, you have completed the model metadata, deployment template, or model weight file configuration in Create Model.

Steps

  1. Enter LLM Studio, and in the left navigation bar, click Model Serving.
  2. On the model service list page, click Deploy New Model (or click the Deploy button on the Model Gallery or Model Management page).

  3. Fill in the deployment parameters as needed and click Confirm.

    Parameter Descriptions

    Parameter Constraints / Description Remarks
    Model Source Select the model source: Model Gallery or a custom model A custom model must have been created in Model Management
    Model Selection Select the model to deploy (e.g., DeepSeek-R1). You can quickly filter models through the dropdown menu Affects model capabilities, inference quality, and resource consumption
    Model Service Name Specify a name for the model service deployed this time
    Length: 2–64 characters
    Characters: Only lowercase letters, numbers, and hyphens (-) are allowed
    Rule: Must start and end with a lowercase letter or number    		 Examples: text-gen-service, model-01
    Instance Count Configure the number of instances to deploy
    Note: More instances provide stronger concurrency but also higher cost    		 Default: 1
    Deployment Cluster Select the cluster to deploy the model service to It is recommended to prefer a cluster that is physically closer to reduce latency
    Namespace Specify the target namespace for the model service deployment Bound to the cluster
    Deployment Template Select a configured deployment template Supports quick reuse of resource and runtime configurations
    Model Weight File Select a configured model weight file Recommended when deploying a custom model
    Runtime Framework Select the inference runtime framework Supports selection based on the model and scenario
    Distributed Inference When enabled, you can configure the number of deployment nodes per instance Suitable for scenarios where a single node has insufficient resources
    Queue Scheduling You can configure the scheduling policy, priority, and more Supports priority and topology-aware scheduling capabilities

    deploy

After Deployment

After the model is deployed successfully, you can:

Comments