Skip to content

What Is LLM Studio

LLM Studio (Hydra) is a comprehensive AI model service management platform built for enterprise users. It addresses key challenges enterprises face when adopting large models, such as deployment complexity, model selection difficulties, stability issues, and potential security risks.

By offering end-to-end lifecycle services—from model deployment to operational management—the platform helps enterprises and developers efficiently integrate large model capabilities, accelerating digital transformation and intelligent innovation.

Read or Download the LLM Studio User Manual

LLM Studio architecture

Key Capabilities

  • One-Click Deployment and Simplified Operations

    Both web UI and API are supported, so mainstream models go live within minutes. Dynamic inference backends such as vLLM and SGLang support real-time scaling and multi-region deployment.

    ▶ Deploy a new model

  • Traffic Management and Stability

    Intelligent traffic policies based on weights, QPS limits, and more, plus three-level rate limiting (global, API key, tenant) to keep services stable.

    ▶ Smart routing and rate limiting

  • Distributed Inference

    Multi-node, multi-GPU serving for ultra-large models such as DeepSeek and GLM; compatible with NVIDIA, Ascend, Biren, Muxi, and other GPUs, with round-robin, random, and weighted load balancing.

    ▶ Model service overview

  • Accurate Billing and Analytics

    Token-level accounting covering mainstream large models; call volume and input/output token analytics by API key, model type, and time.

    ▶ API key management

  • Unified Multimodal Management

    The Model Plaza brings together text generation, image generation, and vision models. Send one input and compare responses from multiple models, with rich demos and integration guides.

    ▶ Model Plaza

  • Open Ecosystem

    An OpenAI-compatible API with rich call examples; mainstream clients such as Cherry Studio, Cline, LobeChat, and Bob work out of the box.

    ▶ Client practices

Workflow

The platform combines the operations management view + standard user view to achieve unified model management, trial, and rapid deployment. The model usage process consists of three steps + model file download:

flowchart LR
    EXT["External models<br/>DeepSeek · GLM · Qwen, etc."]
    subgraph OPS["Operations management view"]
        direction TB
        A["① Model Plaza management<br/>Import / create models"]
        B["② MaaS model management<br/>Configure trial runtime"]
        C["③ Model Serving management<br/>GPU / memory / framework"]
    end
    subgraph USER["Standard user view"]
        direction TB
        D["Model Plaza<br/>Browse published models"]
        E["Model Experience<br/>Online trial, no deployment"]
        G["Model Service<br/>Select a model to deploy"]
    end
    F["④ Model file download<br/>Dataset"]
    EXT -->|Import / create| A
    A -->|Auto-sync once published| D
    A --> B
    B -->|Manage| E
    C --> F
    F -->|Model files ready| G
View the original hand-drawn workflow

Platform usage process

Step Operations management view Standard user view Documentation
① Model Plaza management Import or create models in Model Plaza Management; published models sync automatically to the user view Browse published models in the Model Plaza as the entry point for trials or deployments Model Plaza Management
② MaaS model management Set a model as a MaaS model and configure its trial runtime environment Try configured models online in Model Experience without local deployment MaaS Model Management
③ Model Serving management Create deployment profiles for resources and environments, e.g. GPU/memory/runtime framework Select a model in Model Service to deploy quickly without manually filling in complex parameters Model Serving Management
④ Model file download Operations staff download model files to designated locations as the foundation for deployment — Dataset management

Quick Start

1 Deploy the platform Install LLM Studio in WS mode Go → 2 Publish models Import or create models and publish Go → 3 Configure trials Set up MaaS trial environments Go → 4 Configure serving Plan GPUs, memory, and frameworks Go → 5 Prepare model files Download models with Dataset Go →

1 Choose a model Browse and compare in Model Plaza Go → 2 Try online Chat and generate images in Model Experience Go → 3 Deploy a service One-click model deployment Go → 4 Create an API key Manage credentials and rate limits Go → 5 Connect a client Cherry Studio / Cline / LobeChat Go →

Security and Audit

The audit feature is a core part of the LLM Studio module. By recording key operation events, it keeps user behavior and system changes traceable to meet compliance, traceability, and security auditing requirements. The main audit items are:

Audit category Covered operations Event identifiers
Model management Create, update, and delete models; enable/disable the MaaS model trial feature; publish and unpublish models Model, EnabledMAASModel, PublishModel, UnPublishModel
Deployment template management Add and update model deployment templates AddDeployTemplates, UpdateDeployTemplates
Model provider management Create, update, and delete model providers ModelProvider
Workspace API key management Create and delete workspace API keys CreateWSAPIKey, DeleteWSAPIKey
Model service management Create and delete workspace model services; update replicas; perform service actions CreateWSModelServing, DeleteWSModelServing, UpdateWSModelServingReplicas, DoModelWSServingAction

These audit items cover every aspect of the model lifecycle, ensuring that every operation is traceable.

For instructions, see Security Audit Logs. For the complete list of audit items, see LLM Studio Audit Items.

Download Center Learn about d.run Token Factory Learn about d.run AI OS

Comments