AI-ML

• 1: Simple AI Inference
• 1.1: Overview
• 1.1.1: ServiceWatch metric
• 1.2: How-to Guides
• 1.3: References
• 1.3.1: API Reference
• 1.4: Data Privacy
• 1.5: Release Note
• 2: Simple AI Training
• 2.1: Overview
• 2.1.1: Server type
• 2.1.2: ServiceWatch metric
• 2.2: How-to guides
• 2.2.1: Using Job Failover
• 2.2.2: Concurrent Checkpointing
• 2.3: Release Note
• 3: CloudML
• 3.1: Overview
• 3.2: How-to guides
• 3.2.1: Kubernetes Cluster Configuration
• 3.3: API Reference
• 3.4: CLI Reference
• 3.5: Release Note
• 4: AI&MLOps Platform
• 4.1: Overview
• 4.2: How-to guides
• 4.2.1: Cluster deployment
• 4.2.2: Kubeflow Usage Guide
• 4.3: API Reference
• 4.4: CLI Reference
• 4.5: Release Note

1 - Simple AI Inference

1.1 - Overview

Service Overview

Simple AI Inference is a serverless service that provides various global foundation models as APIs, offering public or private environments so that LLMs can be used on internal Samsung Cloud Platform resources or externally. By using Simple AI Inference, you can use multiple LLM models through the same API and improve productivity in AI application service development. It also supports compatibility with OpenAI and the LangChain SDK, enabling easy integration with existing development environments and frameworks.

Features

• Convenient LLM Model Usage: As a fully managed serverless service, you can use multiple LLM models through the same API.
• Efficient Cost Management: Costs are charged based on the actual usage of input (Input) and output (Output) tokens.
• Stable Service Provision: We provide stable services through traffic control (TPM/RTM).
• Enterprise security provided: Data is securely protected in a rigorous security environment and is not used for external model training.

Service architecture diagram

Provided Features

Simple AI Inference provides the following features.

• Check convenient LLM model

  • You can easily view the features and primary use cases of LLM models provided through the LLM model catalog.
  • You can view and test the provided LLM model directly on the console screen using PlayGround.
    Reference

    PlayGround is scheduled to be offered after September 2026.

• Shared Use of LLM Model Account: If you request a model to use in Simple AI Inference, all users within the same Account can use it.

• Serverless Service Provision : Users can request the desired model via an API and use it immediately without managing resources, and they pay only for what they use.

• Public/Private endpoint provision: Depending on the user’s inference usage pattern, you can choose to use either a Public or Private endpoint.

• Stable Service Provision: We provide a stable service environment through traffic control (TPM/RPM).

Provided model

The LLM models provided by Simple AI Inference are as follows.

Provision status by region

The regions that provide Simple AI Inference service are as follows.

Preceding Service

There are no services that need to be pre-configured before creating this service.

1.1.1 - ServiceWatch metric

Simple AI Inference sends metrics to ServiceWatch. The metrics provided by default monitoring are data collected at 5‑minute intervals.

Basic Metrics

The following are the basic metrics for the Simple AI Inference namespace. The indicators whose names are displayed in bold below are the key indicators selected among the default indicators provided by Simple AI Inference. The key metrics are used to build service dashboards that are automatically created for each service in ServiceWatch. Each metric provides guidance in the user guide on which statistical values are meaningful when querying that metric, and among the meaningful statistics, the values shown in bold are the primary statistics.

In the service dashboard or monitoring tab, you can view key metrics through primary statistical values. Or you can also view the key metrics on the monitoring tab of the Simple AI Inference detail page. You can also view the usage rate per GPU device in the ServiceWatch metrics menu.

1.2 - How-to Guides

Create Simple AI Inference

To use Simple AI Inference, you must first create an Inference. To create an inference, follow these steps.

1. Click the All Services > AI-ML > Simple AI Inference menu. 1. Go to the Service Home page of Simple AI Inference.

2. Service Home on the page, click the Create Simple AI Inference button. 2. Navigate to the Create Serverless Inference page.

3. Serverless Inference creation page, enter the information required to create the service and select detailed options.

   • In the Service Information Input area, select the options required to create the service.

     Category	Required	Detailed description
     Inference service name	Required	Enter Serverless Inference service name Enter using lowercase English letters and numbers, 3 ~ 25 characters
     Endpoint	Required	Select external access for Simple AI Inference Private: Use only private endpoint access control Private&Public: Use both private and public endpoint access control
     Private endpoint access control	Selection	Add resources within Samsung Cloud Platform and allow access only to those resources Private Access Allowed Resource: Select the resource to grant access to Click the Add button to select the resource to grant access to Select the resource to delete from the resource list, then click the Delete button to remove it If no resources are added, access is granted to all resources on subnets within the same region Can be modified after applying for a Serverless endpoint
     Public endpoint access control	Selection	Set whether to use public endpoint access control Enabled if set, you can add IPs or resources that are allowed access Public Access Allowed IP: After entering the IP range to allow access in CIDR format or as an IP address, you can add it by clicking the Add button Up to 100 entries can be added If not used, access is allowed for all IPs Can be modified after applying for a Serverless endpoint

     Table. Serverless Inference Service Information Input Items

     Caution

     If you do not use public endpoint access control or set it to the entire IP range (Any, 0.0.0.0/0), the registry can be exposed to security attacks such as external scanning and hacking.
   • In the Additional Information Input area, enter or select the required information.

     Category	Required status	Detailed description
     tag	Selection	Add Tag Up to 50 per resource can be added After clicking the Add Tag button, enter or select Key, Value values

     Table. Serverless Inference additional information input fields

4. Summary Verify the detailed information and estimated charges generated in the panel, then click the Create button.

5. When the popup notifying creation opens, click the Confirm button. 5. The creation request has been completed.

   • When creation is complete, check the created items on the Serverless Inference List page.

Check usage by LLM model

On the Service Home page of Simple AI Inference, you can view the list of LLMs and token usage per model.

1. All Services > AI-ML > Simple AI Inference Click the menu. 1. Go to the Service Home page of Simple AI Inference.
2. Check the per-model usage of LLMs in the LLM Model Usage list on the dashboard of Service Home.

   Category	Detailed description
   Model name	LLM name clicking the name moves to the Report tab on the model’s detail page
   Model type	LLM type information for each model, see Provided model
   Token usage (1 Week)	Token usage for the past week as of today

   Table. Simple AI Inference LLM model usage items

View Serverless Inference details

Follow these steps to view detailed information about Serverless Inference.

1. All Services > AI-ML > Simple AI Inference Click the menu. 1. Go to the Service Home page of Simple AI Inference.

2. On the Service Home page, click the Serverless Inference menu. 2. Serverless Inference List Go to the page.

   Item	Explanation
   Create Service	Serverless Inference can be created When the button is clicked, navigate to the Serverless Inference creation page For creation method, see Create Simple AI Inference
   Inference service name	Serverless Inference name
   Model ID	Model ID value When the Model ID is clicked, navigate to the detailed page of that model For detailed information about the model, see View model detailed information
   Model name	Model Name When clicking the Model ID, navigate to the model’s detail page For detailed information about the model, see View model detailed information
   Planned model termination date	Model’s scheduled end-of-service date
   Latency	Average response time
   Throughput	The average number of tokens the model generates per second
   Uptime	System uptime ratio that allows the system to operate normally without service interruption and handle user requests Green: 95% or higher Yellow: 80% or higher ~ less than 95% Red: less than 80%
   Service cancellation	Serverless Inference can be terminated When the button is clicked, navigate to the Serverless Inference termination page For termination instructions, see Terminate Inference

   Table. Serverless Inference list information

   Reference

   Clicking Model ID or Model name takes you to the Model Catalog’s model detail page, where you can view the model’s detailed information.

3. Serverless Inference List page, click the Inference service name to view detailed information. 3. Serverless Inference Details Go to the page.

   • Serverless Inference Detailed page consists of Details, Report, Tags, Job History tabs.

Detailed Information

Serverless Inference List page lets you view detailed information of the selected resource and modify the information if necessary.

Report

On the Serverless Inference List page, you can view the daily LLM call count and token usage for the selected resource.

Tag

On the Serverless Inference List page, you can view the tag information of the selected resource, and you can add, modify, or delete it.

Job History

You can view the operation history of the selected resource on the Serverless Inference List page.

View model detailed information

You can view the models provided by Simple AI Inference and their detailed information. To view the model details, follow these steps.

1. All Services > AI-ML > Simple AI Inference Click the menu. 1. Go to the Service Home page of Simple AI Inference.
2. On the Service Home page, click the Model Catalog menu. 2. Navigate to the Model Catalog page.
3. On the Model Catalog page, click the model whose detailed information you want to view. 3. Model Catalog Navigate to the detailed page.

   Item	Explanation
   License	Click the button to view the model’s license information.
   Overview	Basic description of the model
   Sales criteria	Model developer
   Category	Scope of model usage
   latest version	Provided version
   Release date	Model release year and date
   Model ID	Model ID information
   Maximum token	Maximum token size
   Output modelities	Model output method
   Input modelities	Model input method
   language	Model language types
   Deployment type	Model deployment method
   Token Limits	Token limit value
   Reqeust Limits	request limit

   Table. Simple AI Inference Provided Model Details

Managing API Keys

You must create and register an API key to use Simple AI Inference in Severless Inference.

Create API Key

To generate an API key, follow these steps.

1. Click the All Services > AI-ML > Simple AI Inference menu. 1. Go to the Service Home page of Simple AI Inference.

2. On the Service Home page, click the API Key menu. 2. Navigate to the API key page.

3. On the API key page, click the Create key button. 3. Create API Key Go to the detail page.

4. On the API Key Creation page, after entering the information required to generate an API key, click the Create button.

   Category	Required status	Detailed description
   Inference type	Required	Select inference type
   Expiration period	Required	Enter the expiration period of the API key permanent checking the item allows use without any time restriction
   Usage	Selection	Enter the purpose of using the API key within 128 characters

   Table. Serverless Inference Service Information Input Items

   Caution

   If you do not use public endpoint access control or set it to the entire IP range (Any, 0.0.0.0/0), the registry can be exposed to security attacks such as external scanning and hacking.

5. When the popup informing you to create an API key opens, click the Confirm button.

   • When an API key is created, it is downloaded once at the time of creation.

Check API Key

To check the API key, follow these steps.

1. All Services > AI-ML > Simple AI Inference Click the menu. 1. Go to the Service Home page of Simple AI Inference.
2. On the Service Home page, click the API Key menu. 2. Go to the API key page.

   Item	Explanation
   Authentication key	Authentication key information
   Inference type	Inference type with a registered authentication key
   Creation timestamp	Authentication key generation time
   Expiration date and time	Authentication key expiration time
   Delete	Delete the selected authentication key It becomes active when you select the authentication key to delete from the key list
   More	Change the usage status of the selected authentication key Disable When selected, the authentication key is not deleted, only its functionality is blocked
   Key generation	Create API key When the button is clicked, go to the Create API Key page Refer to Create API Key for how to create an API key

   Table. Simple AI Inference provided model detailed information

Terminate Inference

Terminate Serverless Inference

To cancel Serverless Inference, follow the steps below.

1. Click the All Services > AI-ML > Simple AI Inference menu. 1. Go to the Service Home page of Simple AI Inference.
2. On the Service Home page, click the Serverless Inference menu. 2. Serverless Inference List Go to the page.
3. On the Serverless Inference List page, click the Cancel Service button of the Serverless Inference you want to delete.
4. When the pop-up notifying service termination opens, enter the service name and click the Confirm button.

1.3 - References

References

You can view the API Reference supported by Simple AI Inference.

1.3.1 - API Reference

API Reference Overview

The API Reference supported by Simple AI Inference is as follows.

Chat Completions API

POST /v1/chat/completions

Overview

The Chat Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
      "messages": [
      {
        "role": "assistant",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What is the capital of South Korea?"
      }
    ]
}'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
      "messages": [
      {
        "role": "assistant",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What is the capital of South Korea?"
      }
    ]
}'

Response

200 OK

Error Code

Example

{
  "id": "chatcmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "chat.completion",
  "created": 1749702816,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": null,
        "content": "The capital of South Korea is Seoul.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 62,
    "completion_tokens": 8,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}

{
  "id": "chatcmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "chat.completion",
  "created": 1749702816,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": null,
        "content": "The capital of South Korea is Seoul.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 62,
    "completion_tokens": 8,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}

Prompt Guard response

If the prompt_safety_model parameter is set, the guard model checks the prompt first.

• safe: model The request is processed unchanged using the model specified in the parameter.
• unsafe: Returns a guard result in the following form and the request is terminated.

{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}

{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}

Reasoning configuration

Through the chat_template_kwargs parameter, you can control the reasoning (inference mode) configuration for each model. The default behavior and supported options vary by model.

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "zai-org/GLM-5.2",
    "messages": [
      {
        "role": "user",
        "content": "Please solve a complex math problem."
      }
    ],
    "chat_template_kwargs": {
      "reasoning_effort": "high"
    }
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "zai-org/GLM-5.2",
    "messages": [
      {
        "role": "user",
        "content": "Please solve a complex math problem."
      }
    ],
    "chat_template_kwargs": {
      "reasoning_effort": "high"
    }
  }'

Reference

• Chat Completions API vLLM document
• Chat Completions API OpenAI documentation

Completions API

POST /v1/completions

Overview

The Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
    "prompt": "What is the capital of South Korea?",
    "temperature": 0.7
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
    "prompt": "What is the capital of South Korea?",
    "temperature": 0.7
  }'

Response

200 OK

Error Code

Example

{
  "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "text_completion",
  "created": 1749702612,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
      "logprobs": null,
      "finish_reason": "length",
      "stop_reason": null,
      "prompt_logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 25,
    "completion_tokens": 16,
    "prompt_tokens_details": null
  }
}

{
  "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "text_completion",
  "created": 1749702612,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
      "logprobs": null,
      "finish_reason": "length",
      "stop_reason": null,
      "prompt_logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 25,
    "completion_tokens": 16,
    "prompt_tokens_details": null
  }
}

Reference

• Completions API vLLM documentation
• Completions API OpenAI documentation

Embedding API

POST /v1/embeddings

Overview

The Embedding API converts given text into high-dimensional vectors (embeddings), enabling support for various natural language processing (NLP) tasks such as similarity calculation between texts, clustering, and search.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/embeddings \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Embedding-8B",
    "input": "What is the capital of France?",
	"encoding_format": "float"
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/embeddings \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Embedding-8B",
    "input": "What is the capital of France?",
	"encoding_format": "float"
  }'

Response

200 OK

Error Code

Example

{
  "id":"embd-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object":"list",
  "created":1749035024,
  "model":"Qwen/Qwen3-VL-Embedding-8B",
  "data":[
    {
      "index":0,
      "object":"embedding",
      "embedding":
      [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
      -0 ....(omitted)

      -0.05706787109375,-0.0147705078125]
    }
  ],
  "usage":
  {
    "prompt_tokens":9,
    "total_tokens":9,
    "completion_tokens":0,
    "prompt_tokens_details":null
  }
}

{
  "id":"embd-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object":"list",
  "created":1749035024,
  "model":"Qwen/Qwen3-VL-Embedding-8B",
  "data":[
    {
      "index":0,
      "object":"embedding",
      "embedding":
      [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
      -0 ....(omitted)

      -0.05706787109375,-0.0147705078125]
    }
  ],
  "usage":
  {
    "prompt_tokens":9,
    "total_tokens":9,
    "completion_tokens":0,
    "prompt_tokens_details":null
  }
}

Reference

• Embedding API vLLM documentation
• Embedding API OpenAI documentation

Rerank API

POST /v2/rerank

Overview

The Rerank API applies an embedding model or a cross‑encoder model to calculate similarity scores between a single query and each document in a list, and reorders the document ranking based on these scores. Generally, the score of a sentence pair represents the similarity between the two sentences on a scale from 0 to 1.

• Embedding-based model: After converting the query and documents each into vectors, we measure the similarity between vectors (e.g., cosine similarity) and calculate a score.
• Reranker(Cross-Encoder) based model: The query and document are input as a pair to the model for evaluation.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v2/rerank \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Reranker-8B",
    "query": "What is the capital of France?",
    "documents": [
      "The capital of France is Paris.",
      "France capital city is known for the Eiffel Tower.",
      "Paris is located in the north-central part of France."
    ],
    "top_n": 2,
    "truncate_prompt_tokens": 512
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v2/rerank \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Reranker-8B",
    "query": "What is the capital of France?",
    "documents": [
      "The capital of France is Paris.",
      "France capital city is known for the Eiffel Tower.",
      "Paris is located in the north-central part of France."
    ],
    "top_n": 2,
    "truncate_prompt_tokens": 512
  }'

Response

200 OK

Error Code

Example

{
  "id": "score-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "model": "Qwen/Qwen3-VL-Reranker-8B",
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 54
  },
  "results": [
    {
      "index": 0,
      "document": {
        "text": "The capital of France is Paris.",
        "multi_modal": null
      },
      "relevance_score": 0.9237253665924072
    },
    {
      "index": 2,
      "document": {
        "text": "Paris is located in the north-central part of France.",
        "multi_modal": null
      },
      "relevance_score": 0.9181006550788879
    }
  ]
}

{
  "id": "score-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "model": "Qwen/Qwen3-VL-Reranker-8B",
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 54
  },
  "results": [
    {
      "index": 0,
      "document": {
        "text": "The capital of France is Paris.",
        "multi_modal": null
      },
      "relevance_score": 0.9237253665924072
    },
    {
      "index": 2,
      "document": {
        "text": "Paris is located in the north-central part of France.",
        "multi_modal": null
      },
      "relevance_score": 0.9181006550788879
    }
  ]
}

Reference

• Rerank API vLLM document

Responses API

POST /v1/responses

Overview

The Responses API is compatible with OpenAI’s Responses API and can be used in the OpenAI Python client. It can generate text or JSON output from text, image, or file inputs, and supports function calls and built-in tools (web search, file search, etc.).

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "What is the capital of South Korea?"
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "What is the capital of South Korea?"
  }'

Response

200 OK

Error Code

Example

{
  "id": "resp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "object": "response",
  "created_at": 1741476542,
  "status": "completed",
  "completed_at": 1741476543,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": null,
  "model": "openai/gpt-oss-120b",
  "output": [
    {
      "type": "message",
      "id": "msg_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "The capital of South Korea is Seoul.",
          "annotations": []
        }
      ]
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "store": true,
  "temperature": 1.0,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1.0,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 54,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 8,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 62
  },
  "metadata": {}
}

{
  "id": "resp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "object": "response",
  "created_at": 1741476542,
  "status": "completed",
  "completed_at": 1741476543,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": null,
  "model": "openai/gpt-oss-120b",
  "output": [
    {
      "type": "message",
      "id": "msg_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "The capital of South Korea is Seoul.",
          "annotations": []
        }
      ]
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "store": true,
  "temperature": 1.0,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1.0,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 54,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 8,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 62
  },
  "metadata": {}
}

Prompt Guard response

If the prompt_safety_model parameter is set, the guard model checks the prompt first.

• safe: model The request is processed unchanged with the model specified in the model parameter.
• unsafe: Returns a guard result in the following form and the request is terminated.

{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}

{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}

Reasoning configuration

You can control the model-specific reasoning (inference mode) configuration via the chat_template_kwargs or reasoning parameters.

• gpt-oss-120b: reasoning parameter’s effort field specifies the inference depth. * (low/medium/high, default medium)
• Other models: chat_template_kwargs parameter is used, and please refer to Chat Completions API - Reasoning configuration for configuration.

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "Please solve a complex math problem.",
    "reasoning": {
      "effort": "high"
    }
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "Please solve a complex math problem.",
    "reasoning": {
      "effort": "high"
    }
  }'

Reference

• Responses API vLLM Documentation
• Responses API OpenAI documentation
• OpenAI Developer Documentation

Tokenize API

POST /tokenize

Overview

The Tokenize API converts text to token IDs. Supports two request types: Completion (prompt-based) and Chat (messages-based). Compatible with vLLM’s Tokenize API.

Request

Context

Path Parameters

Query Parameters

Body Parameters - Common

Body Parameters - Completion method (prompt-based)

Body Parameters - Chat method (based on messages)

Example

curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "prompt": "Hello, world!"
  }'

curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "prompt": "Hello, world!"
  }'

curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {
        "role": "user",
        "content": "hi"
      }
    ]
  }'

curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {
        "role": "user",
        "content": "hi"
      }
    ]
  }'

Response

200 OK

Error Code

Example

{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": null
}

{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": null
}

{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": ["<|im_start|>", "user", "<|im_sep|>", "hi", "<|im_end|>", ""]
}

{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": ["<|im_start|>", "user", "<|im_sep|>", "hi", "<|im_end|>", ""]
}

Reference

• Tokenize API vLLM documentation

Models API

GET /v1/models

Overview

The Models API returns a list of Simple AI Inference models. Compatible with OpenAI’s Models API.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Since it is a GET request, there is no body.

Example

curl -X 'GET' \
  {Simple AI Inference endpoint}/v1/models \
  -H 'Authorization: bearer sai-xxxxxxx...'

curl -X 'GET' \
  {Simple AI Inference endpoint}/v1/models \
  -H 'Authorization: bearer sai-xxxxxxx...'

Response

200 OK

Error Code

Example

{
  "data": [
    {
      "id": "openai/gpt-oss-120b",
      "created": 1780979126,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Embedding-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Reranker-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    }
  ],
  "object": "list"
}

{
  "data": [
    {
      "id": "openai/gpt-oss-120b",
      "created": 1780979126,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Embedding-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Reranker-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    }
  ],
  "object": "list"
}

Reference

• Models API OpenAI documentation

1.4 - Data Privacy

Data Personal Information Protection and Security

Simple AI Inference operates its service with the principle that the confidentiality and security of customer data are paramount. Inference requests and response data transmitted through the service are processed only within the scope necessary to provide AI inference functionality and operate the service. This data will not be used for purposes other than providing the service.

Simple AI Inference adheres to the following principles.

• Customer request and response data are used solely for processing purposes related to service provision.
• We do not provide or share customers’ request and response content with third parties without the customer’s consent.
• Customer request and response data are not used as training data for AI model training or performance improvement.

1.5 - Release Note

Simple AI Inference

2 - Simple AI Training

2.1 - Overview

Service Overview

Simple AI Training is a fully managed AI Training service that enables data scientists and machine learning engineers to train models at scale without the burden of infrastructure management. Through the Simple AI Training service, you can easily train models and obtain results by preparing only data and training code, without separate AI infrastructure or platforms for the environment.

Features

• Eliminate the complexity of infrastructure management: In the service background of Simple AI Training, elements required for the service are automatically provisioned, and after the service ends, resources are reclaimed, so separate infrastructure operation is not needed.
• Easy and fast model training: Instead of complex command-line interface (CLI), you can run model training tasks with just a few clicks through the web console interface. * The user only needs to specify the storage location of the training script and data, as well as the GPU instance.
• Highly Secure Data Transfer: It directly integrates with cloud data storage (Object Storage, etc.) to process large volumes of data while maintaining security without concerns of external leakage.
• Stable model training: Even if infrastructure errors occur during training, the system automatically detects and recovers from failures, providing an uninterrupted training environment.
• Efficient Cost Management: For lower‑priority training, you can use idle GPUs, or Samsung Cloud Platform automatically handles Spot interruption and resumption, enabling cost‑effective training.

Service configuration diagram

Provided features

Simple AI Training provides the following features.

• Serverless environment provision: Automate all processes from infrastructure setup, data loading, model training, to result storage, allowing you to focus solely on training.
• Distributed Training and Warm Pool Support: Automatically distribute large models or massive datasets across multiple instances, and during continuous training jobs, instantly reuse instances to reduce infrastructure provisioning time.
• Custom Container Support: You can import a user’s Docker container image for training (BYOC: Bring Your Own Container).
• Model Training Availability: Supports GPU Failover and Checkpointing, enabling smooth model training.
  • GPU Failover: During training, if a GPU error occurs, the system automatically detects and recovers from the fault, preventing interruption caused by the error.
  • Curcurrent Checkpointing: By storing intermediate training results in Object Storage, training can be resumed from the checkpoint if a failure occurs.
• Safe Data Access: Integrated with the cloud’s data storage (Object Storage), it can process data while maintaining security without worrying about external leaks.
• Providing Various Pricing Plans: By offering various pricing plans, you can reduce costs and conduct efficient training.

Provided server

Simple AI Training provides the g2 server type (H100) and the g3 server type (B300). For detailed specifications of the server type, refer to 서버 타입.

Provision status by region

The regions that provide the Simple AI Training service are as follows.

Preceding Service

This is a list of services that must be pre‑configured before creating the service. For detailed information, refer to the guide provided for each service and prepare in advance.

2.1.1 - Server type

Simple AI Training is categorized according to the provided GPU type, and the GPU used for Simple AI Training is determined by the server type selected when creating a Training Job. Select the server type according to the specifications of the task you want to run in Simple AI Training. The server types supported by Simple AI Training are as follows.

at.g3v16b1

g2 server type

The g2 server type is a GPU Bare Metal Server that uses NVIDIA H100 SXM GPUs, making it suitable for large-scale high-performance AI computation.

• Provide 8 NVIDIA Hopper Architecture-based H100 GPUs
• Provides 1,979 TFLOPS of FP8 Tensor Core performance per GPU and 989 TFLOPS of FP16 Tensor Core performance.
• Supports up to 96 vCPUs and 2,048 GB of memory
• Supports up to 1,600 Gb/s NVIDIA InfiniBand RDMA network.
• Service network up to 100 Gbps
• 900 GB/s GPU P2P communication via NVSwitch within the node

g3 server type

The g3 server type is a GPU Bare Metal Server that uses the NVIDIA B300 SXM GPU, making it suitable not only for large-scale high-performance AI computation but also for LLM inference and AI deployment for generative AI.

• Provides 8 NVIDIA Blackwell Ultra Architecture-based B300 GPUs
• Provides 13.5 PFLOPS FP4 Tensor Core and 4.5 PFLOPS FP8 Tensor Core performance per GPU.
• Supports up to 128 vCPUs and 4,096 GB of memory
• Supports up to 6,400 Gb/s NVIDIA InfiniBand RDMA network.
• Service network up to 100 Gbps
• 1.8 TB/s GPU P2P communication via NVSwitch within a node

2.1.2 - ServiceWatch metric

Simple AI Training sends metrics to ServiceWatch. The metrics provided by default monitoring are data collected at 5‑minute intervals.

Basic Metrics

The following are the basic metrics for the Simple AI Training namespace. The indicators whose names are displayed in bold below are the key indicators selected from the basic indicators provided by Simple AI Training. The main metrics are used to build service dashboards that are automatically created for each service in ServiceWatch. Each metric guides users via the user guide on which statistical value is meaningful when querying the metric, and among the meaningful statistics, the values displayed in bold are the primary statistics.

In the service dashboard or monitoring tab, you can view key metrics through primary statistical values. Or you can also view the key metrics on the monitoring tab of the Simple AI Training detail page. In ServiceWatch’s metrics menu, you can also view utilization by GPU device.

2.2 - How-to guides

You can create a Training Job for the Simple AI Training service, choose the AI training method, and proceed with training.

Creating a Training Job

To use the Simple AI Training service, you must first create a Training Job. To create a Trainging Job, follow these steps.

1. All Services > AI/ML > Simple AI Training Click the menu. 1. Go to the Service Home page of Simple AI Training.

2. On the Service Home page, click the Create Training Job button. 2. Create Training Job Go to the page.

3. On the Training Job creation page, enter the information required for service creation and select detailed options.

   • Select the required information related to the Training Job in the Required Information Input area.

     Category	required status	Detailed description
     Learning type	Required	Select training mode On-Demand Training: Securely conduct training by preempting a server at the desired time Concurrent Checkpointing feature applied to support efficient checkpoint storage Spot Training: Conduct cost‑effective training for lower‑priority jobs using idle GPUs Concurrent Checkpointing and Mixed workload features applied to support training continuity (automatic pause and resume) For detailed information on the Concurrent Checkpointing feature, see the Concurrent Checkpointing 개요 reference
     Training Job name	Required	Enter the Training Job name Enter using lowercase English letters, numbers, and special characters (-.) within 3 ~ 63 characters The name must start and end with a lowercase English letter or number
     Distributed Framework	Required	Select version of the distributed Framework PyTorch, DeepSpeed selectable

     Table. Required input fields for Training Job
   • Select the options required to create a Training Job in the Service Information Input area.

     Category	required status	Detailed description
     Job Failover	Selection	Select whether to use the Job Failover feature Spot Training cannot be used with Job Failover For detailed information about the Job Failover feature, see Job Failover 사용하기
     Resource allocation	Required	Select the number of GPUs and memory size to use for training
     Number of nodes	Required	Set the scale of distributed training Distributed training is possible from 2 nodes onward
     Shared Memory	Required	Set the memory to be shared between processes for distributed training and distributed data processing.

     Table. Training Job service information input items
   • In the AI Training Image Information Input area, select the options required to create the service.

     Category	required status	Detailed description
     AI Training Image URL	Required	Enter the user’s container registry (SCR, Docker Hub, etc.) address
     User ID	Selection	User ID of the image repository
     Password	Required	Password for the image repository

     Table. Training Job AI Training Image Information Input Items
   • Training Command and Volume Information Input area, please input or select the required information.

     Category	required status	Detailed description
     Storage connection	Selection	Select whether to use an additional volume When used, enter the additional volume mount path and training script URL File Storage Volume Mount Path: Data path to use when connecting to File Storage (e.g., /root) Import Training Script (Object Storage): Enter the script URL when connecting to Object Storage
     Command	Required	Enter command information within 3 to 1,024.

     Table. Training Job AI Training Image Information Input Items
   • Additional Information Input area, please enter or select the required information.

     Category	required status	Detailed description
     tag	Selection	Add Tag Up to 50 can be added per resource After clicking the Add Tag button, enter or select Key, Value values

     Table. Training Job training command and volume information input items

4. Summary Check the detailed information generated in the panel, and click the Create button.

5. When the popup notifying creation opens, click the Confirm button.

   • When creation is complete, check the created resources on the Training Job List page.

Check detailed information of Training Job

You can view and edit the complete resource list and detailed information of the Training Job service. To view the details of a Training Job, follow these steps.

1. Click the All Services > AI/ML > Simple AI Training menu. 1. Go to the Service Home page of Simple AI Training.
2. On the Service Home page, click the Training Job menu. 2. Navigate to the Training Job List page.
3. On the Training Job List page, click the resource to view detailed information. 3. Navigate to the Training Job Details page.
   • Training Job Details page consists of the Details, Monitoring, Logs, Tags tabs.

     Category	Detailed description
     Service status	CloudML status Creating: Creating Deployed: Created / operating normally Updating: Updating settings Terminating: Deleting Error: Error occurred
     Delete Training Job	Button to cancel the service

     Table. Training Job detail page items

Detailed Information

Training Job List page allows you to view detailed information of the selected resource.

Monitoring

Training Job List page allows you to view the monitoring information of the selected resource.

log

On the Training Job List page, you can view the log information of the selected resource.

Tag

Training Job list page lets you view the tag information of the selected resource, and you can add, modify, or delete it.

Checking Training Job logs

You can view the logs of a Training Job in the ServiceWatch service. To view the logs of the Training Job, follow these steps.

1. Click the All Services > AI/ML > Simple AI Training menu. 1. Go to the Service Home page of Simple AI Training.
2. On the Service Home page, click the Training Job menu. 2. Go to the Training Job List page.
3. Training Job list page: select the resource to view logs. 3. Navigate to the Training Job Details page.
4. On the Training Job Details page, click the Log tab. 4. The log information for this job is displayed.
5. Click the name below the Training Job name in the log information. 5. Navigate to the ServiceWatch Log Group Details page.
6. Log Group Details page, click the Log Stream tab. 6. The list of log streams is displayed.
7. Click the log stream name to verify (example: master). 7. The logs of the stream are displayed in chronological order.
   • When a failover occurs, the logs of the training that was interrupted due to the failure and the logs of the training that was restarted after reallocation are recorded together in a single stream.
   • If the initialization log displayed at the start of training (e.g., Starting dataset initialization) appears more than once, you can confirm that the training was rescheduled and restarted.

Delete Training Job

You can delete unused Training Jobs. To delete a Training Job, follow these steps.

1. All Services > AI/ML > Simple AI Training Click the menu. 1. Go to the Service Home page of Simple AI Training.
2. On the Service Home page, click the Training Job menu. 2. Go to the Training Job List page.
3. On the Training Job List page, select the resources to delete, then click the Delete button at the top of the list.
   • Click the resource to delete, go to the Training Job Details page, and you can also delete it individually.
4. When a pop-up notifying deletion opens, click the Confirm button.

Using the Training Workspace

You can use the Simple AI Training service by using the Training Workspace.

Create Training Workspace

To create a Training Workspace, follow the steps below.

1. Click the All Services > AI/ML > Simple AI Training menu. 1. Go to the Service Home page of Simple AI Training.

2. Click the Training Workspace menu on the Service Home page. 2. Go to the Training Workspace List page.

3. On the Training Workspace List page, click the Create Service button. 3. Go to the Create Training Workspace page.

4. Enter the information required to create the service and select detailed options.

   • In the Service Information Input area, select the options required to create a Training Workspace.

     Category	required status	Detailed description
     Training Workspace name	Required	Enter the Training Workspace name Enter using lowercase English letters, numbers, and special characters (-.) within 63 characters
     Resource allocation	Required	Select the number of GPUs and memory size to use for training
     Number of nodes	Required	Set the scale of distributed training Distributed training is possible from 2 nodes onward
     Contract period	Required	Select the service usage agreement period

     Table. Training Workspace service information input items

5. Summary Check the detailed information and estimated billing amount generated in the panel, and click the Create button.

6. When the popup notifying creation opens, click the Confirm button.

   • When creation is complete, check the resources you created on the Training Workspace List page.

Edit Training Workspace

You can modify the number of nodes in the Training Workspace. To modify the Training Workspace, follow these steps.

1. Click the All Services > AI/ML > Simple AI Training menu. 1. Go to the Service Home page of Simple AI Training.
2. On the Service Home page, click the Training Workspace menu. 2. Navigate to the Training Workspace List page.
3. On the Training Workspace List page, click the More > Edit button of the resource you want to modify. 3. Go to the Training Workspace Edit page.
4. On the Training Workspace Edit page, check and modify the node count.
5. When the edit is complete, click the Confirm button.

Delete Training Workspace

You can delete an unused Training Workspace. To delete the Training Workspace, follow these steps.

1. All Services > AI/ML > Simple AI Training Click the menu. 1. Go to the Service Home page of Simple AI Training.
2. On the Service Home page, click the Training Workspace menu. 2. Navigate to the Training Workspace List page.
3. On the Training Workspace List page, select the resource to delete, then click the Cancel Service button at the top of the list.
   • You can also delete the resource individually by clicking the More > Service Cancellation button.
4. When a pop-up indicating deletion opens, click the Confirm button.

2.2.1 - Using Job Failover

Job Failover is a feature that automatically reallocates to normal resources to continue training, ensuring that training is not interrupted even if hardware failures such as GPUs occur while training with On-Demand Training. By using the Job Failover feature, you can ensure training continuity without the user having to manually detect failures or recreate the job when system errors or hardware faults are detected.

Job Failover Overview

During a Training Job, various hardware issues such as GPU errors or node failures can occur on the node where the training is deployed. When you use the Job Failover feature, the system automatically avoids the failed node when such hardware failures are detected, reassigns the Training Job to healthy resources, and resumes training. The main operation of Job Failover is as follows.

• Fault Detection: Continuously monitors the GPU and hardware status of nodes running training to detect anomalies.
• Automatic Reallocation: If a hardware failure is detected, the system excludes the failed node and reallocates the Training Job to healthy nodes, restarting the training.
• Unnecessary relocation prevention: User code errors, configuration errors, and similar issues that are not hardware failures are excluded from the Failover target.
• Retry Count Limit: Failover is performed only within the specified maximum number of attempts (3), and if this is exceeded, training ends in failure.

Check error cause

When a training interruption occurs, the system checks the node’s GPU and hardware status to determine whether the cause is a hardware failure or a non‑hardware issue such as a user application or configuration. Whether to perform failover is determined based on this judgment result.

If it is judged to be a hardware failure

If a physical or hardware-level error that prevents normal GPU usage is detected, it is considered a hardware failure and a failover is performed. This error prevents further training on the affected node, so the failed node is excluded and the workload is reallocated to healthy resources to resume training. Examples of errors that are considered hardware failures are as follows.

If it is judged to be an internal error

If the cause of the training interruption is determined to be user application code or configuration issues rather than a hardware failure, there is a high likelihood that the same problem will recur even after redeployment, so we reject the Failover and terminate in an internalerror state. Examples of errors that are considered internal errors are as follows.