Simple AI Inference

• 1: Overview
• 1.1: ServiceWatch metric
• 2: How-to Guides
• 3: References
• 3.1: API Reference
• 4: Data Privacy
• 5: Release Note

1 - Overview

Service Overview

Simple AI Inference is a serverless service that provides various global foundation models as APIs, offering public or private environments so that LLMs can be used on internal Samsung Cloud Platform resources or externally. By using Simple AI Inference, you can use multiple LLM models through the same API and improve productivity in AI application service development. It also supports compatibility with OpenAI and the LangChain SDK, enabling easy integration with existing development environments and frameworks.

Features

• Convenient LLM Model Usage: As a fully managed serverless service, you can use multiple LLM models through the same API.
• Efficient Cost Management: Costs are charged based on the actual usage of input (Input) and output (Output) tokens.
• Stable Service Provision: We provide stable services through traffic control (TPM/RTM).
• Enterprise security provided: Data is securely protected in a rigorous security environment and is not used for external model training.

Service architecture diagram

Provided Features

Simple AI Inference provides the following features.

• Check convenient LLM model

  • You can easily view the features and primary use cases of LLM models provided through the LLM model catalog.
  • You can view and test the provided LLM model directly on the console screen using PlayGround.
    Reference

    PlayGround is scheduled to be offered after September 2026.

• Shared Use of LLM Model Account: If you request a model to use in Simple AI Inference, all users within the same Account can use it.

• Serverless Service Provision : Users can request the desired model via an API and use it immediately without managing resources, and they pay only for what they use.

• Public/Private endpoint provision: Depending on the user’s inference usage pattern, you can choose to use either a Public or Private endpoint.

• Stable Service Provision: We provide a stable service environment through traffic control (TPM/RPM).

Provided model

The LLM models provided by Simple AI Inference are as follows.

Provision status by region

The regions that provide Simple AI Inference service are as follows.

Preceding Service

There are no services that need to be pre-configured before creating this service.

1.1 - ServiceWatch metric

Simple AI Inference sends metrics to ServiceWatch. The metrics provided by default monitoring are data collected at 5‑minute intervals.

Basic Metrics

The following are the basic metrics for the Simple AI Inference namespace. The indicators whose names are displayed in bold below are the key indicators selected among the default indicators provided by Simple AI Inference. The key metrics are used to build service dashboards that are automatically created for each service in ServiceWatch. Each metric provides guidance in the user guide on which statistical values are meaningful when querying that metric, and among the meaningful statistics, the values shown in bold are the primary statistics.

In the service dashboard or monitoring tab, you can view key metrics through primary statistical values. Or you can also view the key metrics on the monitoring tab of the Simple AI Inference detail page. You can also view the usage rate per GPU device in the ServiceWatch metrics menu.

2 - How-to Guides

Create Simple AI Inference

To use Simple AI Inference, you must first create an Inference. To create an inference, follow these steps.

1. Click the All Services > AI-ML > Simple AI Inference menu. 1. Go to the Service Home page of Simple AI Inference.

2. Service Home on the page, click the Create Simple AI Inference button. 2. Navigate to the Create Serverless Inference page.

3. Serverless Inference creation page, enter the information required to create the service and select detailed options.

   • In the Service Information Input area, select the options required to create the service.

     Category	Required	Detailed description
     Inference service name	Required	Enter Serverless Inference service name Enter using lowercase English letters and numbers, 3 ~ 25 characters
     Endpoint	Required	Select external access for Simple AI Inference Private: Use only private endpoint access control Private&Public: Use both private and public endpoint access control
     Private endpoint access control	Selection	Add resources within Samsung Cloud Platform and allow access only to those resources Private Access Allowed Resource: Select the resource to grant access to Click the Add button to select the resource to grant access to Select the resource to delete from the resource list, then click the Delete button to remove it If no resources are added, access is granted to all resources on subnets within the same region Can be modified after applying for a Serverless endpoint
     Public endpoint access control	Selection	Set whether to use public endpoint access control Enabled if set, you can add IPs or resources that are allowed access Public Access Allowed IP: After entering the IP range to allow access in CIDR format or as an IP address, you can add it by clicking the Add button Up to 100 entries can be added If not used, access is allowed for all IPs Can be modified after applying for a Serverless endpoint

     Table. Serverless Inference Service Information Input Items

     Caution

     If you do not use public endpoint access control or set it to the entire IP range (Any, 0.0.0.0/0), the registry can be exposed to security attacks such as external scanning and hacking.
   • In the Additional Information Input area, enter or select the required information.

     Category	Required status	Detailed description
     tag	Selection	Add Tag Up to 50 per resource can be added After clicking the Add Tag button, enter or select Key, Value values

     Table. Serverless Inference additional information input fields

4. Summary Verify the detailed information and estimated charges generated in the panel, then click the Create button.

5. When the popup notifying creation opens, click the Confirm button. 5. The creation request has been completed.

   • When creation is complete, check the created items on the Serverless Inference List page.

Check usage by LLM model

On the Service Home page of Simple AI Inference, you can view the list of LLMs and token usage per model.

1. All Services > AI-ML > Simple AI Inference Click the menu. 1. Go to the Service Home page of Simple AI Inference.
2. Check the per-model usage of LLMs in the LLM Model Usage list on the dashboard of Service Home.

   Category	Detailed description
   Model name	LLM name clicking the name moves to the Report tab on the model’s detail page
   Model type	LLM type information for each model, see Provided model
   Token usage (1 Week)	Token usage for the past week as of today

   Table. Simple AI Inference LLM model usage items

View Serverless Inference details

Follow these steps to view detailed information about Serverless Inference.

1. All Services > AI-ML > Simple AI Inference Click the menu. 1. Go to the Service Home page of Simple AI Inference.

2. On the Service Home page, click the Serverless Inference menu. 2. Serverless Inference List Go to the page.

   Item	Explanation
   Create Service	Serverless Inference can be created When the button is clicked, navigate to the Serverless Inference creation page For creation method, see Create Simple AI Inference
   Inference service name	Serverless Inference name
   Model ID	Model ID value When the Model ID is clicked, navigate to the detailed page of that model For detailed information about the model, see View model detailed information
   Model name	Model Name When clicking the Model ID, navigate to the model’s detail page For detailed information about the model, see View model detailed information
   Planned model termination date	Model’s scheduled end-of-service date
   Latency	Average response time
   Throughput	The average number of tokens the model generates per second
   Uptime	System uptime ratio that allows the system to operate normally without service interruption and handle user requests Green: 95% or higher Yellow: 80% or higher ~ less than 95% Red: less than 80%
   Service cancellation	Serverless Inference can be terminated When the button is clicked, navigate to the Serverless Inference termination page For termination instructions, see Terminate Inference

   Table. Serverless Inference list information

   Reference

   Clicking Model ID or Model name takes you to the Model Catalog’s model detail page, where you can view the model’s detailed information.

3. Serverless Inference List page, click the Inference service name to view detailed information. 3. Serverless Inference Details Go to the page.

   • Serverless Inference Detailed page consists of Details, Report, Tags, Job History tabs.

Detailed Information

Serverless Inference List page lets you view detailed information of the selected resource and modify the information if necessary.

Report

On the Serverless Inference List page, you can view the daily LLM call count and token usage for the selected resource.

Tag

On the Serverless Inference List page, you can view the tag information of the selected resource, and you can add, modify, or delete it.

Job History

You can view the operation history of the selected resource on the Serverless Inference List page.

View model detailed information

You can view the models provided by Simple AI Inference and their detailed information. To view the model details, follow these steps.

1. All Services > AI-ML > Simple AI Inference Click the menu. 1. Go to the Service Home page of Simple AI Inference.
2. On the Service Home page, click the Model Catalog menu. 2. Navigate to the Model Catalog page.
3. On the Model Catalog page, click the model whose detailed information you want to view. 3. Model Catalog Navigate to the detailed page.

   Item	Explanation
   License	Click the button to view the model’s license information.
   Overview	Basic description of the model
   Sales criteria	Model developer
   Category	Scope of model usage
   latest version	Provided version
   Release date	Model release year and date
   Model ID	Model ID information
   Maximum token	Maximum token size
   Output modelities	Model output method
   Input modelities	Model input method
   language	Model language types
   Deployment type	Model deployment method
   Token Limits	Token limit value
   Reqeust Limits	request limit

   Table. Simple AI Inference Provided Model Details

Managing API Keys

You must create and register an API key to use Simple AI Inference in Severless Inference.

Create API Key

To generate an API key, follow these steps.

1. Click the All Services > AI-ML > Simple AI Inference menu. 1. Go to the Service Home page of Simple AI Inference.

2. On the Service Home page, click the API Key menu. 2. Navigate to the API key page.

3. On the API key page, click the Create key button. 3. Create API Key Go to the detail page.

4. On the API Key Creation page, after entering the information required to generate an API key, click the Create button.

   Category	Required status	Detailed description
   Inference type	Required	Select inference type
   Expiration period	Required	Enter the expiration period of the API key permanent checking the item allows use without any time restriction
   Usage	Selection	Enter the purpose of using the API key within 128 characters

   Table. Serverless Inference Service Information Input Items

   Caution

   If you do not use public endpoint access control or set it to the entire IP range (Any, 0.0.0.0/0), the registry can be exposed to security attacks such as external scanning and hacking.

5. When the popup informing you to create an API key opens, click the Confirm button.

   • When an API key is created, it is downloaded once at the time of creation.

Check API Key

To check the API key, follow these steps.

1. All Services > AI-ML > Simple AI Inference Click the menu. 1. Go to the Service Home page of Simple AI Inference.
2. On the Service Home page, click the API Key menu. 2. Go to the API key page.

   Item	Explanation
   Authentication key	Authentication key information
   Inference type	Inference type with a registered authentication key
   Creation timestamp	Authentication key generation time
   Expiration date and time	Authentication key expiration time
   Delete	Delete the selected authentication key It becomes active when you select the authentication key to delete from the key list
   More	Change the usage status of the selected authentication key Disable When selected, the authentication key is not deleted, only its functionality is blocked
   Key generation	Create API key When the button is clicked, go to the Create API Key page Refer to Create API Key for how to create an API key

   Table. Simple AI Inference provided model detailed information

Terminate Inference

Terminate Serverless Inference

To cancel Serverless Inference, follow the steps below.

1. Click the All Services > AI-ML > Simple AI Inference menu. 1. Go to the Service Home page of Simple AI Inference.
2. On the Service Home page, click the Serverless Inference menu. 2. Serverless Inference List Go to the page.
3. On the Serverless Inference List page, click the Cancel Service button of the Serverless Inference you want to delete.
4. When the pop-up notifying service termination opens, enter the service name and click the Confirm button.

3 - References

References

You can view the API Reference supported by Simple AI Inference.

3.1 - API Reference

API Reference Overview

The API Reference supported by Simple AI Inference is as follows.

Chat Completions API

POST /v1/chat/completions

Overview

The Chat Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
      "messages": [
      {
        "role": "assistant",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What is the capital of South Korea?"
      }
    ]
}'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
      "messages": [
      {
        "role": "assistant",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What is the capital of South Korea?"
      }
    ]
}'

Response

200 OK

Error Code

Example

{
  "id": "chatcmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "chat.completion",
  "created": 1749702816,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": null,
        "content": "The capital of South Korea is Seoul.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 62,
    "completion_tokens": 8,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}

{
  "id": "chatcmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "chat.completion",
  "created": 1749702816,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": null,
        "content": "The capital of South Korea is Seoul.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 62,
    "completion_tokens": 8,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}

Prompt Guard response

If the prompt_safety_model parameter is set, the guard model checks the prompt first.

• safe: model The request is processed unchanged using the model specified in the parameter.
• unsafe: Returns a guard result in the following form and the request is terminated.

{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}

{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}

Reasoning configuration

Through the chat_template_kwargs parameter, you can control the reasoning (inference mode) configuration for each model. The default behavior and supported options vary by model.

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "zai-org/GLM-5.2",
    "messages": [
      {
        "role": "user",
        "content": "Please solve a complex math problem."
      }
    ],
    "chat_template_kwargs": {
      "reasoning_effort": "high"
    }
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "zai-org/GLM-5.2",
    "messages": [
      {
        "role": "user",
        "content": "Please solve a complex math problem."
      }
    ],
    "chat_template_kwargs": {
      "reasoning_effort": "high"
    }
  }'

Reference

• Chat Completions API vLLM document
• Chat Completions API OpenAI documentation

Completions API

POST /v1/completions

Overview

The Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
    "prompt": "What is the capital of South Korea?",
    "temperature": 0.7
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
    "prompt": "What is the capital of South Korea?",
    "temperature": 0.7
  }'

Response

200 OK

Error Code

Example

{
  "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "text_completion",
  "created": 1749702612,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
      "logprobs": null,
      "finish_reason": "length",
      "stop_reason": null,
      "prompt_logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 25,
    "completion_tokens": 16,
    "prompt_tokens_details": null
  }
}

{
  "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "text_completion",
  "created": 1749702612,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
      "logprobs": null,
      "finish_reason": "length",
      "stop_reason": null,
      "prompt_logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 25,
    "completion_tokens": 16,
    "prompt_tokens_details": null
  }
}

Reference

• Completions API vLLM documentation
• Completions API OpenAI documentation

Embedding API

POST /v1/embeddings

Overview

The Embedding API converts given text into high-dimensional vectors (embeddings), enabling support for various natural language processing (NLP) tasks such as similarity calculation between texts, clustering, and search.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/embeddings \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Embedding-8B",
    "input": "What is the capital of France?",
	"encoding_format": "float"
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/embeddings \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Embedding-8B",
    "input": "What is the capital of France?",
	"encoding_format": "float"
  }'

Response

200 OK

Error Code

Example

{
  "id":"embd-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object":"list",
  "created":1749035024,
  "model":"Qwen/Qwen3-VL-Embedding-8B",
  "data":[
    {
      "index":0,
      "object":"embedding",
      "embedding":
      [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
      -0 ....(omitted)

      -0.05706787109375,-0.0147705078125]
    }
  ],
  "usage":
  {
    "prompt_tokens":9,
    "total_tokens":9,
    "completion_tokens":0,
    "prompt_tokens_details":null
  }
}

{
  "id":"embd-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object":"list",
  "created":1749035024,
  "model":"Qwen/Qwen3-VL-Embedding-8B",
  "data":[
    {
      "index":0,
      "object":"embedding",
      "embedding":
      [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
      -0 ....(omitted)

      -0.05706787109375,-0.0147705078125]
    }
  ],
  "usage":
  {
    "prompt_tokens":9,
    "total_tokens":9,
    "completion_tokens":0,
    "prompt_tokens_details":null
  }
}

Reference

• Embedding API vLLM documentation
• Embedding API OpenAI documentation

Rerank API

POST /v2/rerank

Overview

The Rerank API applies an embedding model or a cross‑encoder model to calculate similarity scores between a single query and each document in a list, and reorders the document ranking based on these scores. Generally, the score of a sentence pair represents the similarity between the two sentences on a scale from 0 to 1.

• Embedding-based model: After converting the query and documents each into vectors, we measure the similarity between vectors (e.g., cosine similarity) and calculate a score.
• Reranker(Cross-Encoder) based model: The query and document are input as a pair to the model for evaluation.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v2/rerank \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Reranker-8B",
    "query": "What is the capital of France?",
    "documents": [
      "The capital of France is Paris.",
      "France capital city is known for the Eiffel Tower.",
      "Paris is located in the north-central part of France."
    ],
    "top_n": 2,
    "truncate_prompt_tokens": 512
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v2/rerank \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Reranker-8B",
    "query": "What is the capital of France?",
    "documents": [
      "The capital of France is Paris.",
      "France capital city is known for the Eiffel Tower.",
      "Paris is located in the north-central part of France."
    ],
    "top_n": 2,
    "truncate_prompt_tokens": 512
  }'

Response

200 OK

Error Code

Example

{
  "id": "score-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "model": "Qwen/Qwen3-VL-Reranker-8B",
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 54
  },
  "results": [
    {
      "index": 0,
      "document": {
        "text": "The capital of France is Paris.",
        "multi_modal": null
      },
      "relevance_score": 0.9237253665924072
    },
    {
      "index": 2,
      "document": {
        "text": "Paris is located in the north-central part of France.",
        "multi_modal": null
      },
      "relevance_score": 0.9181006550788879
    }
  ]
}

{
  "id": "score-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "model": "Qwen/Qwen3-VL-Reranker-8B",
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 54
  },
  "results": [
    {
      "index": 0,
      "document": {
        "text": "The capital of France is Paris.",
        "multi_modal": null
      },
      "relevance_score": 0.9237253665924072
    },
    {
      "index": 2,
      "document": {
        "text": "Paris is located in the north-central part of France.",
        "multi_modal": null
      },
      "relevance_score": 0.9181006550788879
    }
  ]
}

Reference

• Rerank API vLLM document

Responses API

POST /v1/responses

Overview

The Responses API is compatible with OpenAI’s Responses API and can be used in the OpenAI Python client. It can generate text or JSON output from text, image, or file inputs, and supports function calls and built-in tools (web search, file search, etc.).

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "What is the capital of South Korea?"
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "What is the capital of South Korea?"
  }'

Response

200 OK

Error Code

Example

{
  "id": "resp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "object": "response",
  "created_at": 1741476542,
  "status": "completed",
  "completed_at": 1741476543,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": null,
  "model": "openai/gpt-oss-120b",
  "output": [
    {
      "type": "message",
      "id": "msg_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "The capital of South Korea is Seoul.",
          "annotations": []
        }
      ]
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "store": true,
  "temperature": 1.0,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1.0,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 54,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 8,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 62
  },
  "metadata": {}
}

{
  "id": "resp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "object": "response",
  "created_at": 1741476542,
  "status": "completed",
  "completed_at": 1741476543,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": null,
  "model": "openai/gpt-oss-120b",
  "output": [
    {
      "type": "message",
      "id": "msg_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "The capital of South Korea is Seoul.",
          "annotations": []
        }
      ]
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "store": true,
  "temperature": 1.0,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1.0,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 54,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 8,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 62
  },
  "metadata": {}
}

Prompt Guard response

If the prompt_safety_model parameter is set, the guard model checks the prompt first.

• safe: model The request is processed unchanged with the model specified in the model parameter.
• unsafe: Returns a guard result in the following form and the request is terminated.

{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}

{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}

Reasoning configuration

You can control the model-specific reasoning (inference mode) configuration via the chat_template_kwargs or reasoning parameters.

• gpt-oss-120b: reasoning parameter’s effort field specifies the inference depth. * (low/medium/high, default medium)
• Other models: chat_template_kwargs parameter is used, and please refer to Chat Completions API - Reasoning configuration for configuration.

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "Please solve a complex math problem.",
    "reasoning": {
      "effort": "high"
    }
  }'

curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "Please solve a complex math problem.",
    "reasoning": {
      "effort": "high"
    }
  }'

Reference

• Responses API vLLM Documentation
• Responses API OpenAI documentation
• OpenAI Developer Documentation

Tokenize API

POST /tokenize

Overview

The Tokenize API converts text to token IDs. Supports two request types: Completion (prompt-based) and Chat (messages-based). Compatible with vLLM’s Tokenize API.

Request

Context

Path Parameters

Query Parameters

Body Parameters - Common

Body Parameters - Completion method (prompt-based)

Body Parameters - Chat method (based on messages)

Example

curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "prompt": "Hello, world!"
  }'

curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "prompt": "Hello, world!"
  }'

curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {
        "role": "user",
        "content": "hi"
      }
    ]
  }'

curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {
        "role": "user",
        "content": "hi"
      }
    ]
  }'

Response

200 OK

Error Code

Example

{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": null
}

{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": null
}

{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": ["<|im_start|>", "user", "<|im_sep|>", "hi", "<|im_end|>", ""]
}

{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": ["<|im_start|>", "user", "<|im_sep|>", "hi", "<|im_end|>", ""]
}

Reference

• Tokenize API vLLM documentation

Models API

GET /v1/models

Overview

The Models API returns a list of Simple AI Inference models. Compatible with OpenAI’s Models API.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Since it is a GET request, there is no body.

Example

curl -X 'GET' \
  {Simple AI Inference endpoint}/v1/models \
  -H 'Authorization: bearer sai-xxxxxxx...'

curl -X 'GET' \
  {Simple AI Inference endpoint}/v1/models \
  -H 'Authorization: bearer sai-xxxxxxx...'

Response

200 OK

Error Code

Example

{
  "data": [
    {
      "id": "openai/gpt-oss-120b",
      "created": 1780979126,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Embedding-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Reranker-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    }
  ],
  "object": "list"
}

{
  "data": [
    {
      "id": "openai/gpt-oss-120b",
      "created": 1780979126,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Embedding-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Reranker-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    }
  ],
  "object": "list"
}

Reference

• Models API OpenAI documentation

4 - Data Privacy

Data Personal Information Protection and Security

Simple AI Inference operates its service with the principle that the confidentiality and security of customer data are paramount. Inference requests and response data transmitted through the service are processed only within the scope necessary to provide AI inference functionality and operate the service. This data will not be used for purposes other than providing the service.

Simple AI Inference adheres to the following principles.

• Customer request and response data are used solely for processing purposes related to service provision.
• We do not provide or share customers’ request and response content with third parties without the customer’s consent.
• Customer request and response data are not used as training data for AI model training or performance improvement.

5 - Release Note

Simple AI Inference