This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

References

References

You can view the API Reference supported by Simple AI Inference.

CategoryExplanation
API ReferenceList of APIs supported by Simple AI Inference
Table. Simple AI Inference Reference List

1 - API Reference

API Reference Overview

The API Reference supported by Simple AI Inference is as follows.

API nameAPIDetailed description
Chat Completions APIPOST /v1/chat/completionsIt is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.
Completions APIPOST /v1/completionsIt is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.
Embedding APIPOST /v1/embeddingsYou can convert text into high-dimensional vectors (embeddings) and use them for various natural language processing (NLP) tasks such as similarity calculation between texts, clustering, and search.
Rerank APIPOST /v2/rerankPredict the relevance between a single query and each item in a document list by applying an embedding model or a cross‑encoder model.
Responses APIPOST /v1/responsesCompatible with OpenAI’s Responses API, it can generate text or JSON output from text, image, or file inputs, and supports function calls and built-in tools.
Tokenize APIPOST /tokenizeConverts text to token IDs. Supports Completion mode and Chat mode.
Models APIGET /v1/modelsReturns a list of deployed models. Compatible with OpenAI’s Models API.
Table. Simple AI Inference supported API list

Chat Completions API

POST /v1/chat/completions

Overview

The Chat Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

Request

Context

KeyTypeDescriptionExample
Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
Request MethodstringHTTP methods used in API requestsPOST
HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
Body ParametersobjectParameters included in the request body{“model”: “google/gemma-4-31B-it”, “messages”: [{“role”: “user”, “content”: “hello”}], “stream”: true }
Table. Chat Completions API - Context

Path Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Chat Completions API - Path Parameters

Query Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Chat Completions API - Query Parameters

Body Parameters

NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
model-stringSpecify the model to use for generating responses“google/gemma-4-31B-it”
messagesrolestringMessage list containing conversation history[ { “role” : “user” , “content” : “message” }]
frequency_penalty-numberAdjust the penalty for repeated tokens0-2.0 ~ 2.00.5
logit_bias-objectAdjust the probability of a specific token (example: { “100”: 2.0 })nullKey: Token ID, Value: -100 ~ 100{ “100”: 2.0 }
logprobs-booleanReturns token probabilities for the top logprobs countfalsetrue, falsetrue
max_completion_tokens-integerLimit the maximum number of generated tokensNone0 ~ model maximum value100
max_tokens (Deprecated)-integerLimit the maximum number of generated tokensNone0 ~ model maximum value100
n-integerSpecify the number of responses to generate13
presence_penalty-numberAdjust the penalty for tokens in the existing text.0-2.0 ~ 2.01.0
seed-integerSpecify the seed value for controlling randomnessNone
stop-string / array / nullStop generation when a specific string appears.null"\n"
stream-booleanWhether to return results in streaming modefalsetrue/falsetrue
stream_optionsinclude_usage, continuous_usage_statsobjectControl streaming options (e.g., whether to include usage statistics)null{ “include_usage”: true }
temperature-numberAdjust the creativity of the generated output (higher values are more random)10.0 ~ 1.00.7
tool_choice-stringAdjust which Tool is invoked by the model
  • none: Do not invoke any Tool
  • auto: Let the model decide whether to generate a message or invoke a Tool
  • required: The model must invoke one or more Tools
  • when there is no tool: none
  • when there is a tool: auto
tools-arraylist of Tools that the model can invoke
  • only functions are supported as Tools
  • supports up to 128 functions
None
top_logprobs-integerSpecify the number of tokens with the highest probability for an integer between 0 and 20
  • Each is associated with a log probability value
  • logprobs must be set to true
  • Shows the probability values for the top k of completions
None0 ~ 203
top_p-numberLimit token sampling probability (higher values consider more tokens)10.0 ~ 1.00.9
prompt_safety_model-stringSpecify a guard model for prompt inspection. When set, the guard model checks the prompt first, returning the guard result if it is deemed unsafe, and processing the request with the model specified in the model parameter if it is safe.“meta-llama/Llama-Guard-4-12B”
chat_template_kwargs-objectAdditional keyword arguments to pass to the template renderer. Used for per-model reasoning configuration (see Reasoning configuration for details)null{ “enable_thinking”: true }
Table. Chat Completions API - Body Parameters

Example

Color mode
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
      "messages": [
      {
        "role": "assistant",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What is the capital of South Korea?"
      }
    ]
}'
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
      "messages": [
      {
        "role": "assistant",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What is the capital of South Korea?"
      }
    ]
}'
Code block. Chat Completions API Request Example

Response

200 OK

NameTypeDescription
idstringUnique identifier of the response
objectstringResponse object’s type (example: “chat.completion”)
createdintegerCreation time (Unix timestamp, in seconds)
modelstringName of the model used
choicesarrayList of generated response options
choices[].indexintegerThe index of the corresponding choice
choices[].messageobjectGenerated message object
choices[].message.rolestringThe role of the message author (e.g., “assistant”)
choices[].message.contentstringThe actual content of the generated message
choices[].message.reasoningstringThe actual content of the generated inference message
choices[].message.tool_callsarray (optional)Tool invocation information (may be included depending on model/settings)
choices[].finish_reasonstring or nullReason why the response was terminated (e.g., “stop”, “length”, etc)
choices[].stop_reasonobject or nullAdditional stop reason details
choices[].logprobsobject or nullLog probability information per token (included depending on settings)
usageobjectToken Usage Statistics
usage.prompt_tokensintegerNumber of tokens used in the input prompt
usage.completion_tokensintegerNumber of tokens used in the generated response
usage.total_tokensintegerTotal token count (input + output)
Table. Chat Completions API - 200 OK

Error Code

HTTP status codeErrorCode description
400Bad Request
422When a request is denied due to policies such as Prompt Guard
500Internal Server Error
Table. Chat Completions API - Error Code

Example

Color mode
{
  "id": "chatcmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "chat.completion",
  "created": 1749702816,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": null,
        "content": "The capital of South Korea is Seoul.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 62,
    "completion_tokens": 8,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}
{
  "id": "chatcmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "chat.completion",
  "created": 1749702816,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": null,
        "content": "The capital of South Korea is Seoul.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 62,
    "completion_tokens": 8,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}
Code block. Chat Completions API Response Example

Prompt Guard response

If the prompt_safety_model parameter is set, the guard model checks the prompt first.

  • safe: model The request is processed unchanged using the model specified in the parameter.
  • unsafe: Returns a guard result in the following form and the request is terminated.
Color mode
{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}
{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}
Code block. Chat Completions API Prompt Guard response Example (unsafe)

Reasoning configuration

Through the chat_template_kwargs parameter, you can control the reasoning (inference mode) configuration for each model. The default behavior and supported options vary by model.

modelbasic reasoningConfigure chat_template_kwargsExplanation
zai-org/GLM-5.2On (Think Max)
  • Think Max(default): {“reasoning_effort”: “max”} or omit — deepest reasoning
  • Think High: {“reasoning_effort”: “high”} — balanced depth and latency
  • Non-think: {“enable_thinking”: false} — fast response, no reasoning
Adjust inference depth with reasoning_effort, disable with enable_thinking=false
Qwen/Qwen3.6-27BOn
  • Disabled: {“enable_thinking”: false}
Reasoning is enabled by default and can be disabled when needed.
google/gemma-4-31B-itOff
  • Enable: {“enable_thinking”: true}
By default, reasoning is disabled, but it can be enabled when needed.
openai/gpt-oss-120bmedium
  • {“reasoning_effort”: “low”}
  • {“reasoning_effort”: “medium”} (default)
  • {“reasoning_effort”: “high”}
Adjust inference depth with reasoning_effort (default: medium)
Table. Chat Completions API - Model-specific Reasoning Configuration
Color mode
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "zai-org/GLM-5.2",
    "messages": [
      {
        "role": "user",
        "content": "Please solve a complex math problem."
      }
    ],
    "chat_template_kwargs": {
      "reasoning_effort": "high"
    }
  }'
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/chat/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "zai-org/GLM-5.2",
    "messages": [
      {
        "role": "user",
        "content": "Please solve a complex math problem."
      }
    ],
    "chat_template_kwargs": {
      "reasoning_effort": "high"
    }
  }'
Code block. Chat Completions API Reasoning setting Example

Reference

Completions API

POST /v1/completions

Overview

The Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

Request

Context

KeyTypeDescriptionExample
Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
Request MethodstringHTTP methods used in API requestsPOST
HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
Body ParametersobjectParameters included in the request body{“model”: “google/gemma-4-31B-it”, “prompt” : “hello”, “stream”: true }
Table. Completions API - Context

Path Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Completions API - Path Parameters

Query Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Completions API - Query Parameters

Body Parameters

NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
model-stringSpecify the model to use for generating responses“google/gemma-4-31B-it”
prompt-array, stringUser input text""
echo-booleanWhether to include the input text in the outputfalsetrue/falsetrue
frequency_penalty-numberAdjust the penalty for repeated tokens0-2.0 ~ 2.00.5
logit_bias-objectAdjust the probability of a specific token (example: { “100”: 2.0 })nullKey: Token ID, Value: -100~100{ “100”: 2.0 }
logprobs-integerReturns token probabilities for the top logprobs countnull1 ~ 55
max_completion_tokens-integerLimit the maximum number of generated tokensNone0~model maximum value100
max_tokens (Deprecated)-integerLimit the maximum number of generated tokensNone0~model maximum value100
n-integerSpecify the number of responses to generate13
presence_penalty-numberAdjust the penalty for tokens in the existing text.0-2.0 ~ 2.01.0
seed-integerSpecify the seed value for controlling randomnessNone
stop-string / array / nullStop generation when a specific string appears.null"\n"
stream-booleanWhether to return results in streaming modefalsetrue/falsetrue
stream_optionsinclude_usage, continuous_usage_statsobjectControl streaming options (e.g., whether to include usage statistics)null{ “include_usage”: true }
temperature-numberAdjust the creativity of the generated output (higher values are more random)10.0 ~ 1.00.7
top_p-numberLimit the sampling probability of tokens (higher values consider more tokens)10.0 ~ 1.00.9
Table. Completions API - Body Parameters

Example

Color mode
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
    "prompt": "What is the capital of South Korea?",
    "temperature": 0.7
  }'
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/completions \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "google/gemma-4-31B-it",
    "prompt": "What is the capital of South Korea?",
    "temperature": 0.7
  }'
Code block. Completions API Request Example

Response

200 OK

NameTypeDescription
idstringUnique identifier of the response
objectstringResponse object’s type (example: “text_completion”)
createdintegerCreation time (Unix timestamp, in seconds)
modelstringName of the model used
choicesarrayList of generated response options
choices[].indexnumberThe index of the corresponding choice
choices[].textstringGenerated text object
choices[].logprobsobjectLog probability information per token (included depending on settings)
choices[].finish_reasonstring or nullReason why the response was terminated (e.g., “stop”, “length”, etc)
choices[].stop_reasonobject or nullAdditional stop reason details
choices[].prompt_logprobsobject or nullLog probability per input prompt token (nullable)
usageobjectToken Usage Statistics
usage.prompt_tokensnumberNumber of tokens used in the input prompt
usage.total_tokensnumberTotal token count (input + output)
usage.completion_tokensnumberNumber of tokens used in the generated response
usage.prompt_tokens_detailsobjectPrompt token usage details
Table. Completions API - 200 OK

Error Code

HTTP status codeErrorCode description
400Bad Request
422When a request is denied due to policies such as Prompt Guard
500Internal Server Error
Table. Completions API - Error Code

Example

Color mode
{
  "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "text_completion",
  "created": 1749702612,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
      "logprobs": null,
      "finish_reason": "length",
      "stop_reason": null,
      "prompt_logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 25,
    "completion_tokens": 16,
    "prompt_tokens_details": null
  }
}
{
  "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object": "text_completion",
  "created": 1749702612,
  "model": "google/gemma-4-31B-it",
  "choices": [
    {
      "index": 0,
      "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
      "logprobs": null,
      "finish_reason": "length",
      "stop_reason": null,
      "prompt_logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 25,
    "completion_tokens": 16,
    "prompt_tokens_details": null
  }
}
Code block. Completions API Response Example

Reference

Embedding API

POST /v1/embeddings

Overview

The Embedding API converts given text into high-dimensional vectors (embeddings), enabling support for various natural language processing (NLP) tasks such as similarity calculation between texts, clustering, and search.

Request

Context

KeyTypeDescriptionExample
Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
Request MethodstringHTTP methods used in API requestsPOST
HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
Body ParametersobjectParameters included in the request body{ “model”: “Qwen/Qwen3-VL-Embedding-8B”, “input”: “What is the capital of France?”}
Table. Embedding API - Context

Path Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Embedding API - Path Parameters

Query Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Embedding API - Query Parameters

Body Parameters

NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
model-stringSpecify the model to use for generating responses“Qwen/Qwen3-VL-Embedding-8B”
input-arrayUser’s search query or question“What is the capital of France?"
encoding_format-stringSpecify the format for returning the embeddingfloat“float”, “base64”[0.01319122314453125,0.057220458984375, … (omitted)
truncate_prompt_tokens-integerLimit the number of input tokens> 0100
Table. Embedding API - Body Parameters

Example

Color mode
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/embeddings \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Embedding-8B",
    "input": "What is the capital of France?",
	"encoding_format": "float"
  }'
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/embeddings \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Embedding-8B",
    "input": "What is the capital of France?",
	"encoding_format": "float"
  }'
Code block. Embedding API Request Example

Response

200 OK

NameTypeDescription
idstringUnique identifier of the response
objectstringResponse object’s type (example: “list”)
creatednumberCreation time (Unix timestamp, in seconds)
modelstringName of the model used
dataarrayArray of objects containing embedding results
data.indexnumberOrder index of the input text (example: indicates the order when there are multiple input texts)
data.objectstringData item type
data.embeddingarrayEmbedding vector values of the input text (composed of a float array according to the model’s embedding dimensions)
usageobjectToken Usage Statistics
usage.prompt_tokensnumberNumber of tokens used in the input prompt
usage.total_tokensnumberTotal token count (input + output)
usage.completion_tokensnumberNumber of tokens used in the generated response
usage.prompt_tokens_detailsobjectPrompt token details
Table. Embedding API - 200 OK

Error Code

HTTP status codeErrorCode description
400Bad Request
422When the request is denied due to policies such as Prompt Guard
500Internal Server Error
Table. Embedding API - Error Code

Example

Color mode
{
  "id":"embd-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object":"list",
  "created":1749035024,
  "model":"Qwen/Qwen3-VL-Embedding-8B",
  "data":[
    {
      "index":0,
      "object":"embedding",
      "embedding":
      [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
      -0 ....(omitted)

      -0.05706787109375,-0.0147705078125]
    }
  ],
  "usage":
  {
    "prompt_tokens":9,
    "total_tokens":9,
    "completion_tokens":0,
    "prompt_tokens_details":null
  }
}
{
  "id":"embd-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "object":"list",
  "created":1749035024,
  "model":"Qwen/Qwen3-VL-Embedding-8B",
  "data":[
    {
      "index":0,
      "object":"embedding",
      "embedding":
      [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
      -0 ....(omitted)

      -0.05706787109375,-0.0147705078125]
    }
  ],
  "usage":
  {
    "prompt_tokens":9,
    "total_tokens":9,
    "completion_tokens":0,
    "prompt_tokens_details":null
  }
}
Code block. Embedding API Response Example

Reference

Rerank API

POST /v2/rerank

Overview

The Rerank API applies an embedding model or a cross‑encoder model to calculate similarity scores between a single query and each document in a list, and reorders the document ranking based on these scores. Generally, the score of a sentence pair represents the similarity between the two sentences on a scale from 0 to 1.

  • Embedding-based model: After converting the query and documents each into vectors, we measure the similarity between vectors (e.g., cosine similarity) and calculate a score.
  • Reranker(Cross-Encoder) based model: The query and document are input as a pair to the model for evaluation.

Request

Context

KeyTypeDescriptionExample
Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
Request MethodstringHTTP methods used in API requestsPOST
HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
Body ParametersobjectParameters included in the request body{ “model”: “Qwen/Qwen3-VL-Reranker-8B”, “query”: …, “documents”: […] }
Table. Rerank API - Context

Path Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Rerank API - Path Parameters

Query Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Rerank API - Query Parameters

Body Parameters

NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
model-stringSpecify the model to use for generating responses“Qwen/Qwen3-VL-Reranker-8B”
query-stringUser’s search query or question“What is the capital of France?"
documents-arrayList of documents to be reorderedMaximum model input length limit[“The capital of France is Paris.”]
top_n-integerSpecify the number of top documents to return (0 returns all)0> 05
truncate_prompt_tokens-integerLimit the number of input tokens> 0100
Table. Rerank API - Body Parameters

Example

Color mode
curl -X 'POST' \
   {Simple AI Inference endpoint}/v2/rerank \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Reranker-8B",
    "query": "What is the capital of France?",
    "documents": [
      "The capital of France is Paris.",
      "France capital city is known for the Eiffel Tower.",
      "Paris is located in the north-central part of France."
    ],
    "top_n": 2,
    "truncate_prompt_tokens": 512
  }'
curl -X 'POST' \
   {Simple AI Inference endpoint}/v2/rerank \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-Reranker-8B",
    "query": "What is the capital of France?",
    "documents": [
      "The capital of France is Paris.",
      "France capital city is known for the Eiffel Tower.",
      "Paris is located in the north-central part of France."
    ],
    "top_n": 2,
    "truncate_prompt_tokens": 512
  }'
Code block. Rerank API Request Example

Response

200 OK

NameTypeDescription
idstringUnique identifier of the API response (UUID format)
modelstringName of the model that generated the result
usageobjectObject containing resource information used in the request
usage.prompt_tokensintegerNumber of tokens used in the input prompt
usage.total_tokensintegerTotal number of tokens used for request processing
resultsarrayAn array containing the results of documents related to the query.
results[].indexintegerIndex number in the result array
results[].documentobjectAn object containing the contents of the retrieved document
results[].document.textstringThe actual text content of the retrieved document
results[].document.multi_modalobject or nullMultimodal document information
results[].relevance_scorefloatScore indicating the relevance between the query and the document (0 ~ 1)
Table. Rerank API - 200 OK

Error Code

HTTP status codeErrorCode description
400Bad Request
422When the request is denied due to policies such as Prompt Guard
500Internal Server Error
Table. Rerank API - Error Code

Example

Color mode
{
  "id": "score-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "model": "Qwen/Qwen3-VL-Reranker-8B",
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 54
  },
  "results": [
    {
      "index": 0,
      "document": {
        "text": "The capital of France is Paris.",
        "multi_modal": null
      },
      "relevance_score": 0.9237253665924072
    },
    {
      "index": 2,
      "document": {
        "text": "Paris is located in the north-central part of France.",
        "multi_modal": null
      },
      "relevance_score": 0.9181006550788879
    }
  ]
}
{
  "id": "score-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "model": "Qwen/Qwen3-VL-Reranker-8B",
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 54
  },
  "results": [
    {
      "index": 0,
      "document": {
        "text": "The capital of France is Paris.",
        "multi_modal": null
      },
      "relevance_score": 0.9237253665924072
    },
    {
      "index": 2,
      "document": {
        "text": "Paris is located in the north-central part of France.",
        "multi_modal": null
      },
      "relevance_score": 0.9181006550788879
    }
  ]
}
Code block. Rerank API Response Example

Reference

Responses API

POST /v1/responses

Overview

The Responses API is compatible with OpenAI’s Responses API and can be used in the OpenAI Python client. It can generate text or JSON output from text, image, or file inputs, and supports function calls and built-in tools (web search, file search, etc.).

Request

Context

KeyTypeDescriptionExample
Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
Request MethodstringHTTP methods used in API requestsPOST
HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
Body ParametersobjectParameters included in the request body{“model”: “openai/gpt-oss-120b”, “input”: “What is the capital of South Korea?” }
Table. Responses API - Context

Path Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Responses API - Path Parameters

Query Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Responses API - Query Parameters

Body Parameters

NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
model-stringModel ID to use for response generation“openai/gpt-oss-120b”
input-string / arrayText/image/file input for the model. String or InputItem array“Tell me a story” or [{ “role” : “user”, “content” : “message” }]
instructions-stringSystem (developer) message inserted into the model contextnull“You are a helpful assistant."
temperature-numberSampling temperature. The higher, the more random; the lower, the more deterministic.10 ~ 20.7
top_p-numberLimit the nucleus sampling probability. Changing it together with temperature is not recommended.10 ~ 10.9
top_logprobs-integerMaximum log-probability token count to return at each token positionnull0 ~ 203
stream-booleanWhether to return results in streaming modefalsetrue/falsetrue
stream_optionsinclude_usageobjectControl streaming options (e.g., whether to include usage statistics)null{ “include_usage”: true }
tools-arrayList of tools the model can invoke (built-in tools + function)
  • Built-in tools: web_search, file_search, code_interpreter etc.
  • function: supports up to 128
[]
tool_choice-string / objectHow the model selects tools
  • none: do not call a tool
  • auto: the model chooses whether to generate a message or call a tool
  • required: the model calls one or more tools
  • When there is no tool: none
  • When there is a tool: auto
prompt_safety_model-stringSpecify a guard model for prompt inspection. When set, the prompt is first checked by the guard model, and if judged unsafe, the guard result is returned; if safe, the request is processed with the model specified in the model parameter.“meta-llama/Llama-Guard-4-12B”
chat_template_kwargs-objectAdditional keyword arguments to pass to the template renderer. Used for per-model reasoning configuration (gpt-oss-120b uses the reasoning parameter, see Reasoning configuration for details).null{ “enable_thinking”: true }
reasoning-objectgpt-oss-120b model’s reasoning configuration. Specify inference depth with the effort field (low/medium/high, default medium)null{ “effort”: “high” }
Table. Responses API - Body Parameters

Example

Color mode
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "What is the capital of South Korea?"
  }'
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "What is the capital of South Korea?"
  }'
Code block. Responses API Request Example

Response

200 OK

NameTypeDescription
idstringUnique identifier of the response
objectstringResponse object type (always “response”)
created_atintegerCreation time (Unix timestamp, in seconds)
completed_atinteger or nullCompletion time (exists only when in completed state)
statusstringResponse status (completed/failed/in_progress/cancelled/queued/incomplete)
modelstringUsed model name
outputarrayArray of output items generated by the model
output[].typestringOutput item type (example: “message”)
output[].idstringOutput Item ID
output[].statusstringItem status (example: “completed”)
output[].rolestringMessage author role (example: “assistant”)
output[].contentarrayContent array
output[].content[].typestringContent type (example: “output_text”)
output[].content[].textstringGenerated text
output[].content[].annotationsarrayannotation array
errorobject or nullError information
incomplete_detailsobject or nullReason for incompletion (reason: max_output_tokens / content_filter)
instructionsstring or nullSystem/Developer Message
max_output_tokensinteger or nullMaximum output token count
parallel_tool_callsbooleanWhether to allow parallel tool invocation
previous_response_idstring or nullPrevious response ID
reasoningobject or nullreasoning composition (effort, summary)
storebooleanWhether to save the response
temperaturenumberSampling temperature
textobjectText response configuration (format, etc)
tool_choicestring / objectTool selection method
toolsarrayTool List
top_pnumberTop P value
truncationstringCutting strategy
usageobjectToken Usage Statistics
usage.input_tokensintegerNumber of input tokens
usage.input_tokens_details.cached_tokensintegerNumber of cached tokens
usage.output_tokensintegerNumber of output tokens
usage.output_tokens_details.reasoning_tokensintegerreasoning token count
usage.total_tokensintegerTotal token count
metadataobjectmetadata
Table. Responses API - 200 OK

Error Code

HTTP status codeErrorCode description
400Bad Request
422When the request is denied due to policies such as Prompt Guard
500Internal Server Error
Table. Responses API - Error Code

Example

Color mode
{
  "id": "resp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "object": "response",
  "created_at": 1741476542,
  "status": "completed",
  "completed_at": 1741476543,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": null,
  "model": "openai/gpt-oss-120b",
  "output": [
    {
      "type": "message",
      "id": "msg_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "The capital of South Korea is Seoul.",
          "annotations": []
        }
      ]
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "store": true,
  "temperature": 1.0,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1.0,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 54,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 8,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 62
  },
  "metadata": {}
}
{
  "id": "resp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "object": "response",
  "created_at": 1741476542,
  "status": "completed",
  "completed_at": 1741476543,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": null,
  "model": "openai/gpt-oss-120b",
  "output": [
    {
      "type": "message",
      "id": "msg_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "The capital of South Korea is Seoul.",
          "annotations": []
        }
      ]
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "store": true,
  "temperature": 1.0,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1.0,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 54,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 8,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 62
  },
  "metadata": {}
}
Code block. Responses API Response Example

Prompt Guard response

If the prompt_safety_model parameter is set, the guard model checks the prompt first.

  • safe: model The request is processed unchanged with the model specified in the model parameter.
  • unsafe: Returns a guard result in the following form and the request is terminated.
Color mode
{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}
{
  "guard_result": "unsafe",
  "categories": ["S1", "S2"],
  "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
  "messages": [
    "Cannot fulfill the request due to violent content.",
    "Cannot respond as it may promote illegal activities."
  ]
}
Code block. Responses API Prompt Guard response Example (unsafe)

Reasoning configuration

You can control the model-specific reasoning (inference mode) configuration via the chat_template_kwargs or reasoning parameters.

  • gpt-oss-120b: reasoning parameter’s effort field specifies the inference depth. * (low/medium/high, default medium)
  • Other models: chat_template_kwargs parameter is used, and please refer to Chat Completions API - Reasoning configuration for configuration.
modelParameterbasic reasoningConfiguration method
openai/gpt-oss-120breasoningmedium{ “effort”: “low” } / { “effort”: “medium” } (default) / { “effort”: “high” }
Other modelschat_template_kwargsVaries by modelChat Completions API - Reasoning Configuration see
Table. Responses API - Model-specific Reasoning Configuration
Color mode
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "Please solve a complex math problem.",
    "reasoning": {
      "effort": "high"
    }
  }'
curl -X 'POST' \
   {Simple AI Inference endpoint}/v1/responses \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "input": "Please solve a complex math problem.",
    "reasoning": {
      "effort": "high"
    }
  }'
Code block. Responses API Reasoning Setting Example

Reference

Tokenize API

POST /tokenize

Overview

The Tokenize API converts text to token IDs. Supports two request types: Completion (prompt-based) and Chat (messages-based). Compatible with vLLM’s Tokenize API.

Request

Context

KeyTypeDescriptionExample
Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
Request MethodstringHTTP methods used in API requestsPOST
HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
Body ParametersobjectParameters included in the request body{ “model”: “openai/gpt-oss-120b”, “prompt”: “Hello, world!” }
Table. Tokenize API - Context

Path Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Tokenize API - Path Parameters

Query Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Tokenize API - Query Parameters

Body Parameters - Common

NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
model-stringSpecify the model to use for tokenization“openai/gpt-oss-120b”
Table. Tokenize API - Body Parameters (Common)

Body Parameters - Completion method (prompt-based)

NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
prompt-stringText to be tokenized“Hello, world!"
add_special_tokens-booleanIf true, add special tokens (e.g., BOS) to the prompt.truetrue / falsetrue
return_token_strs-booleanIf true, also return the token string corresponding to the token ID.falsetrue / falsetrue
Table. Tokenize API - Body Parameters (Completion mode)

Body Parameters - Chat method (based on messages)

NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
messagesrolestringMessage list containing conversation history[{ “role”: “user”, “content”: “hi” }]
add_generation_prompt-booleanIf true, add a generation prompt to the chat template. Cannot set to true simultaneously with continue_final_message.truetrue / falsetrue
continue_final_message-booleanIf true, the last message is formatted in an open form without EOS. Instead of the model starting a new message, it continues that message. Cannot be set to true together with add_generation_prompt.falsetrue / falsefalse
add_special_tokens-booleanIf true, insert additional special tokens such as BOS in addition to the special tokens added by the chat template. Most models handle special tokens with the chat template, so the default false is recommended.falsetrue / falsefalse
return_token_strs-booleanIf true, also return the token string corresponding to the token ID.falsetrue / falsetrue
chat_template-stringJinja template to use for conversion. Provide it if not defined in the tokenizer.null
chat_template_kwargs-objectAdditional keyword arguments to pass to the template renderernull{ “add_generation_prompt”: true }
tools-arrayList of tools the model can invokenull
Table. Tokenize API - Body Parameters (Chat mode)

Example

Color mode
curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "prompt": "Hello, world!"
  }'
curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "prompt": "Hello, world!"
  }'
curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {
        "role": "user",
        "content": "hi"
      }
    ]
  }'
curl -X 'POST' \
  {Simple AI Inference endpoint}/tokenize \
  -H 'Authorization: bearer sai-xxxxxxx...' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {
        "role": "user",
        "content": "hi"
      }
    ]
  }'
Code block. Tokenize API Request Example

Response

200 OK

NameTypeDescription
countintegerNumber of tokenized tokens
max_model_lenintegerMaximum token length supported by the model
tokensarrayList of tokenized token IDs
token_strsarray or nullList of token strings corresponding to token IDs (returned only when return_token_strs is true)
Table. Tokenize API - 200 OK

Error Code

HTTP status codeErrorCode description
400Bad Request (model field missing, request body missing, etc)
404Model Not Found (unsupported model)
500Internal Server Error
Table. Tokenize API - Error Code

Example

Color mode
{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": null
}
{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": null
}
{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": ["<|im_start|>", "user", "<|im_sep|>", "hi", "<|im_end|>", ""]
}
{
  "max_model_len": 1024,
  "count": 6,
  "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
  "token_strs": ["<|im_start|>", "user", "<|im_sep|>", "hi", "<|im_end|>", ""]
}
Code block. Tokenize API Response Example

Reference

Models API

GET /v1/models

Overview

The Models API returns a list of Simple AI Inference models. Compatible with OpenAI’s Models API.

Request

Context

KeyTypeDescriptionExample
Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
Request MethodstringHTTP methods used in API requestsGET
HeadersobjectHeader information required for the request{ “Authorization”: “bearer sai-xxxxxxx…” }
Body Parameters--Since it is a GET request, there is no body.
Table. Models API - Context

Path Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Models API - Path Parameters

Query Parameters

NametypeRequiredDescriptionDefault valueBoundary valueExample
None
Table. Models API - Query Parameters

Body Parameters

Since it is a GET request, there is no body.

Example

Color mode
curl -X 'GET' \
  {Simple AI Inference endpoint}/v1/models \
  -H 'Authorization: bearer sai-xxxxxxx...'
curl -X 'GET' \
  {Simple AI Inference endpoint}/v1/models \
  -H 'Authorization: bearer sai-xxxxxxx...'
Code block. Models API Request Example

Response

200 OK

NameTypeDescription
objectstringResponse object’s type (“list”)
dataarrayModel object list
data[].idstringModel identifier
data[].objectstringObject type (“model”)
data[].createdintegerModel creation time (Unix timestamp, in seconds)
data[].owned_bystringModel owner
Table. Models API - 200 OK

Error Code

HTTP status codeErrorCode description
401Unauthorized (apikey missing or invalid)
500Internal Server Error
Table. Models API - Error Code

Example

Color mode
{
  "data": [
    {
      "id": "openai/gpt-oss-120b",
      "created": 1780979126,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Embedding-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Reranker-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    }
  ],
  "object": "list"
}
{
  "data": [
    {
      "id": "openai/gpt-oss-120b",
      "created": 1780979126,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Embedding-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    },
    {
      "id": "Qwen/Qwen3-VL-Reranker-8B",
      "created": 1781512915,
      "object": "model",
      "owned_by": "SCP Simple AI Inference"
    }
  ],
  "object": "list"
}
Code block. Models API Response Example

Reference