This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

API Reference

    API Reference Overview

    The API Reference supported by Simple AI Inference is as follows.

    API nameAPIDetailed description
    Chat Completions APIPOST /v1/chat/completionsIt is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.
    Completions APIPOST /v1/completionsIt is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.
    Embedding APIPOST /v1/embeddingsYou can convert text into high-dimensional vectors (embeddings) and use them for various natural language processing (NLP) tasks such as similarity calculation between texts, clustering, and search.
    Rerank APIPOST /v2/rerankPredict the relevance between a single query and each item in a document list by applying an embedding model or a cross‑encoder model.
    Responses APIPOST /v1/responsesCompatible with OpenAI’s Responses API, it can generate text or JSON output from text, image, or file inputs, and supports function calls and built-in tools.
    Tokenize APIPOST /tokenizeConverts text to token IDs. Supports Completion mode and Chat mode.
    Models APIGET /v1/modelsReturns a list of deployed models. Compatible with OpenAI’s Models API.
    Table. Simple AI Inference supported API list

    Chat Completions API

    POST /v1/chat/completions
    

    Overview

    The Chat Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

    Request

    Context

    KeyTypeDescriptionExample
    Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
    Request MethodstringHTTP methods used in API requestsPOST
    HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
    Body ParametersobjectParameters included in the request body{“model”: “google/gemma-4-31B-it”, “messages”: [{“role”: “user”, “content”: “hello”}], “stream”: true }
    Table. Chat Completions API - Context

    Path Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Chat Completions API - Path Parameters

    Query Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Chat Completions API - Query Parameters

    Body Parameters

    NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
    model-stringSpecify the model to use for generating responses“google/gemma-4-31B-it”
    messagesrolestringMessage list containing conversation history[ { “role” : “user” , “content” : “message” }]
    frequency_penalty-numberAdjust the penalty for repeated tokens0-2.0 ~ 2.00.5
    logit_bias-objectAdjust the probability of a specific token (example: { “100”: 2.0 })nullKey: Token ID, Value: -100 ~ 100{ “100”: 2.0 }
    logprobs-booleanReturns token probabilities for the top logprobs countfalsetrue, falsetrue
    max_completion_tokens-integerLimit the maximum number of generated tokensNone0 ~ model maximum value100
    max_tokens (Deprecated)-integerLimit the maximum number of generated tokensNone0 ~ model maximum value100
    n-integerSpecify the number of responses to generate13
    presence_penalty-numberAdjust the penalty for tokens in the existing text.0-2.0 ~ 2.01.0
    seed-integerSpecify the seed value for controlling randomnessNone
    stop-string / array / nullStop generation when a specific string appears.null"\n"
    stream-booleanWhether to return results in streaming modefalsetrue/falsetrue
    stream_optionsinclude_usage, continuous_usage_statsobjectControl streaming options (e.g., whether to include usage statistics)null{ “include_usage”: true }
    temperature-numberAdjust the creativity of the generated output (higher values are more random)10.0 ~ 1.00.7
    tool_choice-stringAdjust which Tool is invoked by the model
    • none: Do not invoke any Tool
    • auto: Let the model decide whether to generate a message or invoke a Tool
    • required: The model must invoke one or more Tools
    • when there is no tool: none
    • when there is a tool: auto
    tools-arraylist of Tools that the model can invoke
    • only functions are supported as Tools
    • supports up to 128 functions
    None
    top_logprobs-integerSpecify the number of tokens with the highest probability for an integer between 0 and 20
    • Each is associated with a log probability value
    • logprobs must be set to true
    • Shows the probability values for the top k of completions
    None0 ~ 203
    top_p-numberLimit token sampling probability (higher values consider more tokens)10.0 ~ 1.00.9
    prompt_safety_model-stringSpecify a guard model for prompt inspection. When set, the guard model checks the prompt first, returning the guard result if it is deemed unsafe, and processing the request with the model specified in the model parameter if it is safe.“meta-llama/Llama-Guard-4-12B”
    chat_template_kwargs-objectAdditional keyword arguments to pass to the template renderer. Used for per-model reasoning configuration (see Reasoning configuration for details)null{ “enable_thinking”: true }
    Table. Chat Completions API - Body Parameters

    Example

    Color mode
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/chat/completions \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "google/gemma-4-31B-it",
          "messages": [
          {
            "role": "assistant",
            "content": "You are a helpful assistant."
          },
          {
            "role": "user",
            "content": "What is the capital of South Korea?"
          }
        ]
    }'
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/chat/completions \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "google/gemma-4-31B-it",
          "messages": [
          {
            "role": "assistant",
            "content": "You are a helpful assistant."
          },
          {
            "role": "user",
            "content": "What is the capital of South Korea?"
          }
        ]
    }'
    Code block. Chat Completions API Request Example

    Response

    200 OK

    NameTypeDescription
    idstringUnique identifier of the response
    objectstringResponse object’s type (example: “chat.completion”)
    createdintegerCreation time (Unix timestamp, in seconds)
    modelstringName of the model used
    choicesarrayList of generated response options
    choices[].indexintegerThe index of the corresponding choice
    choices[].messageobjectGenerated message object
    choices[].message.rolestringThe role of the message author (e.g., “assistant”)
    choices[].message.contentstringThe actual content of the generated message
    choices[].message.reasoningstringThe actual content of the generated inference message
    choices[].message.tool_callsarray (optional)Tool invocation information (may be included depending on model/settings)
    choices[].finish_reasonstring or nullReason why the response was terminated (e.g., “stop”, “length”, etc)
    choices[].stop_reasonobject or nullAdditional stop reason details
    choices[].logprobsobject or nullLog probability information per token (included depending on settings)
    usageobjectToken Usage Statistics
    usage.prompt_tokensintegerNumber of tokens used in the input prompt
    usage.completion_tokensintegerNumber of tokens used in the generated response
    usage.total_tokensintegerTotal token count (input + output)
    Table. Chat Completions API - 200 OK

    Error Code

    HTTP status codeErrorCode description
    400Bad Request
    422When a request is denied due to policies such as Prompt Guard
    500Internal Server Error
    Table. Chat Completions API - Error Code

    Example

    Color mode
    {
      "id": "chatcmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "object": "chat.completion",
      "created": 1749702816,
      "model": "google/gemma-4-31B-it",
      "choices": [
        {
          "index": 0,
          "message": {
            "role": "assistant",
            "reasoning": null,
            "content": "The capital of South Korea is Seoul.",
            "tool_calls": []
          },
          "logprobs": null,
          "finish_reason": "stop",
          "stop_reason": null
        }
      ],
      "usage": {
        "prompt_tokens": 54,
        "total_tokens": 62,
        "completion_tokens": 8,
        "prompt_tokens_details": null
      },
      "prompt_logprobs": null
    }
    {
      "id": "chatcmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "object": "chat.completion",
      "created": 1749702816,
      "model": "google/gemma-4-31B-it",
      "choices": [
        {
          "index": 0,
          "message": {
            "role": "assistant",
            "reasoning": null,
            "content": "The capital of South Korea is Seoul.",
            "tool_calls": []
          },
          "logprobs": null,
          "finish_reason": "stop",
          "stop_reason": null
        }
      ],
      "usage": {
        "prompt_tokens": 54,
        "total_tokens": 62,
        "completion_tokens": 8,
        "prompt_tokens_details": null
      },
      "prompt_logprobs": null
    }
    Code block. Chat Completions API Response Example

    Prompt Guard response

    If the prompt_safety_model parameter is set, the guard model checks the prompt first.

    • safe: model The request is processed unchanged using the model specified in the parameter.
    • unsafe: Returns a guard result in the following form and the request is terminated.
    Color mode
    {
      "guard_result": "unsafe",
      "categories": ["S1", "S2"],
      "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
      "messages": [
        "Cannot fulfill the request due to violent content.",
        "Cannot respond as it may promote illegal activities."
      ]
    }
    {
      "guard_result": "unsafe",
      "categories": ["S1", "S2"],
      "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
      "messages": [
        "Cannot fulfill the request due to violent content.",
        "Cannot respond as it may promote illegal activities."
      ]
    }
    Code block. Chat Completions API Prompt Guard response Example (unsafe)

    Reasoning configuration

    Through the chat_template_kwargs parameter, you can control the reasoning (inference mode) configuration for each model. The default behavior and supported options vary by model.

    modelbasic reasoningConfigure chat_template_kwargsExplanation
    zai-org/GLM-5.2On (Think Max)
    • Think Max(default): {“reasoning_effort”: “max”} or omit — deepest reasoning
    • Think High: {“reasoning_effort”: “high”} — balanced depth and latency
    • Non-think: {“enable_thinking”: false} — fast response, no reasoning
    Adjust inference depth with reasoning_effort, disable with enable_thinking=false
    Qwen/Qwen3.6-27BOn
    • Disabled: {“enable_thinking”: false}
    Reasoning is enabled by default and can be disabled when needed.
    google/gemma-4-31B-itOff
    • Enable: {“enable_thinking”: true}
    By default, reasoning is disabled, but it can be enabled when needed.
    openai/gpt-oss-120bmedium
    • {“reasoning_effort”: “low”}
    • {“reasoning_effort”: “medium”} (default)
    • {“reasoning_effort”: “high”}
    Adjust inference depth with reasoning_effort (default: medium)
    Table. Chat Completions API - Model-specific Reasoning Configuration
    Color mode
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/chat/completions \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "zai-org/GLM-5.2",
        "messages": [
          {
            "role": "user",
            "content": "Please solve a complex math problem."
          }
        ],
        "chat_template_kwargs": {
          "reasoning_effort": "high"
        }
      }'
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/chat/completions \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "zai-org/GLM-5.2",
        "messages": [
          {
            "role": "user",
            "content": "Please solve a complex math problem."
          }
        ],
        "chat_template_kwargs": {
          "reasoning_effort": "high"
        }
      }'
    Code block. Chat Completions API Reasoning setting Example

    Reference

    Completions API

    POST /v1/completions
    

    Overview

    The Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

    Request

    Context

    KeyTypeDescriptionExample
    Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
    Request MethodstringHTTP methods used in API requestsPOST
    HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
    Body ParametersobjectParameters included in the request body{“model”: “google/gemma-4-31B-it”, “prompt” : “hello”, “stream”: true }
    Table. Completions API - Context

    Path Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Completions API - Path Parameters

    Query Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Completions API - Query Parameters

    Body Parameters

    NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
    model-stringSpecify the model to use for generating responses“google/gemma-4-31B-it”
    prompt-array, stringUser input text""
    echo-booleanWhether to include the input text in the outputfalsetrue/falsetrue
    frequency_penalty-numberAdjust the penalty for repeated tokens0-2.0 ~ 2.00.5
    logit_bias-objectAdjust the probability of a specific token (example: { “100”: 2.0 })nullKey: Token ID, Value: -100~100{ “100”: 2.0 }
    logprobs-integerReturns token probabilities for the top logprobs countnull1 ~ 55
    max_completion_tokens-integerLimit the maximum number of generated tokensNone0~model maximum value100
    max_tokens (Deprecated)-integerLimit the maximum number of generated tokensNone0~model maximum value100
    n-integerSpecify the number of responses to generate13
    presence_penalty-numberAdjust the penalty for tokens in the existing text.0-2.0 ~ 2.01.0
    seed-integerSpecify the seed value for controlling randomnessNone
    stop-string / array / nullStop generation when a specific string appears.null"\n"
    stream-booleanWhether to return results in streaming modefalsetrue/falsetrue
    stream_optionsinclude_usage, continuous_usage_statsobjectControl streaming options (e.g., whether to include usage statistics)null{ “include_usage”: true }
    temperature-numberAdjust the creativity of the generated output (higher values are more random)10.0 ~ 1.00.7
    top_p-numberLimit the sampling probability of tokens (higher values consider more tokens)10.0 ~ 1.00.9
    Table. Completions API - Body Parameters

    Example

    Color mode
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/completions \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "google/gemma-4-31B-it",
        "prompt": "What is the capital of South Korea?",
        "temperature": 0.7
      }'
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/completions \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "google/gemma-4-31B-it",
        "prompt": "What is the capital of South Korea?",
        "temperature": 0.7
      }'
    Code block. Completions API Request Example

    Response

    200 OK

    NameTypeDescription
    idstringUnique identifier of the response
    objectstringResponse object’s type (example: “text_completion”)
    createdintegerCreation time (Unix timestamp, in seconds)
    modelstringName of the model used
    choicesarrayList of generated response options
    choices[].indexnumberThe index of the corresponding choice
    choices[].textstringGenerated text object
    choices[].logprobsobjectLog probability information per token (included depending on settings)
    choices[].finish_reasonstring or nullReason why the response was terminated (e.g., “stop”, “length”, etc)
    choices[].stop_reasonobject or nullAdditional stop reason details
    choices[].prompt_logprobsobject or nullLog probability per input prompt token (nullable)
    usageobjectToken Usage Statistics
    usage.prompt_tokensnumberNumber of tokens used in the input prompt
    usage.total_tokensnumberTotal token count (input + output)
    usage.completion_tokensnumberNumber of tokens used in the generated response
    usage.prompt_tokens_detailsobjectPrompt token usage details
    Table. Completions API - 200 OK

    Error Code

    HTTP status codeErrorCode description
    400Bad Request
    422When a request is denied due to policies such as Prompt Guard
    500Internal Server Error
    Table. Completions API - Error Code

    Example

    Color mode
    {
      "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "object": "text_completion",
      "created": 1749702612,
      "model": "google/gemma-4-31B-it",
      "choices": [
        {
          "index": 0,
          "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
          "logprobs": null,
          "finish_reason": "length",
          "stop_reason": null,
          "prompt_logprobs": null
        }
      ],
      "usage": {
        "prompt_tokens": 9,
        "total_tokens": 25,
        "completion_tokens": 16,
        "prompt_tokens_details": null
      }
    }
    {
      "id": "cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "object": "text_completion",
      "created": 1749702612,
      "model": "google/gemma-4-31B-it",
      "choices": [
        {
          "index": 0,
          "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
          "logprobs": null,
          "finish_reason": "length",
          "stop_reason": null,
          "prompt_logprobs": null
        }
      ],
      "usage": {
        "prompt_tokens": 9,
        "total_tokens": 25,
        "completion_tokens": 16,
        "prompt_tokens_details": null
      }
    }
    Code block. Completions API Response Example

    Reference

    Embedding API

    POST /v1/embeddings
    

    Overview

    The Embedding API converts given text into high-dimensional vectors (embeddings), enabling support for various natural language processing (NLP) tasks such as similarity calculation between texts, clustering, and search.

    Request

    Context

    KeyTypeDescriptionExample
    Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
    Request MethodstringHTTP methods used in API requestsPOST
    HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
    Body ParametersobjectParameters included in the request body{ “model”: “Qwen/Qwen3-VL-Embedding-8B”, “input”: “What is the capital of France?”}
    Table. Embedding API - Context

    Path Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Embedding API - Path Parameters

    Query Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Embedding API - Query Parameters

    Body Parameters

    NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
    model-stringSpecify the model to use for generating responses“Qwen/Qwen3-VL-Embedding-8B”
    input-arrayUser’s search query or question“What is the capital of France?"
    encoding_format-stringSpecify the format for returning the embeddingfloat“float”, “base64”[0.01319122314453125,0.057220458984375, … (omitted)
    truncate_prompt_tokens-integerLimit the number of input tokens> 0100
    Table. Embedding API - Body Parameters

    Example

    Color mode
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/embeddings \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "Qwen/Qwen3-VL-Embedding-8B",
        "input": "What is the capital of France?",
    	"encoding_format": "float"
      }'
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/embeddings \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "Qwen/Qwen3-VL-Embedding-8B",
        "input": "What is the capital of France?",
    	"encoding_format": "float"
      }'
    Code block. Embedding API Request Example

    Response

    200 OK

    NameTypeDescription
    idstringUnique identifier of the response
    objectstringResponse object’s type (example: “list”)
    creatednumberCreation time (Unix timestamp, in seconds)
    modelstringName of the model used
    dataarrayArray of objects containing embedding results
    data.indexnumberOrder index of the input text (example: indicates the order when there are multiple input texts)
    data.objectstringData item type
    data.embeddingarrayEmbedding vector values of the input text (composed of a float array according to the model’s embedding dimensions)
    usageobjectToken Usage Statistics
    usage.prompt_tokensnumberNumber of tokens used in the input prompt
    usage.total_tokensnumberTotal token count (input + output)
    usage.completion_tokensnumberNumber of tokens used in the generated response
    usage.prompt_tokens_detailsobjectPrompt token details
    Table. Embedding API - 200 OK

    Error Code

    HTTP status codeErrorCode description
    400Bad Request
    422When the request is denied due to policies such as Prompt Guard
    500Internal Server Error
    Table. Embedding API - Error Code

    Example

    Color mode
    {
      "id":"embd-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "object":"list",
      "created":1749035024,
      "model":"Qwen/Qwen3-VL-Embedding-8B",
      "data":[
        {
          "index":0,
          "object":"embedding",
          "embedding":
          [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
          -0 ....(omitted)
    
          -0.05706787109375,-0.0147705078125]
        }
      ],
      "usage":
      {
        "prompt_tokens":9,
        "total_tokens":9,
        "completion_tokens":0,
        "prompt_tokens_details":null
      }
    }
    {
      "id":"embd-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "object":"list",
      "created":1749035024,
      "model":"Qwen/Qwen3-VL-Embedding-8B",
      "data":[
        {
          "index":0,
          "object":"embedding",
          "embedding":
          [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
          -0 ....(omitted)
    
          -0.05706787109375,-0.0147705078125]
        }
      ],
      "usage":
      {
        "prompt_tokens":9,
        "total_tokens":9,
        "completion_tokens":0,
        "prompt_tokens_details":null
      }
    }
    Code block. Embedding API Response Example

    Reference

    Rerank API

    POST /v2/rerank
    

    Overview

    The Rerank API applies an embedding model or a cross‑encoder model to calculate similarity scores between a single query and each document in a list, and reorders the document ranking based on these scores. Generally, the score of a sentence pair represents the similarity between the two sentences on a scale from 0 to 1.

    • Embedding-based model: After converting the query and documents each into vectors, we measure the similarity between vectors (e.g., cosine similarity) and calculate a score.
    • Reranker(Cross-Encoder) based model: The query and document are input as a pair to the model for evaluation.

    Request

    Context

    KeyTypeDescriptionExample
    Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
    Request MethodstringHTTP methods used in API requestsPOST
    HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
    Body ParametersobjectParameters included in the request body{ “model”: “Qwen/Qwen3-VL-Reranker-8B”, “query”: …, “documents”: […] }
    Table. Rerank API - Context

    Path Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Rerank API - Path Parameters

    Query Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Rerank API - Query Parameters

    Body Parameters

    NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
    model-stringSpecify the model to use for generating responses“Qwen/Qwen3-VL-Reranker-8B”
    query-stringUser’s search query or question“What is the capital of France?"
    documents-arrayList of documents to be reorderedMaximum model input length limit[“The capital of France is Paris.”]
    top_n-integerSpecify the number of top documents to return (0 returns all)0> 05
    truncate_prompt_tokens-integerLimit the number of input tokens> 0100
    Table. Rerank API - Body Parameters

    Example

    Color mode
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v2/rerank \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "Qwen/Qwen3-VL-Reranker-8B",
        "query": "What is the capital of France?",
        "documents": [
          "The capital of France is Paris.",
          "France capital city is known for the Eiffel Tower.",
          "Paris is located in the north-central part of France."
        ],
        "top_n": 2,
        "truncate_prompt_tokens": 512
      }'
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v2/rerank \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "Qwen/Qwen3-VL-Reranker-8B",
        "query": "What is the capital of France?",
        "documents": [
          "The capital of France is Paris.",
          "France capital city is known for the Eiffel Tower.",
          "Paris is located in the north-central part of France."
        ],
        "top_n": 2,
        "truncate_prompt_tokens": 512
      }'
    Code block. Rerank API Request Example

    Response

    200 OK

    NameTypeDescription
    idstringUnique identifier of the API response (UUID format)
    modelstringName of the model that generated the result
    usageobjectObject containing resource information used in the request
    usage.prompt_tokensintegerNumber of tokens used in the input prompt
    usage.total_tokensintegerTotal number of tokens used for request processing
    resultsarrayAn array containing the results of documents related to the query.
    results[].indexintegerIndex number in the result array
    results[].documentobjectAn object containing the contents of the retrieved document
    results[].document.textstringThe actual text content of the retrieved document
    results[].document.multi_modalobject or nullMultimodal document information
    results[].relevance_scorefloatScore indicating the relevance between the query and the document (0 ~ 1)
    Table. Rerank API - 200 OK

    Error Code

    HTTP status codeErrorCode description
    400Bad Request
    422When the request is denied due to policies such as Prompt Guard
    500Internal Server Error
    Table. Rerank API - Error Code

    Example

    Color mode
    {
      "id": "score-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "model": "Qwen/Qwen3-VL-Reranker-8B",
      "usage": {
        "prompt_tokens": 54,
        "total_tokens": 54
      },
      "results": [
        {
          "index": 0,
          "document": {
            "text": "The capital of France is Paris.",
            "multi_modal": null
          },
          "relevance_score": 0.9237253665924072
        },
        {
          "index": 2,
          "document": {
            "text": "Paris is located in the north-central part of France.",
            "multi_modal": null
          },
          "relevance_score": 0.9181006550788879
        }
      ]
    }
    {
      "id": "score-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "model": "Qwen/Qwen3-VL-Reranker-8B",
      "usage": {
        "prompt_tokens": 54,
        "total_tokens": 54
      },
      "results": [
        {
          "index": 0,
          "document": {
            "text": "The capital of France is Paris.",
            "multi_modal": null
          },
          "relevance_score": 0.9237253665924072
        },
        {
          "index": 2,
          "document": {
            "text": "Paris is located in the north-central part of France.",
            "multi_modal": null
          },
          "relevance_score": 0.9181006550788879
        }
      ]
    }
    Code block. Rerank API Response Example

    Reference

    Responses API

    POST /v1/responses
    

    Overview

    The Responses API is compatible with OpenAI’s Responses API and can be used in the OpenAI Python client. It can generate text or JSON output from text, image, or file inputs, and supports function calls and built-in tools (web search, file search, etc.).

    Request

    Context

    KeyTypeDescriptionExample
    Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
    Request MethodstringHTTP methods used in API requestsPOST
    HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
    Body ParametersobjectParameters included in the request body{“model”: “openai/gpt-oss-120b”, “input”: “What is the capital of South Korea?” }
    Table. Responses API - Context

    Path Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Responses API - Path Parameters

    Query Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Responses API - Query Parameters

    Body Parameters

    NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
    model-stringModel ID to use for response generation“openai/gpt-oss-120b”
    input-string / arrayText/image/file input for the model. String or InputItem array“Tell me a story” or [{ “role” : “user”, “content” : “message” }]
    instructions-stringSystem (developer) message inserted into the model contextnull“You are a helpful assistant."
    temperature-numberSampling temperature. The higher, the more random; the lower, the more deterministic.10 ~ 20.7
    top_p-numberLimit the nucleus sampling probability. Changing it together with temperature is not recommended.10 ~ 10.9
    top_logprobs-integerMaximum log-probability token count to return at each token positionnull0 ~ 203
    stream-booleanWhether to return results in streaming modefalsetrue/falsetrue
    stream_optionsinclude_usageobjectControl streaming options (e.g., whether to include usage statistics)null{ “include_usage”: true }
    tools-arrayList of tools the model can invoke (built-in tools + function)
    • Built-in tools: web_search, file_search, code_interpreter etc.
    • function: supports up to 128
    []
    tool_choice-string / objectHow the model selects tools
    • none: do not call a tool
    • auto: the model chooses whether to generate a message or call a tool
    • required: the model calls one or more tools
    • When there is no tool: none
    • When there is a tool: auto
    prompt_safety_model-stringSpecify a guard model for prompt inspection. When set, the prompt is first checked by the guard model, and if judged unsafe, the guard result is returned; if safe, the request is processed with the model specified in the model parameter.“meta-llama/Llama-Guard-4-12B”
    chat_template_kwargs-objectAdditional keyword arguments to pass to the template renderer. Used for per-model reasoning configuration (gpt-oss-120b uses the reasoning parameter, see Reasoning configuration for details).null{ “enable_thinking”: true }
    reasoning-objectgpt-oss-120b model’s reasoning configuration. Specify inference depth with the effort field (low/medium/high, default medium)null{ “effort”: “high” }
    Table. Responses API - Body Parameters

    Example

    Color mode
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/responses \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "openai/gpt-oss-120b",
        "input": "What is the capital of South Korea?"
      }'
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/responses \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "openai/gpt-oss-120b",
        "input": "What is the capital of South Korea?"
      }'
    Code block. Responses API Request Example

    Response

    200 OK

    NameTypeDescription
    idstringUnique identifier of the response
    objectstringResponse object type (always “response”)
    created_atintegerCreation time (Unix timestamp, in seconds)
    completed_atinteger or nullCompletion time (exists only when in completed state)
    statusstringResponse status (completed/failed/in_progress/cancelled/queued/incomplete)
    modelstringUsed model name
    outputarrayArray of output items generated by the model
    output[].typestringOutput item type (example: “message”)
    output[].idstringOutput Item ID
    output[].statusstringItem status (example: “completed”)
    output[].rolestringMessage author role (example: “assistant”)
    output[].contentarrayContent array
    output[].content[].typestringContent type (example: “output_text”)
    output[].content[].textstringGenerated text
    output[].content[].annotationsarrayannotation array
    errorobject or nullError information
    incomplete_detailsobject or nullReason for incompletion (reason: max_output_tokens / content_filter)
    instructionsstring or nullSystem/Developer Message
    max_output_tokensinteger or nullMaximum output token count
    parallel_tool_callsbooleanWhether to allow parallel tool invocation
    previous_response_idstring or nullPrevious response ID
    reasoningobject or nullreasoning composition (effort, summary)
    storebooleanWhether to save the response
    temperaturenumberSampling temperature
    textobjectText response configuration (format, etc)
    tool_choicestring / objectTool selection method
    toolsarrayTool List
    top_pnumberTop P value
    truncationstringCutting strategy
    usageobjectToken Usage Statistics
    usage.input_tokensintegerNumber of input tokens
    usage.input_tokens_details.cached_tokensintegerNumber of cached tokens
    usage.output_tokensintegerNumber of output tokens
    usage.output_tokens_details.reasoning_tokensintegerreasoning token count
    usage.total_tokensintegerTotal token count
    metadataobjectmetadata
    Table. Responses API - 200 OK

    Error Code

    HTTP status codeErrorCode description
    400Bad Request
    422When the request is denied due to policies such as Prompt Guard
    500Internal Server Error
    Table. Responses API - Error Code

    Example

    Color mode
    {
      "id": "resp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "object": "response",
      "created_at": 1741476542,
      "status": "completed",
      "completed_at": 1741476543,
      "error": null,
      "incomplete_details": null,
      "instructions": null,
      "max_output_tokens": null,
      "model": "openai/gpt-oss-120b",
      "output": [
        {
          "type": "message",
          "id": "msg_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
          "status": "completed",
          "role": "assistant",
          "content": [
            {
              "type": "output_text",
              "text": "The capital of South Korea is Seoul.",
              "annotations": []
            }
          ]
        }
      ],
      "parallel_tool_calls": true,
      "previous_response_id": null,
      "reasoning": {
        "effort": null,
        "summary": null
      },
      "store": true,
      "temperature": 1.0,
      "text": {
        "format": {
          "type": "text"
        }
      },
      "tool_choice": "auto",
      "tools": [],
      "top_p": 1.0,
      "truncation": "disabled",
      "usage": {
        "input_tokens": 54,
        "input_tokens_details": {
          "cached_tokens": 0
        },
        "output_tokens": 8,
        "output_tokens_details": {
          "reasoning_tokens": 0
        },
        "total_tokens": 62
      },
      "metadata": {}
    }
    {
      "id": "resp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "object": "response",
      "created_at": 1741476542,
      "status": "completed",
      "completed_at": 1741476543,
      "error": null,
      "incomplete_details": null,
      "instructions": null,
      "max_output_tokens": null,
      "model": "openai/gpt-oss-120b",
      "output": [
        {
          "type": "message",
          "id": "msg_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
          "status": "completed",
          "role": "assistant",
          "content": [
            {
              "type": "output_text",
              "text": "The capital of South Korea is Seoul.",
              "annotations": []
            }
          ]
        }
      ],
      "parallel_tool_calls": true,
      "previous_response_id": null,
      "reasoning": {
        "effort": null,
        "summary": null
      },
      "store": true,
      "temperature": 1.0,
      "text": {
        "format": {
          "type": "text"
        }
      },
      "tool_choice": "auto",
      "tools": [],
      "top_p": 1.0,
      "truncation": "disabled",
      "usage": {
        "input_tokens": 54,
        "input_tokens_details": {
          "cached_tokens": 0
        },
        "output_tokens": 8,
        "output_tokens_details": {
          "reasoning_tokens": 0
        },
        "total_tokens": 62
      },
      "metadata": {}
    }
    Code block. Responses API Response Example

    Prompt Guard response

    If the prompt_safety_model parameter is set, the guard model checks the prompt first.

    • safe: model The request is processed unchanged with the model specified in the model parameter.
    • unsafe: Returns a guard result in the following form and the request is terminated.
    Color mode
    {
      "guard_result": "unsafe",
      "categories": ["S1", "S2"],
      "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
      "messages": [
        "Cannot fulfill the request due to violent content.",
        "Cannot respond as it may promote illegal activities."
      ]
    }
    {
      "guard_result": "unsafe",
      "categories": ["S1", "S2"],
      "categories_description": ["Violent Crimes", "Non-Violent Crimes"],
      "messages": [
        "Cannot fulfill the request due to violent content.",
        "Cannot respond as it may promote illegal activities."
      ]
    }
    Code block. Responses API Prompt Guard response Example (unsafe)

    Reasoning configuration

    You can control the model-specific reasoning (inference mode) configuration via the chat_template_kwargs or reasoning parameters.

    • gpt-oss-120b: reasoning parameter’s effort field specifies the inference depth. * (low/medium/high, default medium)
    • Other models: chat_template_kwargs parameter is used, and please refer to Chat Completions API - Reasoning configuration for configuration.
    modelParameterbasic reasoningConfiguration method
    openai/gpt-oss-120breasoningmedium{ “effort”: “low” } / { “effort”: “medium” } (default) / { “effort”: “high” }
    Other modelschat_template_kwargsVaries by modelChat Completions API - Reasoning Configuration see
    Table. Responses API - Model-specific Reasoning Configuration
    Color mode
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/responses \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "openai/gpt-oss-120b",
        "input": "Please solve a complex math problem.",
        "reasoning": {
          "effort": "high"
        }
      }'
    curl -X 'POST' \
       {Simple AI Inference endpoint}/v1/responses \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "openai/gpt-oss-120b",
        "input": "Please solve a complex math problem.",
        "reasoning": {
          "effort": "high"
        }
      }'
    Code block. Responses API Reasoning Setting Example

    Reference

    Tokenize API

    POST /tokenize
    

    Overview

    The Tokenize API converts text to token IDs. Supports two request types: Completion (prompt-based) and Chat (messages-based). Compatible with vLLM’s Tokenize API.

    Request

    Context

    KeyTypeDescriptionExample
    Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
    Request MethodstringHTTP methods used in API requestsPOST
    HeadersobjectHeader information required for the request{ “Content-Type”: “application/json”, “Authorization”: “bearer sai-xxxxxxx…” }
    Body ParametersobjectParameters included in the request body{ “model”: “openai/gpt-oss-120b”, “prompt”: “Hello, world!” }
    Table. Tokenize API - Context

    Path Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Tokenize API - Path Parameters

    Query Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Tokenize API - Query Parameters

    Body Parameters - Common

    NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
    model-stringSpecify the model to use for tokenization“openai/gpt-oss-120b”
    Table. Tokenize API - Body Parameters (Common)

    Body Parameters - Completion method (prompt-based)

    NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
    prompt-stringText to be tokenized“Hello, world!"
    add_special_tokens-booleanIf true, add special tokens (e.g., BOS) to the prompt.truetrue / falsetrue
    return_token_strs-booleanIf true, also return the token string corresponding to the token ID.falsetrue / falsetrue
    Table. Tokenize API - Body Parameters (Completion mode)

    Body Parameters - Chat method (based on messages)

    NameName SubtypeRequiredDescriptionDefault valueBoundary valueExample
    messagesrolestringMessage list containing conversation history[{ “role”: “user”, “content”: “hi” }]
    add_generation_prompt-booleanIf true, add a generation prompt to the chat template. Cannot set to true simultaneously with continue_final_message.truetrue / falsetrue
    continue_final_message-booleanIf true, the last message is formatted in an open form without EOS. Instead of the model starting a new message, it continues that message. Cannot be set to true together with add_generation_prompt.falsetrue / falsefalse
    add_special_tokens-booleanIf true, insert additional special tokens such as BOS in addition to the special tokens added by the chat template. Most models handle special tokens with the chat template, so the default false is recommended.falsetrue / falsefalse
    return_token_strs-booleanIf true, also return the token string corresponding to the token ID.falsetrue / falsetrue
    chat_template-stringJinja template to use for conversion. Provide it if not defined in the tokenizer.null
    chat_template_kwargs-objectAdditional keyword arguments to pass to the template renderernull{ “add_generation_prompt”: true }
    tools-arrayList of tools the model can invokenull
    Table. Tokenize API - Body Parameters (Chat mode)

    Example

    Color mode
    curl -X 'POST' \
      {Simple AI Inference endpoint}/tokenize \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "openai/gpt-oss-120b",
        "prompt": "Hello, world!"
      }'
    curl -X 'POST' \
      {Simple AI Inference endpoint}/tokenize \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "openai/gpt-oss-120b",
        "prompt": "Hello, world!"
      }'
    curl -X 'POST' \
      {Simple AI Inference endpoint}/tokenize \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "openai/gpt-oss-120b",
        "messages": [
          {
            "role": "user",
            "content": "hi"
          }
        ]
      }'
    curl -X 'POST' \
      {Simple AI Inference endpoint}/tokenize \
      -H 'Authorization: bearer sai-xxxxxxx...' \
      -H 'Content-Type: application/json' \
      -d '{
        "model": "openai/gpt-oss-120b",
        "messages": [
          {
            "role": "user",
            "content": "hi"
          }
        ]
      }'
    Code block. Tokenize API Request Example

    Response

    200 OK

    NameTypeDescription
    countintegerNumber of tokenized tokens
    max_model_lenintegerMaximum token length supported by the model
    tokensarrayList of tokenized token IDs
    token_strsarray or nullList of token strings corresponding to token IDs (returned only when return_token_strs is true)
    Table. Tokenize API - 200 OK

    Error Code

    HTTP status codeErrorCode description
    400Bad Request (model field missing, request body missing, etc)
    404Model Not Found (unsupported model)
    500Internal Server Error
    Table. Tokenize API - Error Code

    Example

    Color mode
    {
      "max_model_len": 1024,
      "count": 6,
      "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
      "token_strs": null
    }
    {
      "max_model_len": 1024,
      "count": 6,
      "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
      "token_strs": null
    }
    {
      "max_model_len": 1024,
      "count": 6,
      "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
      "token_strs": ["<|im_start|>", "user", "<|im_sep|>", "hi", "<|im_end|>", ""]
    }
    {
      "max_model_len": 1024,
      "count": 6,
      "tokens": [638357778, 638357778, 399020470, 1618501362, 2382766391, 2765235376],
      "token_strs": ["<|im_start|>", "user", "<|im_sep|>", "hi", "<|im_end|>", ""]
    }
    Code block. Tokenize API Response Example

    Reference

    Models API

    GET /v1/models
    

    Overview

    The Models API returns a list of Simple AI Inference models. Compatible with OpenAI’s Models API.

    Request

    Context

    KeyTypeDescriptionExample
    Base URLstringSimple AI Inference URL for API requestsSimple AI Inference endpoint
    Request MethodstringHTTP methods used in API requestsGET
    HeadersobjectHeader information required for the request{ “Authorization”: “bearer sai-xxxxxxx…” }
    Body Parameters--Since it is a GET request, there is no body.
    Table. Models API - Context

    Path Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Models API - Path Parameters

    Query Parameters

    NametypeRequiredDescriptionDefault valueBoundary valueExample
    None
    Table. Models API - Query Parameters

    Body Parameters

    Since it is a GET request, there is no body.

    Example

    Color mode
    curl -X 'GET' \
      {Simple AI Inference endpoint}/v1/models \
      -H 'Authorization: bearer sai-xxxxxxx...'
    curl -X 'GET' \
      {Simple AI Inference endpoint}/v1/models \
      -H 'Authorization: bearer sai-xxxxxxx...'
    Code block. Models API Request Example

    Response

    200 OK

    NameTypeDescription
    objectstringResponse object’s type (“list”)
    dataarrayModel object list
    data[].idstringModel identifier
    data[].objectstringObject type (“model”)
    data[].createdintegerModel creation time (Unix timestamp, in seconds)
    data[].owned_bystringModel owner
    Table. Models API - 200 OK

    Error Code

    HTTP status codeErrorCode description
    401Unauthorized (apikey missing or invalid)
    500Internal Server Error
    Table. Models API - Error Code

    Example

    Color mode
    {
      "data": [
        {
          "id": "openai/gpt-oss-120b",
          "created": 1780979126,
          "object": "model",
          "owned_by": "SCP Simple AI Inference"
        },
        {
          "id": "Qwen/Qwen3-VL-Embedding-8B",
          "created": 1781512915,
          "object": "model",
          "owned_by": "SCP Simple AI Inference"
        },
        {
          "id": "Qwen/Qwen3-VL-Reranker-8B",
          "created": 1781512915,
          "object": "model",
          "owned_by": "SCP Simple AI Inference"
        }
      ],
      "object": "list"
    }
    {
      "data": [
        {
          "id": "openai/gpt-oss-120b",
          "created": 1780979126,
          "object": "model",
          "owned_by": "SCP Simple AI Inference"
        },
        {
          "id": "Qwen/Qwen3-VL-Embedding-8B",
          "created": 1781512915,
          "object": "model",
          "owned_by": "SCP Simple AI Inference"
        },
        {
          "id": "Qwen/Qwen3-VL-Reranker-8B",
          "created": 1781512915,
          "object": "model",
          "owned_by": "SCP Simple AI Inference"
        }
      ],
      "object": "list"
    }
    Code block. Models API Response Example

    Reference