API Reference Overview

The API Reference supported by AIOS is as follows.

Rerank API

POST /rerank, /v1/rerank, /v2/rerank

Overview

The Rerank API applies an embedding model or cross-encoder model to predict the relevance between a single query and each item in a document list. Generally, the score of a sentence pair represents the similarity between the two sentences on a scale of 0 to 1.

• Embedding-based model: Converts the query and document into vectors and measures the similarity between the vectors (e.g., cosine similarity) to calculate the score.
• Reranker (Cross-Encoder) based model: Evaluates the query and document as a pair.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   'https://aios.private.kr-west1.e.samsungsdscloud.com/rerank' \ 
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "sds/bge-reranker-v2-m3",

Here is the translation of the given text:

"query": "What is the capital of France?",
"documents": [
  "The capital of France is Paris.",
  "France capital city is known for the Eiffel Tower.",
  "Paris is located in the north-central part of France."
],
"top_n": 2, 
"truncate_prompt_tokens": 512

}

Response

200 OK

Error Code

Example

{
  "id": "rerank-scp-aios-rerank",
  "model": "sds/sds/bge-m3",
  "usage": {
    "total_tokens": 65
  },
  "results": [
    {
      "index": 0,
      "document": {
        "text": "The capital of France is Paris."
      },
      "relevance_score": 0.8291233777999878
    },
    {
      "index": 1,
      "document": {
        "text": "France capital city is known for the Eiffel Tower."
      },
      "relevance_score": 0.6996355652809143
    }
  ]
}

Reference

• Rerank API vLLM documentation

Score API

POST /score, /v1/score

Overview

The Score API predicts the similarity between two sentences. This API uses one of two models to calculate the score:

• Reranker (Cross-Encoder) model: Takes a pair of sentences as input and directly predicts the similarity score.
• Embedding model: Generates embedding vectors for each sentence and calculates the cosine similarity to derive the score.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
  'https://aios.private.kr-west1.e.samsungsdscloud.com/score' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "sds/bge-reranker-v2-m3",
  "encoding_format": "float",
"text_1": [
  "What is the largest planet in the solar system?",
  "What is the chemical symbol for water?"
],
"text_2": [
  "Jupiter is the largest planet in the solar system.",
  "The chemical symbol for water is H₂O."
]
}'

Response

200 OK

Error Code

Example

{
  "id": "score-scp-aios-score",
  "object": "list",
  "created": 1748574112,
  "model": "sds/bge-reranker-v2-m3",
  "data": [
    {

Here is the translated text:

  "index": 0,
  "object": "score",
  "score": 1.0
},
{
  "index": 1,
  "object": "score",
  "score": 1.0
}

], “usage”: { “prompt_tokens”: 53, “total_tokens”: 53, “completion_tokens”: 0, “prompt_tokens_details”: null } }

## Reference
* [Score API vLLM documentation](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#score-api_1)




# Chat Completions API

```python
POST /v1/chat/completions

Overview

Chat Completions API is compatible with OpenAI’s Completions API and can be used with the OpenAI Python client.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   'https://aios.private.kr-west1.e.samsungsdscloud.com/v1/chat/completions' \
  -H 'accept: application/json' \

200 OK

Error Code

Example

{
  "id": "chatcmpl-scp-aios-chat-completions",
  "object": "chat.completion",
  "created": 1749702816,
  "model": "meta-llama/Meta-Llama-3.3-70B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning_content": null,
        "content": "The capital of Korea is Seoul.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 54,
    "total_tokens": 62,
    "completion_tokens": 8,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null
}

Reference

• Chat Completions API vLLM documentation
• Chat Completions API OpenAI documentation POST /v1/completions

## Overview

Completions API is compatible with OpenAI's Completions API and can be used with the OpenAI Python client.


## Request 

### Context







  



Key
Type
Description
Example




Base URL
string
API request URL for AIOS
application/json


Request Method
string
HTTP method used for the API request
POST


Headers
object
Header information required for the request
{ “accept”: “application/json”, “Content-Type”: “application/json” }


Body Parameters
object
Parameters included in the request body
’{“model”: “meta-llama/Llama-3.3-70B-Instruct”, “prompt” : “hello”, “stream”: “true”}’




  Table. Completions API - Context





### Path Parameters







  



Name
type
Required
Description
Default value
Boundary value
Example




None










  Table. Completions API - Path Parameters





### Query Parameters







  



Name
type
Required
Description
Default value
Boundary value
Example




None










  Table. Completions API - Query Parameters





### Body Parameters







  



Name
Name Sub
type
Required
Description
Default value
Boundary value
Example




model
-
string
✅
Model used to generate the response


“meta-llama/Llama-3.3-70B-Instruct”


prompt
-
array, string
✅
User input text


""


echo
-
boolean
❌
Whether to include the input text in the output
false
true/false
true


frequency_penalty
-
number
❌
Adjust the penalty for repeating tokens
0
-2.0 ~ 2.0
0.5


logit_bias
-
object
❌
Adjust the probability of specific tokens (e.g., { “100”: 2.0 })
null
Key: token ID, Value: -100~100
{ “100”: 2.0 }


logprobs
-
integer
❌
Return the probabilities of the top logprobs tokens
null
1 ~ 5
5


max_completion_tokens
-
integer
❌
Limit the maximum number of generated tokens
None
0~model maximum value
100


max_tokens (Deprecated)
-
integer
❌
Limit the maximum number of generated tokens
None
0~model maximum value
100


n
-
integer
❌
Specify the number of responses to generate
1

3


presence_penalty
-
number
❌
Adjust the penalty for tokens already present in the text
0
-2.0 ~ 2.0
1.0


seed
-
integer
❌
Specify a seed value for randomness control
None




stop
-
string / array / null
❌
Stop generating when a specific string is encountered
null

"\n"


stream
-
boolean
❌
Whether to return the results in a streaming manner
false
true/false
true


stream_options
include_usage, continuous_usage_stats
object
❌
Control streaming options (e.g., include usage statistics)
null

{ “include_usage”: true }


temperature
-
number
❌
Control the creativity of the generated response (higher means more random)
1
0.0 ~ 1.0
0.7


top_p
-
number
❌
Limit the sampling probability of tokens (higher means more tokens considered)
1
0.0 ~ 1.0
0.9




  Table. Completions API - Body Parameters






### Example

```python
curl -X 'POST' \
   'https://aios.private.kr-west1.e.samsungsdscloud.com/v1/completions' \ 
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "meta-llama/Meta-Llama-3.3-70B-Instruct",
    "prompt": "What is the capital of Korea?",
    "temperature": 0.7
  }'

Response

200 OK

| usage.completion_tokens	| number		| Number of tokens used in the generated response |
| usage.prompt_tokens_details	| object		| Details of prompt token usage |
<div class="figure-caption">
  Table. Completions API - 200 OK
</div>

Error Code

Example

{
  "id": "cmpl-scp-aios-completions",
  "object": "text_completion",
  "created": 1749702612,
  "model": "meta-llama/Meta-Llama-3.3-70B-Instruct",
  "choices": [
    {
      "index": 0,
      "text": " \nOur capital city is Seoul. \n\nA. 1\nB. ",
      "logprobs": null,
      "finish_reason": "length",
      "stop_reason": null,
      "prompt_logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 25,
    "completion_tokens": 16,
    "prompt_tokens_details": null
  }
}

Reference

• Completions API vLLM documentation
• Completions API OpenAI documentation

Embedding API

POST /v1/embeddings

Overview

The Embedding API converts text into high-dimensional vectors (embeddings) that can be used for various natural language processing (NLP) tasks, such as calculating text similarity, clustering, and search.

Request

Context

Path Parameters

Query Parameters

Body Parameters

Example

curl -X 'POST' \
   'https://aios.private.kr-west1.e.samsungsdscloud.com/v1/embedding' \ 
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "sds/bge-m3",
    "input": "What is the capital of France?",
	"encoding_format": "float"
  }'

Response

200 OK

Error Code

Example

{
  "id":"embd-scp-aios-embeddings",
  "object":"list","created":1749035024,
  "model":"sds/bge-m3",
  "data":[
    {
      "index":0,
      "object":"embedding",
      "embedding":
      [0.01319122314453125,0.057220458984375,-0.028533935546875,-0.0008697509765625,-0.01422119140625,0.033416748046875,-0.0062408447265625,-0.04364013671875,-0.004497528076171875,0.0008072853088378906,-0.0193328857421875,0.041168212890625,-0.019317626953125,-0.0188751220703125,-0.047088623046875,
      -0 ....(omitted)

      -0.05706787109375,-0.0147705078125]
    }
  ],
  "usage":
  {
    "prompt_tokens":9,
    "total_tokens":9,
    "completion_tokens":0,
    "prompt_tokens_details":null
  }
}

Reference

• Embedding API vLLM documentation
• Embedding API OpenAI documentation