Overview
Service Overview
GPU Server is a virtualized computing service that allows you to freely allocate and use as much infrastructure resources provided by the server such as CPU, GPU, and memory as needed at the desired time without having to purchase them individually. It is suitable for tasks that require fast computing speed such as AI model experimentation, prediction, and inference in a cloud environment, and allows you to flexibly select and use resources with optimized performance according to task type and scale. GPU Server provides the following features:
Provided Features
- GPU Server Management: Users can directly manage creation, deletion, and changes as a Self Service from GPU Server provisioning to monitoring and billing through a web-based Console.
- Provisioning by GPU Quantity: You can configure a virtual server by freely selecting the quantity of H100/A100 GPUs according to project purpose and scale.
- High Performance GPU Provision: Provides high-performance GPU servers at the physical server level using the Pass-through method.
- Storage Connection: Provides additional connected storage besides OS disks. You can connect and use Block Storage, File Storage, and Object Storage.
- Strong Security Application: Protects servers safely by controlling Inbound/Outbound traffic exchanged with external internet or other VPCs (Virtual Private Cloud) through the Security Group service.
- Monitoring: You can check monitoring information such as CPU, Memory, Disk, and GPU status corresponding to computing resources through the Cloud Monitoring service.
- Network Setting Management: The server’s subnet/IP can be easily changed from the values set at initial creation. Provides management functionality that allows you to set use/terminate NAT IP as needed.
- Key Pair Method: Provides a Key Pair method instead of ID/PW access for secure OS access.
- Image Management: You can create and manage Custom Images and provides sharing functionality between projects.
- ServiceWatch Service Integration Provision: You can monitor data through the ServiceWatch service.
Components
GPU Server provides GPU, NVSwitch, and NVLink on top of virtualized computing resources.
- NVSwitch can be activated and used only for instance types that allocate 8 GPUs to a single GPU Server.
Specifications by GPU Type
GPU (Graphic Processing Unit) plays the role of performing calculations necessary to create images that make up the computer screen and is specialized for parallel processing, enabling it to process large amounts of data quickly, handling large-scale parallel operations such as artificial intelligence (AI) and data analysis.
The following are the specifications of GPU types provided by the GPU Server service.
| Item | A100 Type | H100 Type |
|---|---|---|
| Service Provision Method | Pass-through | Pass-through |
| GPU Architecture | NVIDIA Ampere | NVIDIA Hopper |
| GPU Memory | 80 GB | 80 GB |
| GPU Transistors | 54 billion 7N TSMC | 80 billion 4N TSMC |
| FP16 Tensor Core (Dense) | 312 TFLOPs | 989 TFLOPs |
| FP8 Tensor Core (Dense) | Not supported | 1,979 TFLOPs |
| FP4 Tensor Core (Dense) | Not supported | Not supported |
| GPU Memory Bandwidth | 2,039 GB/s HBM2e | 3,352 GB/s HBM3 |
| NVLink Performance | NVLink 3 | NVLink 4 |
| NVLink Signaling Rate | 25 GB/s (x12) | 25 GB/s (x18) |
| NVSwitch GPU-to-GPU Bandwidth | 600 GB/s | 900 GB/s |
| Total NVSwitch Aggregate Bandwidth | 4.8 TB/s | 7.2 TB/s |
Server Type
The server types provided by GPU Server are as follows. For a detailed description of the server types provided by GPU Server, see GPU Server Server Types.
| Item | Server Type | CPU vCore | Memory(GB) | GPU Quantity |
|---|---|---|---|---|
| GPU-A100-1 | g1v16a1 | 16 | 234 | 1 |
| GPU-A100-1 | g1v32a2 | 32 | 468 | 2 |
| GPU-A100-1 | g1v64a4 | 64 | 936 | 4 |
| GPU-A100-1 | g1v128a8 | 128 | 1872 | 8 |
| GPU-H100-2 | g2v12h1 | 12 | 234 | 1 |
| GPU-H100-2 | g2v24h2 | 24 | 468 | 2 |
| GPU-H100-2 | g2v48h4 | 48 | 936 | 4 |
| GPU-H100-2 | g2v96h8 | 96 | 1872 | 8 |
OS and GPU Driver Version
The operating systems (OS) supported by GPU Server are as follows:
| OS | OS Version | GPU Driver Version |
|---|---|---|
| Ubuntu | 22.04 | 535.183.06 |
| Ubuntu | 24.04 | 570.195.03 |
| RHEL | 8.10 | 535.183.06 |
Prerequisite Services
This is a service that must be pre-installed before creating this service. Please prepare by referring to the user guide provided in advance.
| Service Category | Service | Description |
|---|---|---|
| Networking | VPC | Service that provides independent virtual networks in cloud environment |
| Networking | Security Group | Virtual firewall that controls server traffic |