Using NVSwitch on GPU Server

After creating a GPU Server, you can enable the NVSwitch feature in the GPU Server’s VM (Guest OS) and quickly use P2P (GPU to GPU) communication between GPUs.

Exploring NVIDIA NVSwitch for Multi GPU

NVIDIA A100 GPU server is a multi-GPU based on the NVIDIA Ampere architecture, with 8 Ampere 80 GB GPUs installed on the baseboard. The GPUs installed on the baseboard are connected to 6 NVSwitches via NVLink ports. Communication between GPUs on the baseboard is done using the full 600 GBps bandwidth. For this reason, the 8 GPUs installed on the A100 GPU server can be connected and operated like one, thereby maximizing GPU-to-GPU usage.

• NVLink(25 GBps) 12 Lane 8 GPU configuration

• NVSwitch(600 GBps) 6 units 8 GPU configuration diagram

Create GPU NVSwitch

To use the GPU NVSwitch feature, create a GPU Server service on the Samsung Cloud Platform, create a VM Instance (GuestOS) with 8 A100 GPUs assigned, and activate the Fabricmanager.

NVSwitch Installation and Operation Check (Fabric Manager Activation)

To operate NVSwitch, install Fabricmanager on the GPU Instance and follow the next procedure.

1. Install NVIDIA GPU Driver (470.52.02 Version) on the GPU server.

   Sample Code Download

   Copy Code

   Color mode

   $ add-apt-repository ppa:graphics-drivers/ppa
   $ apt-get update
   $ apt-get install nvidia-driver-470-server

   $ add-apt-repository ppa:graphics-drivers/ppa
   $ apt-get update
   $ apt-get install nvidia-driver-470-server

2. Install and run NVIDIA Fabric Manager (470 Version) on the GPU server (For NVSwitch).

   Sample Code Download

   Copy Code

   Color mode

   $ apt-get install cuda-drivers-fabricmanager-470
   $ systemctl enable nvidia-fabricmanager
   $ systemctl start nvidia-fabricmanager

   $ apt-get install cuda-drivers-fabricmanager-470
   $ systemctl enable nvidia-fabricmanager
   $ systemctl start nvidia-fabricmanager

3. Check the status of NVIDIA Fabric Manager running on the GPU server.

   • Normal operation indicates active (running)
     Sample Code Download

     Copy Code

     Color mode

     $ systemctl status nvidia-fabricmanager

     $ systemctl status nvidia-fabricmanager

4. Check the NVSwitch operation status on the GPU server.
   • Normal operation indicates NV12
     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi topo --matrix

     $ nvidia-smi topo --matrix