How-to guides

• 1: Image Management
• 2: Using Multi-instance GPU in GPU Server
• 3: Using NVSwitch on GPU Server
• 4: Keypair Management
• 5: ServiceWatch Agent Install

1 - Image Management

Users can enter the required information for the Image service within the GPU Server service and select detailed options through the Samsung Cloud Platform Console to create the respective service.

Image Generation

You can create an image of a running GPU Server. To create an image of a GPU Server, please refer to Create Image.

Image Check detailed information

Image service allows you to view and edit the full resource list and detailed information. Image detail page consists of detail information, tags, operation history tabs.

To view detailed information of the Image service, follow the steps below.

1. All Services > Compute > GPU Server Click the menu. Navigate to the Service Home page of GPU Server.
2. Click the Image menu on the Service Home page. Go to the Image List page.
3. Image List page, click the resource to view detailed information. You will be taken to the Image Detail page.
   • Image Details page displays status information and additional feature information, and consists of Detail Information, Tag, Work History tabs.

     Category	Detailed description
     Image Status	User-generated Image’s status Active: Available state Queued: When creating Image, Image is uploaded and waiting for processing Importing: When creating Image, Image is uploaded and being processed
     Share to another Account	Image can be shared to another Account Image’s Visibility must be in Shared state to be able to share to another Account
     Delete Image	Button to delete the Image If the Image is deleted, it cannot be recovered

     Table. GPU Server Image status information and additional features

Detailed Information

Image list page allows you to view detailed information of the selected resource and edit the information if necessary.

Tag

On the Image list page, you can view the tag information of the selected resource, and you can add, modify, or delete it.

Work History

You can view the operation history of the selected resource on the Image list page.

Image Resource Management

Describes the control and management functions of the generated image.

Share to another account

To share the Image with another Account, follow the steps below.

1. Log in to the shared Account and click the All Services > Compute > GPU Server menu. Navigate to the GPU Server’s Service Home page.
2. Click the Image menu on the Service Home page. Go to the Image list page.
3. Click the Image to control on the Image List page. It moves to the Image Detail page.
4. Click the Share to another Account button. It navigates to the Share image to another Account page.
   • Share to another Account feature allows you to share the Image with another Account. To share the Image with another Account, the Image’s Visibility must be Shared.
5. Share images to another Account On the page, enter the required information and click the Complete button.

   Category	Required or not	Detailed description
   Image Name	-	Name of the image to share Input not allowed
   Image ID	-	Image ID to share Input not allowed
   Shared Account ID	Required	Enter another Account ID to share Enter within 64 characters using English letters, numbers, special characters-

   Table. Required input items for sharing images to another Account
6. Image Details page’s sharing status can be checked for information.
   • At the initial request, the status is Pending, and when approval is completed in the account to be shared, it changes to Accepted.

Receive sharing from another account

To receive an Image shared from another Account, follow the steps below.

1. Log in to the account to be shared and click the 모든 서비스 > Compute > GPU Server menu. Navigate to the Service Home page of the GPU Server.
2. Click the Image menu on the Service Home page. Go to the Image list page.
3. Image List on the page click the Get Image Share button. Go to the Get Image Share popup window.
4. Receive Image Share In the popup window, enter the resource ID of the Image you want to receive, and click the Confirm button.
5. When image sharing is completed, you can check the shared Image in the Image list.

Image Delete

You can delete unused Images. However, once an Image is deleted it cannot be recovered, so you should fully consider the impact before proceeding with the deletion.

To delete the image, follow the steps below.

1. All Services > Compute > GPU Server Click the menu. Navigate to the GPU Server’s Service Home page.
2. Click the Image menu on the Service Home page. Go to the Image List page.
3. On the Image list page, select the resource to delete and click the Delete button.

• Image list page, select multiple Image check boxes, and click the Delete button at the top of the resource list.

4. When deletion is complete, check on the Image List page whether the resource has been deleted.

2 - Using Multi-instance GPU in GPU Server

After creating a GPU Server, you can enable the MIG (Multi-instance GPU) feature on the GPU Server’s VM (Guest OS) and create an instance to use it.

Multi-instance GPU (NVIDIA A100) Overview

NVIDIA A100 is a Multi-instance GPU (MIG) based on the NVIDIA Ampere architecture, which can be securely divided into up to 7 independent GPU instances to operate CUDA (Compute Unified Device Architecture) applications. The NVIDIA A100 provides independent GPU resources to multiple users by allocating computing resources in a way optimized for GPU usage while utilizing high-bandwidth memory (HBM) and cache. Users can maximize GPU utilization by utilizing workloads that have not reached the maximum computing capacity of the GPU through parallel execution of each workload.

Using Multi-instance GPU Feature

To use the multi-instance GPU feature, you must create a GPU Server service on the Samsung Cloud Platform and then create a VM Instance (GuestOS) with an A100 GPU assigned. After completing the GPU Server creation, you can follow the MIG application order and MIG release order below to apply it.

MIG Application and Usage

To activate MIG and create an instance to assign a task, follow these steps.

MIG Activation

1. Check the GPU status on the VM Instance (GuestOS) before applying MIG.

   • MIG mode is Disabled status, please check.
     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi
     Mon Sep 27 08:37:08 2021
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
     |-------------------------------+----------------------+----------------------|
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  NVDIA A100-SXM...  Off   | 00000000:05:00.0 Off |                    0 |
     | N/A   32C   P0    59W / 400W  |      0MiB / 81251MiB |      0%      Default |
     |                               |                      |             Disabled |
     +-------------------------------+----------------------+----------------------+
     
     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     | No running processes found                                                  |
     +-----------------------------------------------------------------------------+

     $ nvidia-smi
     Mon Sep 27 08:37:08 2021
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
     |-------------------------------+----------------------+----------------------|
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  NVDIA A100-SXM...  Off   | 00000000:05:00.0 Off |                    0 |
     | N/A   32C   P0    59W / 400W  |      0MiB / 81251MiB |      0%      Default |
     |                               |                      |             Disabled |
     +-------------------------------+----------------------+----------------------+
     
     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     | No running processes found                                                  |
     +-----------------------------------------------------------------------------+

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi –L
     GPU 0: NVIDIA A100-SXM-80GB (UUID: GPU-c956838f-494a-92b2-6818-56eb28fe25e0)

     $ nvidia-smi –L
     GPU 0: NVIDIA A100-SXM-80GB (UUID: GPU-c956838f-494a-92b2-6818-56eb28fe25e0)

2. In the VM Instance(GuestOS), enable MIG for each GPU and reboot the VM Instance.

   Sample Code Download

   Copy Code

   Color mode

   $ nvidia-smi –I 0 –mig 1
   Enabled MIG mode for GPU 00000000:05:00.0
   All done.
   
   # reboot

   $ nvidia-smi –I 0 –mig 1
   Enabled MIG mode for GPU 00000000:05:00.0
   All done.
   
   # reboot

Warning: MIG mode is in pending enable state for GPU 00000000:05:00.0: In use by another client. 00000000:05:00.0 is currently being used by one or more other processes (e.g. CUDA application or a monitoring application such as another instance of nvidia-smi).

# systemctl stop nvsm
# systemctl stop dcgm

3. Check the GPU status after applying MIG in the VM Instance(GuestOS).
   • MIG mode must be in Enabled state.
     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi
     Mon Sep 27 09:44:33 2021
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
     |-------------------------------+----------------------+----------------------|
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  NVDIA A100-SXM...  Off   | 00000000:05:00.0 Off |                   On |
     | N/A   32C   P0    59W / 400W  |      0MiB / 81251MiB |      0%      Default |
     |                               |                      |              Enabled |
     +-------------------------------+----------------------+----------------------+
     +-----------------------------------------------------------------------------+
     | MIG devices:                                                                |
     +-----------------------------------------------------------------------------+
     |  GPU  GI  CI  MIG |        Memory-Usage |        Vol|        Shared         |
     |       ID  ID  Dev |          BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
     |                   |                     |        ECC|                       |
     |=============================================================================|
     | No MIG devices found                                                        |
     +-----------------------------------------------------------------------------+
     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     | No running processes found                                                  |
     +-----------------------------------------------------------------------------+

     $ nvidia-smi
     Mon Sep 27 09:44:33 2021
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
     |-------------------------------+----------------------+----------------------|
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  NVDIA A100-SXM...  Off   | 00000000:05:00.0 Off |                   On |
     | N/A   32C   P0    59W / 400W  |      0MiB / 81251MiB |      0%      Default |
     |                               |                      |              Enabled |
     +-------------------------------+----------------------+----------------------+
     +-----------------------------------------------------------------------------+
     | MIG devices:                                                                |
     +-----------------------------------------------------------------------------+
     |  GPU  GI  CI  MIG |        Memory-Usage |        Vol|        Shared         |
     |       ID  ID  Dev |          BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
     |                   |                     |        ECC|                       |
     |=============================================================================|
     | No MIG devices found                                                        |
     +-----------------------------------------------------------------------------+
     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     | No running processes found                                                  |
     +-----------------------------------------------------------------------------+

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi –L
     GPU 0: NVIDIA A100-SXM-80GB (UUID: GPU-c956838f-494a-92b2-6818-56eb28fe25e0)

     $ nvidia-smi –L
     GPU 0: NVIDIA A100-SXM-80GB (UUID: GPU-c956838f-494a-92b2-6818-56eb28fe25e0)

GPU Instance Creation

After activating MIG and checking the status, you can create a GPU Instance.

1. Check the list of MIG GPU instance profiles that can be created.

   Sample Code Download

   Copy Code

   Color mode

   $ nvidia-smi mig -i [GPU ID] -lgip

   $ nvidia-smi mig -i [GPU ID] -lgip

   Sample Code Download

   Copy Code

   Color mode

   $ nvidia-smi mig -i 0 -lgip
   +-----------------------------------------------------------------------------+
   | GPU instance profiles:                                                      |
   | GPU   Name             ID    Instances   Memory     P2P    SM    DEC   ENC  |
   |                              Free/Total   GiB              CE    JPEG  OFA  |
   |=============================================================================|
   |   0 MIG 1g.10gb        19    7/7         9.50       No     14     0     0   |
   |                                                             1     0     0   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 1g.10gb+me     20    1/1         9.50       No     14     0     0   |
   |                                                             1     1     1   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 2g.20gb        14    3/3         19.50      No     28     1     0   |
   |                                                             2     0     0   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 3g.40gb         9    2/2         39.50      No     42     2     0   |
   |                                                             3     0     0   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 4g.40gb         5    1/1         39.50      No     56     2     0   |
   |                                                             4     0     0   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 7g.80gb         0    1/1         79.25      No     98     0     0   |
   |                                                             7     1     1   |
   +-----------------------------------------------------------------------------+

   $ nvidia-smi mig -i 0 -lgip
   +-----------------------------------------------------------------------------+
   | GPU instance profiles:                                                      |
   | GPU   Name             ID    Instances   Memory     P2P    SM    DEC   ENC  |
   |                              Free/Total   GiB              CE    JPEG  OFA  |
   |=============================================================================|
   |   0 MIG 1g.10gb        19    7/7         9.50       No     14     0     0   |
   |                                                             1     0     0   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 1g.10gb+me     20    1/1         9.50       No     14     0     0   |
   |                                                             1     1     1   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 2g.20gb        14    3/3         19.50      No     28     1     0   |
   |                                                             2     0     0   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 3g.40gb         9    2/2         39.50      No     42     2     0   |
   |                                                             3     0     0   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 4g.40gb         5    1/1         39.50      No     56     2     0   |
   |                                                             4     0     0   |
   +-----------------------------------------------------------------------------+
   |   0 MIG 7g.80gb         0    1/1         79.25      No     98     0     0   |
   |                                                             7     1     1   |
   +-----------------------------------------------------------------------------+

2. Check after creating the MIG GPU Instance.

   • GPU Instance creation

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i [GPU ID] -cgi [Profile ID]

     $ nvidia-smi mig -i [GPU ID] -cgi [Profile ID]

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i 0 -cgi 0
     Successfully created GPU instance ID 0 on GPU 0 using profile MIG 7g.80gb (ID 0)

     $ nvidia-smi mig -i 0 -cgi 0
     Successfully created GPU instance ID 0 on GPU 0 using profile MIG 7g.80gb (ID 0)

   • GPU Instance check

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i [GPU ID] -lgi

     $ nvidia-smi mig -i [GPU ID] -lgi

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i 0 -lgi
     +--------------------------------------------------------+
     | GPU instances:                                         |
     | GPU   Name               Profile  Instance  Placement  |
     |                            ID       ID      Start:Size |
     |========================================================|
     |   0  MIG 7g.80gb            0        0         0:8     |
     +--------------------------------------------------------+

     $ nvidia-smi mig -i 0 -lgi
     +--------------------------------------------------------+
     | GPU instances:                                         |
     | GPU   Name               Profile  Instance  Placement  |
     |                            ID       ID      Start:Size |
     |========================================================|
     |   0  MIG 7g.80gb            0        0         0:8     |
     +--------------------------------------------------------+

Compute Instance Creation

If you have created a GPU Instance, you can create a Compute Instance.

1. Check the MIG Compute Instance profile that can be created.

   Sample Code Download

   Copy Code

   Color mode

   $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] -lcip

   $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] -lcip

   Sample Code Download

   Copy Code

   Color mode

   $ nvidia-smi mig -i 0 -gi 0 -lcip
   +---------------------------------------------------------------------------------+
   | Compute instance profiles:                                                      |
   | GPU     GPU     Name            Profile  Instances   Exclusive      Shared      |
   | GPU   Instance                     ID    Free/Total     SM       DEC  ENC  OFA  |
   |         ID                                                       CE   JPEG      |
   |=================================================================================|
   |   0      0      MIG 1c.7g.80gb     0      7/7           14       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+
   |   0      0      MIG 2c.7g.80gb     1      3/3           28       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+
   |   0      0      MIG 3c.7g.80gb     2      2/2           42       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+
   |   0      0      MIG 4c.7g.80gb     3      1/1           56       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+
   |   0      0      MIG 7g.80gb        4*     1/1           98       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+

   $ nvidia-smi mig -i 0 -gi 0 -lcip
   +---------------------------------------------------------------------------------+
   | Compute instance profiles:                                                      |
   | GPU     GPU     Name            Profile  Instances   Exclusive      Shared      |
   | GPU   Instance                     ID    Free/Total     SM       DEC  ENC  OFA  |
   |         ID                                                       CE   JPEG      |
   |=================================================================================|
   |   0      0      MIG 1c.7g.80gb     0      7/7           14       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+
   |   0      0      MIG 2c.7g.80gb     1      3/3           28       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+
   |   0      0      MIG 3c.7g.80gb     2      2/2           42       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+
   |   0      0      MIG 4c.7g.80gb     3      1/1           56       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+
   |   0      0      MIG 7g.80gb        4*     1/1           98       5    0    1    |
   |                                                                  7    1         |
   +---------------------------------------------------------------------------------+

2. Create and check the MIG Compute Instance.

   • MIG Compute Instance creation
     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] -cci [Compute Profile ID]

     $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] -cci [Compute Profile ID]

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i 0 -gi 0 -cci 4
     Successfully created compute instance ID 0 on GPU instance ID 0 using profile MIG 7g.80gb(ID 4)

     $ nvidia-smi mig -i 0 -gi 0 -cci 4
     Successfully created compute instance ID 0 on GPU instance ID 0 using profile MIG 7g.80gb(ID 4)

   • MIG Compute Instance check
     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] –lci

     $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] –lci

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i 0 -gi 0 –lci
     +-----------------------------------------------------------------+
     | Compute instance profiles:                                      |
     | GPU     GPU     Name            Profile  Instances   Placement  |
     | GPU   Instance                     ID      ID        Start:Size |
     |         ID                                                      |
     |=================================================================|
     |   0      0      MIG 7g.80gb         4       0            0:7    |
     +-----------------------------------------------------------------+

     $ nvidia-smi mig -i 0 -gi 0 –lci
     +-----------------------------------------------------------------+
     | Compute instance profiles:                                      |
     | GPU     GPU     Name            Profile  Instances   Placement  |
     | GPU   Instance                     ID      ID        Start:Size |
     |         ID                                                      |
     |=================================================================|
     |   0      0      MIG 7g.80gb         4       0            0:7    |
     +-----------------------------------------------------------------+

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi –L
     GPU 0: NVIDIA A100-SXM-80GB (UUID: GPU-c956838f-494a-92b2-6818-56eb28fe25e0)
       MIG 7g.80gb     Device  0: (UUID: MIG-53e20040-758b-5ecb-948e-c626d03a9a32)

     $ nvidia-smi –L
     GPU 0: NVIDIA A100-SXM-80GB (UUID: GPU-c956838f-494a-92b2-6818-56eb28fe25e0)
       MIG 7g.80gb     Device  0: (UUID: MIG-53e20040-758b-5ecb-948e-c626d03a9a32)

     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi
     Mon Sep 27 09:52:17 2021
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
     |-------------------------------+----------------------+----------------------|
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  NVDIA A100-SXM...  Off   | 00000000:05:00.0 Off |                   On |
     | N/A   32C   P0    49W / 400W  |      0MiB / 81251MiB |     N/A      Default |
     |                               |                      |              Enabled |
     +-------------------------------+----------------------+----------------------+
     
     +-----------------------------------------------------------------------------+
     | MIG devices:                                                                |
     +-----------------------------------------------------------------------------+
     |  GPU  GI  CI  MIG |        Memory-Usage |        Vol|        Shared         |
     |       ID  ID  Dev |          BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
     |                   |                     |        ECC|                       |
     |=============================================================================|
     |   0    0   0    0 |     0MiB / 81251MiB | 98      0 |  7   0    5    1    1 |
     |                   |     1MiB / 13107... |           |                       |
     +-----------------------------------------------------------------------------+
     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     | No running processes found                                                  |
     +-----------------------------------------------------------------------------+

     $ nvidia-smi
     Mon Sep 27 09:52:17 2021
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
     |-------------------------------+----------------------+----------------------|
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  NVDIA A100-SXM...  Off   | 00000000:05:00.0 Off |                   On |
     | N/A   32C   P0    49W / 400W  |      0MiB / 81251MiB |     N/A      Default |
     |                               |                      |              Enabled |
     +-------------------------------+----------------------+----------------------+
     
     +-----------------------------------------------------------------------------+
     | MIG devices:                                                                |
     +-----------------------------------------------------------------------------+
     |  GPU  GI  CI  MIG |        Memory-Usage |        Vol|        Shared         |
     |       ID  ID  Dev |          BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
     |                   |                     |        ECC|                       |
     |=============================================================================|
     |   0    0   0    0 |     0MiB / 81251MiB | 98      0 |  7   0    5    1    1 |
     |                   |     1MiB / 13107... |           |                       |
     +-----------------------------------------------------------------------------+
     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     | No running processes found                                                  |
     +-----------------------------------------------------------------------------+

Using MIG

1. Use the MIG Instance to perform the Job.
   • Work execution example
     Sample Code Download

     Copy Code

     Color mode

     $ docker run --gpus '"device=[GPU ID]:[MIG ID]"' -rm nvcr.io/nvidia/cuda nvidia-smi

     $ docker run --gpus '"device=[GPU ID]:[MIG ID]"' -rm nvcr.io/nvidia/cuda nvidia-smi

     You can check an example of how to perform the task as follows.
     Sample Code Download

     Copy Code

     Color mode

     $ docker run --gpus '"device=0:0"' -rm -it --network=host --shm-size=1g --ipc=host -v /root/.ssh/:/root/.ssh
     
     ================
     == TensorFlow ==
     ================
     
     NVIDIA Release 21.08-tf1 (build 26012104)
     TensorFlow Version 1.15.5
     
     Container image Copyright (c) 2021, NVIDIA CORPORATION. All right reserved.
     ...
     
     # Python process execution
     root@d622a93c9281:/workspace# python /workspace/nvidia-examples/cnn/resnet.py --num_iter 100 
     ...
     PY 3.8.10 (default, Jun 2 2021, 10:49:15)
     [GCC 9.4.0]
     TF 1.15.5
     ...

     $ docker run --gpus '"device=0:0"' -rm -it --network=host --shm-size=1g --ipc=host -v /root/.ssh/:/root/.ssh
     
     ================
     == TensorFlow ==
     ================
     
     NVIDIA Release 21.08-tf1 (build 26012104)
     TensorFlow Version 1.15.5
     
     Container image Copyright (c) 2021, NVIDIA CORPORATION. All right reserved.
     ...
     
     # Python process execution
     root@d622a93c9281:/workspace# python /workspace/nvidia-examples/cnn/resnet.py --num_iter 100 
     ...
     PY 3.8.10 (default, Jun 2 2021, 10:49:15)
     [GCC 9.4.0]
     TF 1.15.5
     ...

2. Check the GPU usage rate. (Creating a JOB process)
   • You can see that when the Job is driven, the process is assigned to the MIG device and the usage rate increases.
     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] -lcip

     $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] -lcip

     You can check the GPU usage rate as follows.
     Sample Code Download

     Copy Code

     Color mode

     +-----------------------------------------------------------------------------+
     | MIG devices:                                                                |
     +-----------------------------------------------------------------------------+
     |  GPU  GI  CI  MIG |        Memory-Usage |        Vol|        Shared         |
     |       ID  ID  Dev |          BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
     |                   |                     |        ECC|                       |
     |=============================================================================|
     |   0    0   0    0 | 66562MiB / 81251MiB | 98      0 |  7   0    5    1    1 |
     |                   |     5MiB / 13107... |           |                       |
     +-----------------------------------------------------------------------------+
     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     |   0     0    0     17483      C   python                           66559MiB |
     +-----------------------------------------------------------------------------+

     +-----------------------------------------------------------------------------+
     | MIG devices:                                                                |
     +-----------------------------------------------------------------------------+
     |  GPU  GI  CI  MIG |        Memory-Usage |        Vol|        Shared         |
     |       ID  ID  Dev |          BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
     |                   |                     |        ECC|                       |
     |=============================================================================|
     |   0    0   0    0 | 66562MiB / 81251MiB | 98      0 |  7   0    5    1    1 |
     |                   |     5MiB / 13107... |           |                       |
     +-----------------------------------------------------------------------------+
     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     |   0     0    0     17483      C   python                           66559MiB |
     +-----------------------------------------------------------------------------+

MIG Instance deletion and release

To delete a MIG instance and release the MIG, follow these procedures.

Compute Instance deletion

• Delete the Compute Instance.
  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] –dci
  $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] -ci [Compute Instance] –dci

  $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] –dci
  $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] -ci [Compute Instance] –dci

  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi mig -i 0 -gi 0 –lci
  +-----------------------------------------------------------------+
  | Compute instance profiles:                                      |
  | GPU     GPU     Name            Profile  Instances   Placement  |
  | GPU   Instance                     ID      ID        Start:Size |
  |         ID                                                      |
  |=================================================================|
  |   0      0      MIG 7g.80gb         4       0            0:7    |
  +-----------------------------------------------------------------+

  $ nvidia-smi mig -i 0 -gi 0 –lci
  +-----------------------------------------------------------------+
  | Compute instance profiles:                                      |
  | GPU     GPU     Name            Profile  Instances   Placement  |
  | GPU   Instance                     ID      ID        Start:Size |
  |         ID                                                      |
  |=================================================================|
  |   0      0      MIG 7g.80gb         4       0            0:7    |
  +-----------------------------------------------------------------+

  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi mig -i 0 -gi 0 –dci
  Successfully destroyed compute instance ID  0 from GPU instance ID  0

  $ nvidia-smi mig -i 0 -gi 0 –dci
  Successfully destroyed compute instance ID  0 from GPU instance ID  0

  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi mig -i 0 -gi 0 –lci
  No compute instances found: Not found

  $ nvidia-smi mig -i 0 -gi 0 –lci
  No compute instances found: Not found

GPU Instance deletion

• Delete the GPU Instance.
  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi mig -i [GPU ID] –dgi
  $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] –dgi

  $ nvidia-smi mig -i [GPU ID] –dgi
  $ nvidia-smi mig -i [GPU ID] -gi [GPU Instance ID] –dgi

  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi mig -i 0 -lgi
  +--------------------------------------------------------+
  | GPU instances:                                         |
  | GPU   Name               Profile  Instance  Placement  |
  |                            ID       ID      Start:Size |
  |========================================================|
  |   0  MIG 7g.80gb            0        0         0:8     |
  +--------------------------------------------------------+

  $ nvidia-smi mig -i 0 -lgi
  +--------------------------------------------------------+
  | GPU instances:                                         |
  | GPU   Name               Profile  Instance  Placement  |
  |                            ID       ID      Start:Size |
  |========================================================|
  |   0  MIG 7g.80gb            0        0         0:8     |
  +--------------------------------------------------------+

  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi mig -i 0 -dgi
  Successfully destroyed GPU instance ID  0 from GPU  0

  $ nvidia-smi mig -i 0 -dgi
  Successfully destroyed GPU instance ID  0 from GPU  0

  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi mig -i 0 -lgi
  No GPU instances found: Not found

  $ nvidia-smi mig -i 0 -lgi
  No GPU instances found: Not found

MIG Function Disablement (Deactivation)

• Disable MIG and then reboot.
  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi -mig 0
  Disabled MIG Mode for GPU 00000000:05:00.0
  
  All done.

  $ nvidia-smi -mig 0
  Disabled MIG Mode for GPU 00000000:05:00.0
  
  All done.

  Sample Code Download

  Copy Code

  Color mode

  $ nvidia-smi
  Mon Sep 30 05:18:28 2021
  +-----------------------------------------------------------------------------+
  | NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
  |-------------------------------+----------------------+----------------------|
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |                               |                      |               MIG M. |
  |===============================+======================+======================|
  |   0  NVDIA A100-SXM...  Off   | 00000000:05:00.0 Off |                    0 |
  | N/A   33C   P0    60W / 400W  |      0MiB / 81251MiB |      0%      Default |
  |                               |                      |             Disabled |
  +-------------------------------+----------------------+----------------------+
  +-----------------------------------------------------------------------------+
  | MIG devices:                                                                |
  +-----------------------------------------------------------------------------+
  |  GPU  GI  CI  MIG |        Memory-Usage |        Vol|        Shared         |
  |       ID  ID  Dev |          BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
  |                   |                     |        ECC|                       |
  |=============================================================================|
  | No MIG devices found                                                        |
  +-----------------------------------------------------------------------------+
  +-----------------------------------------------------------------------------+
  | Processes:                                                                  |
  |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
  |        ID   ID                                                   Usage      |
  |=============================================================================|
  | No running processes found                                                  |
  +-----------------------------------------------------------------------------+

  $ nvidia-smi
  Mon Sep 30 05:18:28 2021
  +-----------------------------------------------------------------------------+
  | NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
  |-------------------------------+----------------------+----------------------|
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |                               |                      |               MIG M. |
  |===============================+======================+======================|
  |   0  NVDIA A100-SXM...  Off   | 00000000:05:00.0 Off |                    0 |
  | N/A   33C   P0    60W / 400W  |      0MiB / 81251MiB |      0%      Default |
  |                               |                      |             Disabled |
  +-------------------------------+----------------------+----------------------+
  +-----------------------------------------------------------------------------+
  | MIG devices:                                                                |
  +-----------------------------------------------------------------------------+
  |  GPU  GI  CI  MIG |        Memory-Usage |        Vol|        Shared         |
  |       ID  ID  Dev |          BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
  |                   |                     |        ECC|                       |
  |=============================================================================|
  | No MIG devices found                                                        |
  +-----------------------------------------------------------------------------+
  +-----------------------------------------------------------------------------+
  | Processes:                                                                  |
  |  GPU   GI   CI       PID   Type   Process name                   GPU Memory |
  |        ID   ID                                                   Usage      |
  |=============================================================================|
  | No running processes found                                                  |
  +-----------------------------------------------------------------------------+

3 - Using NVSwitch on GPU Server

After creating a GPU Server, you can enable the NVSwitch feature in the GPU Server’s VM (Guest OS) and quickly use P2P (GPU to GPU) communication between GPUs.

Exploring NVIDIA NVSwitch for Multi GPU

NVIDIA A100 GPU server is a multi-GPU based on the NVIDIA Ampere architecture, with 8 Ampere 80 GB GPUs installed on the baseboard. The GPUs installed on the baseboard are connected to 6 NVSwitches via NVLink ports. Communication between GPUs on the baseboard is done using the full 600 GBps bandwidth. For this reason, the 8 GPUs installed on the A100 GPU server can be connected and operated like one, thereby maximizing GPU-to-GPU usage.

• NVLink(25 GBps) 12 Lane 8 GPU configuration

• NVSwitch(600 GBps) 6 units 8 GPU configuration diagram

Create GPU NVSwitch

To use the GPU NVSwitch feature, create a GPU Server service on the Samsung Cloud Platform, create a VM Instance (GuestOS) with 8 A100 GPUs assigned, and activate the Fabricmanager.

NVSwitch Installation and Operation Check (Fabric Manager Activation)

To operate NVSwitch, install Fabricmanager on the GPU Instance and follow the next procedure.

1. Install NVIDIA GPU Driver (470.52.02 Version) on the GPU server.

   Sample Code Download

   Copy Code

   Color mode

   $ add-apt-repository ppa:graphics-drivers/ppa
   $ apt-get update
   $ apt-get install nvidia-driver-470-server

   $ add-apt-repository ppa:graphics-drivers/ppa
   $ apt-get update
   $ apt-get install nvidia-driver-470-server

2. Install and run NVIDIA Fabric Manager (470 Version) on the GPU server (For NVSwitch).

   Sample Code Download

   Copy Code

   Color mode

   $ apt-get install cuda-drivers-fabricmanager-470
   $ systemctl enable nvidia-fabricmanager
   $ systemctl start nvidia-fabricmanager

   $ apt-get install cuda-drivers-fabricmanager-470
   $ systemctl enable nvidia-fabricmanager
   $ systemctl start nvidia-fabricmanager

3. Check the status of NVIDIA Fabric Manager running on the GPU server.

   • Normal operation indicates active (running)
     Sample Code Download

     Copy Code

     Color mode

     $ systemctl status nvidia-fabricmanager

     $ systemctl status nvidia-fabricmanager

4. Check the NVSwitch operation status on the GPU server.
   • Normal operation indicates NV12
     Sample Code Download

     Copy Code

     Color mode

     $ nvidia-smi topo --matrix

     $ nvidia-smi topo --matrix

4 - Keypair Management

The user can enter the required information for the Keypair within the GPU Server service through the Samsung Cloud Platform Console, select detailed options, and create the service.

Keypair Create

You can create and use the Keypair service while using the GPU Server service on the Samsung Cloud Platform Console.

To create a keypair, follow these steps.

1. All Services > Compute > GPU Server Click the menu. Navigate to the Service Home page of GPU Server.
2. Click the Keypair menu on the Service Home page. It will go to the Keypair List page.
3. Click the Keypair List page’s Create Keypair button. It navigates to the Create Keypair page.
   • Service Information Input Enter the required information in the area.

     Category	Required or not	Detailed description
     Keypair name	Required	Name of the Keypair to create Enter within 255 characters using English letters, numbers, spaces, and special characters (-, _)
     Keypair type	Required	ssh

     Table. Keypair Service Information Input Items
   • Additional Information Input Enter or select the required information in the area.

     Category	Required or not	Detailed description
     Tag	Select	Add Tag Up to 50 can be added per resource After clicking the Add Tag button, enter or select Key, Value values

     Table. Keypair additional information input items
     Caution

     • After creation is complete, you can download the Key only once for the first time. Since reissuance is not possible, make sure it has been downloaded.
     • Save the downloaded Private Key in a safe place.
4. Check the input information and click the Complete button.
   • When creation is complete, check the created resource on the Keypair List page.

Keypair Check detailed information

Keypair service can view and edit the full resource list and detailed information. Keypair Details page consists of Details, Tags, Activity History tabs.

To view detailed information about the Keypair, follow the steps below.

1. All Services > Compute > GPU Server Click the menu. Navigate to the GPU Server’s Service Home page.
2. Service Home page, click the Keypair menu. Navigate to the Keypair list page.
3. Keypair List Click the resource to view detailed information on the page. Go to the Keypair Detail page.
   • Keypair Details page displays status information and additional feature information, and consists of Details, Tags, Activity History tabs.

Detailed Information

On the Keypair List page, you can view detailed information of the selected resource and, if necessary, edit the information.

Tag

Keypair list page allows you to view the tag information of selected resources, and you can add, modify, or delete them.

Work History

On the Keypair list page, you can view the operation history of the selected resource.

Keypair Resource Management

Describes the control and management functions of the keypair.

Get public key

To retrieve the public key, follow the steps below.

1. Click the All Services > Compute > GPU Server menu. Go to the GPU Server’s Service Home page.

2. Service Home page, click the Keypair menu. Navigate to the Keypair list page.

3. In the Keypair list page, click the More button at the top, then click the Import Public Key button. You will be taken to the Import Public Key page.

   • Required Information Input Enter or select the required information in the area.

     Category	Required	Detailed description
     Keypair name	required	Name of the Keypair to create
     Keypair type	required	ssh
     Public Key	Required	Enter Public Key Load File: Attach File button to select and attach the public key file Only files with the following extension (.pem) can be attached Enter Public Key: Paste the copied public key value Public key value can be copied from the Keypair Details page

     Table. Required input items for fetching public key

4. Check the entered information and click the Complete button.

   • Once creation is complete, check the created resource on the Keypair List page.

Delete Keypair

You can delete unused Keypairs. However, once a Keypair is deleted it cannot be recovered, so please review the impact thoroughly in advance before proceeding with deletion.

To delete a keypair, follow the steps below.

1. All Services > Compute > GPU Server Click the menu. Navigate to the GPU Server’s Service Home page.
2. Click the Keypair menu on the Service Home page. It moves to the Keypair List page.
3. On the Keypair list page, select the resource to delete and click the Delete button.

• On the Keypair list page, select multiple Keypair check boxes, and click the Delete button at the top of the resource list.

4. After deletion is complete, check on the Keypair List page whether the resource has been deleted.

5 - ServiceWatch Agent Install

Users can install the ServiceWatch Agent on the GPU Server to collect custom metrics and logs.

ServiceWatch Agent

The agents that need to be installed on the GPU Server for collecting ServiceWatch’s custom metrics and logs can be divided into two main types. It is Prometheus Exporter and Open Telemetry Collector.

Install Prometheus Exporter for GPU metrics (for Ubuntu)

Install the Prometheus Exporter to collect metrics of the GPU Server according to the steps below.

NVDIA Driver Installation Check

• Check the installed NVDIA Driver.
  Sample Code Download

  Copy Code

  Color mode

  nvidia-smi --query-gpu driver_version --format csv

  nvidia-smi --query-gpu driver_version --format csv

  Sample Code Download

  Copy Code

  Color mode

  driver_version
  535.183.06
  ...
  535.183.06

  driver_version
  535.183.06
  ...
  535.183.06

NVSwitch Configuration and Query (NSCQ) Library Installation

1. Install cuda-keyring.

   Sample Code Download

   Copy Code

   Color mode

   wget https://developer.download.nvidia.com/compute/cuda/repos/<distro>/<arch>/cuda-keyring_1.1-1_all.deb

   wget https://developer.download.nvidia.com/compute/cuda/repos/<distro>/<arch>/cuda-keyring_1.1-1_all.deb

   Sample Code Download

   Copy Code

   Color mode

   sudo dpkg -i cuda-keyring_1.1-1_all.deb
   apt update

   sudo dpkg -i cuda-keyring_1.1-1_all.deb
   apt update

   Sample Code Download

   Copy Code

   Color mode

   nvidia-smi --query-gpu driver_version --format csv

   nvidia-smi --query-gpu driver_version --format csv

   Sample Code Download

   Copy Code

   Color mode

   driver_version
   535.183.06
   ...
   
   535.183.06

   driver_version
   535.183.06
   ...
   
   535.183.06

2. Install libnvidia-nscq.

   Sample Code Download

   Copy Code

   Color mode

   apt-cache policy libnvidia-nscq-535

   apt-cache policy libnvidia-nscq-535

   Sample Code Download

   Copy Code

   Color mode

   libnvidia-nscq-535:
     Installed: (none)
     Candidate: 535.247.01-1
     Version table:
        535.247.01-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
   ...
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        535.216.01-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        535.183.06-1 600  # Install version matching the Driver
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        535.183.01-1 600
   ...
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        535.54.03-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages

   libnvidia-nscq-535:
     Installed: (none)
     Candidate: 535.247.01-1
     Version table:
        535.247.01-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
   ...
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        535.216.01-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        535.183.06-1 600  # Install version matching the Driver
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        535.183.01-1 600
   ...
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        535.54.03-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages

   Sample Code Download

   Copy Code

   Color mode

   apt install libnvidia-nscq-535=535.183.06-1

   apt install libnvidia-nscq-535=535.183.06-1

NVSwitch Device Monitoring API(NVSDM) Library Installation

• NVSDM library install.
  Sample Code Download

  Copy Code

  Color mode

  apt-cache policy libnvsdm

  apt-cache policy libnvsdm

  Sample Code Download

  Copy Code

  Color mode

  libnvsdm:
    Installed: (none)
    Candidate: 580.105.08-1
    Version table:
       580.105.08-1 600
          600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
       580.95.05-1 600
          600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
       580.82.07-1 600
          600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
       580.65.06-1 600
          600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages

  libnvsdm:
    Installed: (none)
    Candidate: 580.105.08-1
    Version table:
       580.105.08-1 600
          600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
       580.95.05-1 600
          600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
       580.82.07-1 600
          600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
       580.65.06-1 600
          600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages

  Sample Code Download

  Copy Code

  Color mode

  apt install libnvsdm=580.105.08-1

  apt install libnvsdm=580.105.08-1

NVIDIA DCGM Installation (for Ubuntu)

Install the DCGM Exporter according to the following steps.

1. DCGM(datacenter-gpu-manager) Installation
2. datacenter-gpu-manager-exporter Installation
3. DCGM Service Activation and Start

DCGM(datacenter-gpu-manager) Installation

It refers to a specific version of NVIDIA’s Data Center GPU Manager (DCGM) tool, which is a package for managing and monitoring NVIDIA data center GPUs. Specifically, cuda12 indicates that this management tool is installed for CUDA version 12, and datacenter‑gpu‑manager‑4 denotes the 4.x version of DCGM. This tool provides various features including GPU status monitoring, diagnostics, alert system, and power/clock management.

1. Check the CUDA version.
   Sample Code Download

   Copy Code

   Color mode

   nvidia-smi | grep CUDA

   nvidia-smi | grep CUDA

   Sample Code Download

   Copy Code

   Color mode

   | NVIDIA-SMI 535.183.06             Driver Version: 535.183.06     CUDA Version: 12.2     |

   | NVIDIA-SMI 535.183.06             Driver Version: 535.183.06     CUDA Version: 12.2     |

   Sample Code Download

   Copy Code

   Color mode

   CUDA_VERSION=12

   CUDA_VERSION=12

2. Install datacenter-gpu-manager-cuda.
   Sample Code Download

   Copy Code

   Color mode

   apt install datacenter-gpu-manager-4-cuda${CUDA_VERSION}

   apt install datacenter-gpu-manager-4-cuda${CUDA_VERSION}

datacenter-gpu-manager-exporter installation

Based on NVIDIA Data Center GPU Manager (DCGM), it is a tool that collects various GPU metrics such as GPU usage, memory usage, temperature, and power consumption, and exposes them so they can be used in monitoring systems like Prometheus.

1. Install datacenter-gpu-manager-exporter.
   Sample Code Download

   Copy Code

   Color mode

   apt install datacenter-gpu-manager-exporter

   apt install datacenter-gpu-manager-exporter

2. Check the DCGM Exporter configuration file.
   Sample Code Download

   Copy Code

   Color mode

   cat /usr/lib/systemd/system/nvidia-dcgm-exporter.service | grep ExecStart

   cat /usr/lib/systemd/system/nvidia-dcgm-exporter.service | grep ExecStart

   Sample Code Download

   Copy Code

   Color mode

   ExecStart=/usr/bin/dcgm-exporter -f /etc/dcgm-exporter/default-counters.csv

   ExecStart=/usr/bin/dcgm-exporter -f /etc/dcgm-exporter/default-counters.csv

3. When installing DCGM Exporter, check the provided settings and remove # from the required metrics, and add # to the unnecessary metrics.
   Sample Code Download

   Copy Code

   Color mode

   vi /etc/dcgm-exporter/default-counters.csv
   ## Example ##
   ...
   DCGM_FI_PROF_PIPE_TENSOR_ACTIVE, gauge, Ratio of cycles the tensor (HMMA) pipe is active.
   DCGM_FI_PROF_DRAM_ACTIVE,        gauge, Ratio of cycles the device memory interface is active sending or receiving data.
   # DCGM_FI_PROF_PIPE_FP64_ACTIVE,   gauge, Ratio of cycles the fp64 pipes are active.
   # DCGM_FI_PROF_PIPE_FP32_ACTIVE,   gauge, Ratio of cycles the fp32 pipes are active.
   ...

   vi /etc/dcgm-exporter/default-counters.csv
   ## Example ##
   ...
   DCGM_FI_PROF_PIPE_TENSOR_ACTIVE, gauge, Ratio of cycles the tensor (HMMA) pipe is active.
   DCGM_FI_PROF_DRAM_ACTIVE,        gauge, Ratio of cycles the device memory interface is active sending or receiving data.
   # DCGM_FI_PROF_PIPE_FP64_ACTIVE,   gauge, Ratio of cycles the fp64 pipes are active.
   # DCGM_FI_PROF_PIPE_FP32_ACTIVE,   gauge, Ratio of cycles the fp32 pipes are active.
   ...

DCGM Service activation and start

1. Activate and start the nvdia-dcgm service.

   Sample Code Download

   Copy Code

   Color mode

   systemctl enable --now nvidia-dcgm

   systemctl enable --now nvidia-dcgm

2. Activate and start the nvdia-dcgm-exporter service.

   Sample Code Download

   Copy Code

   Color mode

   systemctl enable --now nvidia-dcgm-exporter

   systemctl enable --now nvidia-dcgm-exporter

Installation of Prometheus Exporter for GPU metrics (for RHEL)

Install the ServiceWatch Agent according to the steps below to collect metrics of the GPU Server.

NVDIA Driver Installation Check (for RHEL)

1. Check the installed NVDIA Driver.
   Sample Code Download

   Copy Code

   Color mode

   nvidia-smi --query-gpu driver_version --format csv

   nvidia-smi --query-gpu driver_version --format csv

   Sample Code Download

   Copy Code

   Color mode

   driver_version
   535.183.06
   ...
   
   535.183.06

   driver_version
   535.183.06
   ...
   
   535.183.06

NVSwitch Configuration and Query (NSCQ) Library installation (for RHEL)

1. Check the libnvdia-nscq package.

   Sample Code Download

   Copy Code

   Color mode

   rpm -qa | grep libnvidia-nscq libnvidia-nscq-535-535.183.06-1.x86_64

   rpm -qa | grep libnvidia-nscq libnvidia-nscq-535-535.183.06-1.x86_64

2. Add CUDA Repository to DNF.

   Sample Code Download

   Copy Code

   Color mode

   dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo

   dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo

3. NVDIA Driver status initialization

   Sample Code Download

   Copy Code

   Color mode

   dnf module reset nvidia-driver

   dnf module reset nvidia-driver

   Sample Code Download

   Copy Code

   Color mode

   Updating Subscription Management repositories.
   Last metadata expiration check: 0:03:15 ago on Wed 19 Nov 2025 01:23:48 AM EST.
   Dependencies resolved.
   =============================================
   Package Architecture Version Repository Size
   =============================================
   Disabling module profiles:
   nvidia-driver/default
   nvidia-driver/fm
   Resetting modules:
   nvidia-driver
   
   Transaction Summary
   =============================================
   
   Is this ok [y/N]: y

   Updating Subscription Management repositories.
   Last metadata expiration check: 0:03:15 ago on Wed 19 Nov 2025 01:23:48 AM EST.
   Dependencies resolved.
   =============================================
   Package Architecture Version Repository Size
   =============================================
   Disabling module profiles:
   nvidia-driver/default
   nvidia-driver/fm
   Resetting modules:
   nvidia-driver
   
   Transaction Summary
   =============================================
   
   Is this ok [y/N]: y

4. Activate the NVDIA Driver module.

   Sample Code Download

   Copy Code

   Color mode

   dnf module enable nvidia-driver:535-open

   dnf module enable nvidia-driver:535-open

   Sample Code Download

   Copy Code

   Color mode

   Updating Subscription Management repositories.
   Last metadata expiration check: 0:04:22 ago on Wed 19 Nov 2025 01:23:48 AM EST.
   Dependencies resolved.
   =============================================
   
   Package Architecture Version Repository Size
   =============================================
   Enabling module streams:
   nvidia-driver 535-open
     
   Transaction Summary
   =============================================
   
   Is this ok [y/N]: y

   Updating Subscription Management repositories.
   Last metadata expiration check: 0:04:22 ago on Wed 19 Nov 2025 01:23:48 AM EST.
   Dependencies resolved.
   =============================================
   
   Package Architecture Version Repository Size
   =============================================
   Enabling module streams:
   nvidia-driver 535-open
     
   Transaction Summary
   =============================================
   
   Is this ok [y/N]: y

5. Check the libnvdia-nscq module list.

   Sample Code Download

   Copy Code

   Color mode

   dnf list libnvidia-nscq-535 --showduplicates

   dnf list libnvidia-nscq-535 --showduplicates

6. Install libnvdia-nscq.

   Sample Code Download

   Copy Code

   Color mode

   dnf install libnvidia-nscq-535-535.183.06-1

   dnf install libnvidia-nscq-535-535.183.06-1

NVSwitch Device Monitoring API(NVSDM) Library Installation (for RHEL)

1. Check the NVSDM library module list.

   Sample Code Download

   Copy Code

   Color mode

   dnf list libnvsdm --showduplicates

   dnf list libnvsdm --showduplicates

   Sample Code Download

   Copy Code

   Color mode

   libnvsdm:
     Installed: (none)
     Candidate: 580.105.08-1
     Version table:
        580.105.08-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        580.95.05-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        580.82.07-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        580.65.06-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages

   libnvsdm:
     Installed: (none)
     Candidate: 580.105.08-1
     Version table:
        580.105.08-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        580.95.05-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        580.82.07-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages
        580.65.06-1 600
           600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  Packages

2. Install libnvsdm.

   Sample Code Download

   Copy Code

   Color mode

   dnf install libnvsdm-580.105.08-1

   dnf install libnvsdm-580.105.08-1

   Sample Code Download

   Copy Code

   Color mode

   Updating Subscription Management repositories.
   Last metadata expiration check: 0:08:18 ago on Wed 19 Nov 2025 01:05:28 AM EST.
   Dependencies resolved.
   =========================================================================
   
   Package Architecture Version Repository Size
   =========================================================================
   
   Installing:
   libnvsdm x86_64 580.105.08-1 cuda-rhel8-x86_64 675 k
   Installing dependencies:
   infiniband-diags x86_64 48.0-1.el8 rhel-8-for-x86_64-baseos-rpms 323 k
   libibumad x86_64 48.0-1.el8 rhel-8-for-x86_64-baseos-rpms 34 k
   
   Transaction Summary
   =========================================================================
   
   Install 3 Packages
   
   Total download size: 1.0 M
   Installed size: 3.2 M
   Is this ok [y/N]: y

   Updating Subscription Management repositories.
   Last metadata expiration check: 0:08:18 ago on Wed 19 Nov 2025 01:05:28 AM EST.
   Dependencies resolved.
   =========================================================================
   
   Package Architecture Version Repository Size
   =========================================================================
   
   Installing:
   libnvsdm x86_64 580.105.08-1 cuda-rhel8-x86_64 675 k
   Installing dependencies:
   infiniband-diags x86_64 48.0-1.el8 rhel-8-for-x86_64-baseos-rpms 323 k
   libibumad x86_64 48.0-1.el8 rhel-8-for-x86_64-baseos-rpms 34 k
   
   Transaction Summary
   =========================================================================
   
   Install 3 Packages
   
   Total download size: 1.0 M
   Installed size: 3.2 M
   Is this ok [y/N]: y

NVIDIA DCGM Installation (for RHEL)

Install Node Exporter according to the steps below.

1. DCGM(datacenter-gpu-manager) Installation
2. datacenter-gpu-manager-exporter installation
3. DCGM Service Activation and Start

DCGM(datacenter-gpu-manager) Installation (for RHEL)

It refers to a specific version of NVIDIA’s Data Center GPU Manager (DCGM) tool, which is a package for managing and monitoring NVIDIA data center GPUs. Specifically, cuda12 indicates that this management tool is installed for CUDA version 12, and datacenter‑gpu‑manager‑4 denotes the 4.x version of DCGM. This tool provides various functions including GPU status monitoring, diagnostics, alert system, and power/clock management.

1. Add CUDA Repository to DNF.
   Sample Code Download

   Copy Code

   Color mode

   dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo

   dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo

2. Check the CUDA version.
   Sample Code Download

   Copy Code

   Color mode

   nvidia-smi | grep CUDA

   nvidia-smi | grep CUDA

   Sample Code Download

   Copy Code

   Color mode

   | NVIDIA-SMI 535.183.06             Driver Version: 535.183.06     CUDA Version: 12.2     |

   | NVIDIA-SMI 535.183.06             Driver Version: 535.183.06     CUDA Version: 12.2     |

   Sample Code Download

   Copy Code

   Color mode

   CUDA_VERSION=12

   CUDA_VERSION=12

3. Check the list of datacenter-gpu-manager-cuda modules.
   Sample Code Download

   Copy Code

   Color mode

   dnf list datacenter-gpu-manager-4-cuda${CUDA_VERSION} --showduplicates

   dnf list datacenter-gpu-manager-4-cuda${CUDA_VERSION} --showduplicates

   Sample Code Download

   Copy Code

   Color mode

   Updating Subscription Management repositories.
   Unable to read consumer identity
   
   This system is not registered with an entitlement server. You can use subscription-manager to register.
   
   Last metadata expiration check: 0:00:34 ago on Wed 19 Nov 2025 12:26:56 AM EST.
   Available Packages
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.0.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.1.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.1.1-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.2.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.2.2-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.2.3-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.2.3-2    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.3.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.3.1-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.4.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.4.1-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.4.2-1    cuda-rhel8-x86_64

   Updating Subscription Management repositories.
   Unable to read consumer identity
   
   This system is not registered with an entitlement server. You can use subscription-manager to register.
   
   Last metadata expiration check: 0:00:34 ago on Wed 19 Nov 2025 12:26:56 AM EST.
   Available Packages
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.0.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.1.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.1.1-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.2.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.2.2-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.2.3-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.2.3-2    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.3.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.3.1-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.4.0-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.4.1-1    cuda-rhel8-x86_64
   datacenter-gpu-manager-4-cuda12.x86_64   1:4.4.2-1    cuda-rhel8-x86_64

4. Install datacenter-gpu-manager-cuda.
   Sample Code Download

   Copy Code

   Color mode

   dnf install datacenter-gpu-manager-4-cuda${CUDA_VERSION}

   dnf install datacenter-gpu-manager-4-cuda${CUDA_VERSION}

   Sample Code Download

   Copy Code

   Color mode

   Updating Subscription Management repositories.
   Unable to read consumer identity
   
   This system is not registered with an entitlement server. You can use subscription-manager to register.
   
   Last metadata expiration check: 0:07:12 ago on Wed 19 Nov 2025 12:26:56 AM EST.
   Dependencies resolved.
   ===================================================================================================
   Package                                       Architecture   Version     Repository          Size
   ===================================================================================================
   Installing:
    datacenter-gpu-manager-4-cuda12               x86_64         1:4.4.2-1   cuda-rhel8-x86_64   554 M
   Installing dependencies:
    datacenter-gpu-manager-4-core                 x86_64         1:4.4.2-1   cuda-rhel8-x86_64   9.9 M
   Installing weak dependencies:
    datacenter-gpu-manager-4-proprietary          x86_64         1:4.4.2-1   cuda-rhel8-x86_64   5.3 M
    datacenter-gpu-manager-4-proprietary-cuda12   x86_64         1:4.4.2-1   cuda-rhel8-x86_64   289 M
   
   Transaction Summary
   ====================================================================================================
   Install  4 Packages
   ...
   Is this ok [y/N]: y

   Updating Subscription Management repositories.
   Unable to read consumer identity
   
   This system is not registered with an entitlement server. You can use subscription-manager to register.
   
   Last metadata expiration check: 0:07:12 ago on Wed 19 Nov 2025 12:26:56 AM EST.
   Dependencies resolved.
   ===================================================================================================
   Package                                       Architecture   Version     Repository          Size
   ===================================================================================================
   Installing:
    datacenter-gpu-manager-4-cuda12               x86_64         1:4.4.2-1   cuda-rhel8-x86_64   554 M
   Installing dependencies:
    datacenter-gpu-manager-4-core                 x86_64         1:4.4.2-1   cuda-rhel8-x86_64   9.9 M
   Installing weak dependencies:
    datacenter-gpu-manager-4-proprietary          x86_64         1:4.4.2-1   cuda-rhel8-x86_64   5.3 M
    datacenter-gpu-manager-4-proprietary-cuda12   x86_64         1:4.4.2-1   cuda-rhel8-x86_64   289 M
   
   Transaction Summary
   ====================================================================================================
   Install  4 Packages
   ...
   Is this ok [y/N]: y

datacenter-gpu-manager-exporter installation (for RHEL)

It is a tool that, based on NVIDIA Data Center GPU Manager (DCGM), collects various GPU metrics such as GPU usage, memory usage, temperature, and power consumption, and exposes them for use in monitoring systems like Prometheus.

1. Add the CUDA Repository to DNF. (If you have already performed this command, proceed to the next step.)

   Sample Code Download

   Copy Code

   Color mode

   dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo

   dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo

2. Check the CUDA version. (If you have already performed this command, proceed to the next step.)

   Sample Code Download

   Copy Code

   Color mode

   nvidia-smi | grep CUDA

   nvidia-smi | grep CUDA

   Sample Code Download

   Copy Code

   Color mode

   | NVIDIA-SMI 535.183.06             Driver Version: 535.183.06     CUDA Version: 12.2     |

   | NVIDIA-SMI 535.183.06             Driver Version: 535.183.06     CUDA Version: 12.2     |

   Sample Code Download

   Copy Code

   Color mode

   CUDA_VERSION=12

   CUDA_VERSION=12

3. Check the list of datacenter-gpu-manager-exporter modules.

   Sample Code Download

   Copy Code

   Color mode

   dnf list datacenter-gpu-manager-exporter --showduplicates

   dnf list datacenter-gpu-manager-exporter --showduplicates

   Sample Code Download

   Copy Code

   Color mode

   Updating Subscription Management repositories.
   Unable to read consumer identity
   
   This system is not registered with an entitlement server. You can use subscription-manager to register.
   
   Last metadata expiration check: 0:02:11 ago on Wed 19 Nov 2025 12:26:56 AM EST.
   Available Packages
   datacenter-gpu-manager-exporter.x86_64   4.0.1-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.1.0-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.1.1-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.1.3-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.5.0-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.5.1-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.5.2-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.6.0-1   cuda-rhel8-x86_64

   Updating Subscription Management repositories.
   Unable to read consumer identity
   
   This system is not registered with an entitlement server. You can use subscription-manager to register.
   
   Last metadata expiration check: 0:02:11 ago on Wed 19 Nov 2025 12:26:56 AM EST.
   Available Packages
   datacenter-gpu-manager-exporter.x86_64   4.0.1-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.1.0-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.1.1-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.1.3-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.5.0-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.5.1-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.5.2-1   cuda-rhel8-x86_64
   datacenter-gpu-manager-exporter.x86_64   4.6.0-1   cuda-rhel8-x86_64

4. Install datacenter-gpu-manager-cuda. dcgm-exporter 4.5.X requires glibc 2.34 or higher, but since glibc 2.34 is provided in RHEL9, we specify the version as 4.1.3-1 for installation.

   Sample Code Download

   Copy Code

   Color mode

   dnf install datacenter-gpu-manager-exporter-4.1.3-1

   dnf install datacenter-gpu-manager-exporter-4.1.3-1

   Sample Code Download

   Copy Code

   Color mode

   Updating Subscription Management repositories.
   Unable to read consumer identity
     
   This system is not registered with an entitlement server. You can use subscription-manager to register.
   
   Last metadata expiration check: 0:07:12 ago on Wed 19 Nov 2025 12:26:56 AM EST.
   Dependencies resolved.
   ====================================================================================================
   Package                                       Architecture   Version     Repository          Size
   ====================================================================================================
   Installing:
    datacenter-gpu-manager-exporter               x86_64         4.1.3-1     cuda-rhel8-x86_64   26 M
   
   ...
   Is this ok [y/N]: y

   Updating Subscription Management repositories.
   Unable to read consumer identity
     
   This system is not registered with an entitlement server. You can use subscription-manager to register.
   
   Last metadata expiration check: 0:07:12 ago on Wed 19 Nov 2025 12:26:56 AM EST.
   Dependencies resolved.
   ====================================================================================================
   Package                                       Architecture   Version     Repository          Size
   ====================================================================================================
   Installing:
    datacenter-gpu-manager-exporter               x86_64         4.1.3-1     cuda-rhel8-x86_64   26 M
   
   ...
   Is this ok [y/N]: y

   Sample Code Download

   Copy Code

   Color mode

   cat /usr/lib/systemd/system/nvidia-dcgm-exporter.service | grep ExecStart

   cat /usr/lib/systemd/system/nvidia-dcgm-exporter.service | grep ExecStart

   Sample Code Download

   Copy Code

   Color mode

   ExecStart=/usr/bin/dcgm-exporter -f /etc/dcgm-exporter/default-counters.csv

   ExecStart=/usr/bin/dcgm-exporter -f /etc/dcgm-exporter/default-counters.csv

5. When installing DCGM Exporter, check the provided settings and remove # from the required metrics, and add # to the unnecessary metrics.

   Sample Code Download

   Copy Code

   Color mode

   vi /etc/dcgm-exporter/default-counters.csv
   ## Example ##
   ...
   DCGM_FI_PROF_PIPE_TENSOR_ACTIVE, gauge, Ratio of cycles the tensor (HMMA) pipe is active.
   DCGM_FI_PROF_DRAM_ACTIVE,        gauge, Ratio of cycles the device memory interface is active sending or receiving data.
   # DCGM_FI_PROF_PIPE_FP64_ACTIVE,   gauge, Ratio of cycles the fp64 pipes are active.
   # DCGM_FI_PROF_PIPE_FP32_ACTIVE,   gauge, Ratio of cycles the fp32 pipes are active.
   ...

   vi /etc/dcgm-exporter/default-counters.csv
   ## Example ##
   ...
   DCGM_FI_PROF_PIPE_TENSOR_ACTIVE, gauge, Ratio of cycles the tensor (HMMA) pipe is active.
   DCGM_FI_PROF_DRAM_ACTIVE,        gauge, Ratio of cycles the device memory interface is active sending or receiving data.
   # DCGM_FI_PROF_PIPE_FP64_ACTIVE,   gauge, Ratio of cycles the fp64 pipes are active.
   # DCGM_FI_PROF_PIPE_FP32_ACTIVE,   gauge, Ratio of cycles the fp32 pipes are active.
   ...

DCGM Service Activation and Start (for RHEL)

1. Enable and start the nvdia-dcgm service.

   Sample Code Download

   Copy Code

   Color mode

   systemctl enable --now nvidia-dcgm

   systemctl enable --now nvidia-dcgm

2. Activate and start the nvdia-dcgm-exporter service.

   Sample Code Download

   Copy Code

   Color mode

   systemctl enable --now nvidia-dcgm-exporter

   systemctl enable --now nvidia-dcgm-exporter

DCGM Exporter Metrics

DCGM Exporter Key Metrics

Among the metrics provided by DCGM Exporter, the main GPU metrics are as follows.

DCGM Exporter Metric Collection Settings

Please refer to the default metrics of DCGM Exporter at DCGM Exporter > Default Metrics.

• For indicators to be set in addition to the default settings, remove # in default-counters.csv.
• For metrics that are not desired to be collected among the default metrics, add # or delete the item.

# Format
# If line starts with a '#' it is considered a comment
# DCGM FIELD, Prometheus metric type, help message

# Clocks
DCGM_FI_DEV_SM_CLOCK,  gauge, SM clock frequency (in MHz).
DCGM_FI_DEV_MEM_CLOCK, gauge, Memory clock frequency (in MHz).

# Temperature
DCGM_FI_DEV_MEMORY_TEMP, gauge, Memory temperature (in C).
DCGM_FI_DEV_GPU_TEMP,    gauge, GPU temperature (in C).

# Power
DCGM_FI_DEV_POWER_USAGE,              gauge, Power draw (in W).
DCGM_FI_DEV_TOTAL_ENERGY_CONSUMPTION, counter, Total energy consumption since boot (in mJ).

# PCIE
# DCGM_FI_PROF_PCIE_TX_BYTES,  counter, Total number of bytes transmitted through PCIe TX via NVML.
# DCGM_FI_PROF_PCIE_RX_BYTES,  counter, Total number of bytes received through PCIe RX via NVML.
...

# Format
# If line starts with a '#' it is considered a comment
# DCGM FIELD, Prometheus metric type, help message

# Clocks
DCGM_FI_DEV_SM_CLOCK,  gauge, SM clock frequency (in MHz).
DCGM_FI_DEV_MEM_CLOCK, gauge, Memory clock frequency (in MHz).

# Temperature
DCGM_FI_DEV_MEMORY_TEMP, gauge, Memory temperature (in C).
DCGM_FI_DEV_GPU_TEMP,    gauge, GPU temperature (in C).

# Power
DCGM_FI_DEV_POWER_USAGE,              gauge, Power draw (in W).
DCGM_FI_DEV_TOTAL_ENERGY_CONSUMPTION, counter, Total energy consumption since boot (in mJ).

# PCIE
# DCGM_FI_PROF_PCIE_TX_BYTES,  counter, Total number of bytes transmitted through PCIe TX via NVML.
# DCGM_FI_PROF_PCIE_RX_BYTES,  counter, Total number of bytes received through PCIe RX via NVML.
...