The page has been translated by Gen AI.

Vertica

Vertica

Overview

This document provides a guide on how to migrate the Vertica you are using to SCP-V2.
First introduce Data Migration tools and data validation methods, then, based on scenarios, guide the migration example through the commands

Preliminary work

  • When creating the target TO-BE DB for migration, create the CATALOG, DATA, and TEMP areas in the same directory path as the AS-IS
  • Install the same Vertica version as AS-IS (including hotfixes)
  • Number of nodes, node name, and DB name are the same
  • Communication between AS-IS nodes and TO-BE nodes is possible without a password
  • When replicating a cluster, the network performance must be guaranteed for rapid failover

Migration Procedure


Scenario

  • Environment: Vertica operation on-premises, Data size 500GB, analysis tasks
  • Requirement: Migrate to Samsung Cloud Platform v2 Vertica (DbaaS), maximum allowable service downtime is 24 hours

Migration tool

vbr_copy_cluster

  • This is how to use the feature that replicates data backed up locally via the vbr backup utility built into Vertica to a Vertica cluster configured in the Samsung Cloud Platform v2 environment.
  • Using the vbr option called copycluster, you can copy an entire database from one Vertica cluster to another Vertica cluster.
  • Vertica backs up data from the source cluster and restores it to the target cluster in a single operation.
  • When copying data with a VBR script that includes the copycluster option, it overwrites all data on the target cluster.

1. ## Migration Planning

  • Since the entire database is being migrated, we use the cluster replication method vbr copy_cluster.
  • Source Vertica can be migrated while online, but because the data is transferred via a backup using vbr, only query functionality should be performed during the migration period to ensure data consistency. * Data changed since the last backup will not be transferred.
  • Communication between the Source node and the Target node must be possible without a password for it to be usable.
  • Since the data is transmitted without any separate processing, data validation does not need to be performed separately, but when validation is essential, it can be done by comparing row counts for each table and summing each column. * (For numeric format, use sum; for string format, use the sum of string lengths, etc.)

2. ## Create target cluster

  • Create the Target DB by navigating to the menu below through the Samsung Cloud Platform Console.
    Data Analytics > Vertica(DBaaS) > Vertica Creation

3. ## Create a database with the same name as the Source Database in the Target Database

Create a database with the same name as the Source Database in the Target Database.

  • e.g.) htest

    [Target DB]
    $ /opt/vertica/bin/admintools -t create_db -s 192.168.xx.x -d htest
    Info: no password specified, using none
    Database with 1 or 2 nodes cannot be k-safe and it may lose data if it crashes
    Database htest created successfully.


4. ## Target Database Stop


[Target DB]
$ /opt/vertica/bin/admintools -t stop_db -d htest -p htestpasswd
Info: no password specified, using none
Connecting to database
Issuing shutdown command to database
Database htest stopped successfully.


5. ## Create configuration file from Source Database

Refer to /opt/vertica/share/vbr/example_configs/copycluster.ini to create a configration file


[Source DB]
$ vi copycluster.ini
--- copycluster.ini
[Misc]
snapshotName = CopyTest
tempDir = /tmp/vbr
retryCount = 2
retryDelay = 1
passwordFile = /home/htest/.backup_pwd

[Database]
dbName = htest
dbUser = htest
dbPromptForPassword = False

[Transmission]
port_rsync = 50000

[Mapping]
v_htest_node0001 = 192.168.xx.x

[Source DB]
$ vi /home/htest/.backup+pwd
[Passwords]
dbPassword = password!


6. ## Run vbr script from Source Database

Run the vbr script with the –copycluster option using the configuration file (copycluster.ini) created earlier

[Source DB]
$ /opt/vertica/bin/vbr --config-file copycluster.ini --task copycluster
Starting copy of database htest.
Participating nodes: v_htest_node0001.
Snapshotting database.
Snapshot complete.
Determining what data to copy.
[==========================================] 100%
Approximate bytes to copy : 627459683 of 627459683 total.
Syncing data to destination cluster.
[==========================================] 100%
Reinitializing destination catalog.
copycluster complete!


7. ## Start Target Database and Verify Data

Start the Target Database

[Target DB]
$ /opt/vertica/bin/admintools -t start_db -d htest -p htestpasswd
Info: no password specified, using none
Starting nodes:
v_hest_node0001 (192.168.xx.x) Starting Vertica on all nodes. Please wait, database with large catalog may take a while to initialize.
Node Status: v_htest_node0001: (DOWN)
Node Status: v_htest_node0001: (DOWN)
Node Status: v_htest_node0001: (DOWN)
Node Status: v_htest_node0001: (DOWN)
Node Status: v_htest_node0001: (UP)
Database htest started successfully