The unit data backup is used to create an individual unit data backup of any PLDB or DSG replica of the CUDB node.

Three unit data backups can be stored on the CUDB node at the same time. Subsequent unit data backups remove the oldest backup already stored, and keep the newer one.

For safety reasons, it is recommended to store backup files in an external storage.

Run the following command to copy the backup in an external storage:

SC_2_1# scp [-l <bandwidth>] <file> <destination>

Where:

• <bandwidth> represents a value to limit the bandwidth in kbit/s. For CUDB systems deployed on native BSP 8100 on Generic Ericsson Processor version 3 (GEP3) nodes or for CUDB systems deployed on a cloud infrastructure with vCUDB_2CPU_6GB hardware type, it is recommended to set the value to 10 000 for stability reasons; for CUDB systems deployed on native BSP 8100 on Generic Ericsson Processor version 5 (GEP5) nodes or for CUDB systems deployed on a cloud infrastructure with vCUDB_16CPU_47GB hardware type, it is recommended not to use the -l command-line switch to allow scp to use all the available bandwidth.

  Note:

  Avoid creating parallel scp copies of the different backup files.

• <file> represents the output backup files.

• <destination> represents the path where the output backup files will be stored in the external storage.

In case any failure occurs during the backup phase, the system raises the Storage Engine, Backup Fault In DS or Storage Engine, Backup Fault In PLDB alarms.

The cudbManageStore command creates a fresh backup of the data stored in the selected PLDB or DSG replica. The backup is made up of several files which are created in the following directory located on the disk storage system of every participating blade or VM:

/local/cudb/mysql/ndbd/backup/BACKUP/BACKUP-YYYY-MM-DD_HH-mm

The identifiers of this participating blade or VM are as follows:

• For DSGs:

  PL_2_x, where x represents the following sequence:

  • from <Total_PLDB_blades_or_vm> + 1 + 2* <dsId>

  • to <Total_PLDB_blades_or_vm> + 2 + 2 * <dsId> .

• For the PLDB:

  PL_2_x, where x is the sequence:

  • from 3

  • to <Total_PLDB_blades_or_vm> + 2.

The backup components consist of the following files:

• Metadata file (.ctl extension)

• Table records file (.data extension)

• Transaction log file (.log extension)

Typically, the backup files are generated in all the blades or VMs of the database unit. However, in case of degraded clusters (refer to CUDB High Availability for more information on degradation), the blades or VMs not participating in the backup do not generate the files, so these will be created in other blades or VMs of the cluster. See Unit Data Restore for further details on how these files are generated and moved to the slaves replicas.

A CUDB system is composed of several data partitions, including the PLDB and a number of DSGs. According to this structure, a system data backup consists of a number of pieces, each piece being the backup of an individual data partition of the whole CUDB system. One backup of the PLDB and one backup of each of the DSGs make up the full CUDB system data backup.

Refer to CUDB Data Storage Handling for more information.

This type of backup contains the information stored in all database units of the CUDB system.

Unlike most Operation and Maintenance (OAM) operations that affect a single CUDB node, a full CUDB system data backup is a system-wide operation.

The criteria to choose the replica to be used to create the backup is the following:

• The backup of the PLDB is performed always in the master replica.

• The DSG backups are performed in a slave replica, if at least one is available, otherwise they are performed in the master replica.

The distribution of these pieces and the topology of the system makes system data backup to be a “data centric” backup.

To better understand this “data centric” concept, see Figure 1 showing the subscriber data distribution over the system.

Figure 1   Data Distribution Example

Data backup of the whole CUDB system consists of the following:

• A backup of PLDB made of any node of the system.

• A backup of DSG1 made of node 1, 7, or 9.

• A backup of DSG2 made of node 1, 5, or 9.

• A backup of DSG3 made of node 1 ,5, or 7.

• A backup of DSG4 made of node 5, 7, or 9.

System data backup cannot be initiated in the following cases:

• No master PLDB replica exists.

• The master PLDB replica is degraded (refer to CUDB High Availability for more information on degradation).

• All slave replicas and also the master replica of any DSGs are down or degraded in a geographically redundant system.

The execution of a backup does not affect regular traffic operations and traffic operations do not affect the backup. If provisioning continues during a system data backup, it can cause misalignment between the data in the Processing Layer (PLDB) backup and the data in the Data Store Layer (DSG) backup. This misalignment will only become apparent in the unlikely case of a system data restore and the amount of effected Lightweight Directory Access Protocol (LDAP) entries will be negligible compared to the data loss caused by the restoration of the system data backup in most of the usage scenarios. The misalignment can be partially corrected by the reconciliation process. The remaining misalignment can be detected and corrected by application Front End (FE) level procedures.

CUDB offers the function of sending a notification to the Ericsson Provisioning Gateway (PG) when a system data backup is started and finished so that the PG can withhold provisioning operations towards the CUDB system while the backup is being created.

Although this is the default behavior, the operator must decide whether ensuring full consistency between PLDB and DSG backup is preferred or avoiding impact in provisioning every time system data backup is performed. If continuous provisioning is preferred then sending notifications to PG can be disabled by using command-line parameters in the cudbSystemDataBackupAndRestore command. For more information on command parameters for cudbSystemDataBackupAndRestore, refer to CUDB Node Commands and Parameters. For instructions on how to schedule a backup without notifications, refer to Step 3 in Scheduling a Periodic System Data Backup in a CUDB System.

If it is not possible to inform the PG about the backup, the backup fails and the system raises the Storage Engine, Backup Notification Failure To Provisioning Gateway alarm. Refer to Storage Engine, Backup Notification Failure To Provisioning Gateway for more information.

In case any failure occurs during the backup phase, the system raises the Storage Engine, Backup Fault In DS or Storage Engine, Backup Fault In PLDB alarms.

The following output directories are created by the cudbSystemDataBackupAndRestore command:

• Temporary backup files and directories are created on those CUDB nodes where the unit data backups are taken in the directory:

  /home/cudb/systemDataBackup/

• Those temporary backup files, distributed in the entire CUDB system, are copied automatically from those CUDB nodes to the CUDB node where cudbSystemDataBackupAndRestore was ordered, to the following output directory:

  /home/cudb/automatedBackupStorage/<CUDBNodeId>/<DSG>/<BackupFileName>

  Note:

  The automatic copy of the temporary backup files is performed only if the following conditions are met: The CUDB node where cudbSystemDataBackupAndRestore was performed contains enough free space for the backups. cudbSystemDataBackupAndRestore was performed without the -k | --keep-local switch. If any of these conditions are not met, the backup files are kept in the temporary directories, and they will not be copied to one single CUDB node.

  In the above path, the variables stand for the following:

  • <CUDBNodeId> is the CUDB Node Identifier of the node where the backed up data partitions belongs. Refer to CUDB Node Configuration Data Model Description for further information about the CUDB Node Identifier.

  • <DSG> is the DSG Identifier of the DSG where the cluster belongs. If the value is 0, the backup belongs to the PLDB.

  • <BackupFileName> is the name of the backup file: BACKUP-<YYYY-MM-DD_HH-mm-> <CUDBNodeId>.<DSG>.<mysqlndbId>.tar

  This single directory is the final output directory of the system data backup, containing the copies of the system data backup files. For each data partition (PLDB and each of the DSGs) there will be as many TAR files as participating blades or VMs there were in the cluster where the corresponding unit backup was taken.

  Those TAR files contain the following:

  • Metadata file (.ctl extension)

  • Table records file (.data extension)

  • Transaction log file (.log extension)

A sample output of a full CUDB system data backup is shown below:

SC_2_2# cudbSystemDataBackupAndRestore -c

As a result of the example above (where automatic backup file collection was invoked), the following files are generated and stored in the /home/cudb/automatedBackupStorage/ NFS directory:

For the PLDB backup:

/home/cudb/automatedBackupStorage/44/0/BACKUP-2014-05-14_20-11-44.0.3.tar

/home/cudb/automatedBackupStorage/44/0/BACKUP-2014-05-14_20-11-44.0.4.tar

/home/cudb/automatedBackupStorage/44/0/BACKUP-2014-05-14_20-11-44.0.5.tar

/home/cudb/automatedBackupStorage/44/0/BACKUP-2014-05-14_20-11-44.0.6.tar

For DSG1 backup from node 43:

/home/cudb/automatedBackupStorage/43/1/BACKUP-2014-05-14_20-11-43.1.3.tar

/home/cudb/automatedBackupStorage/43/1/BACKUP-2014-05-14_20-11-43.1.4.tar

For DSG2 backup from node 43:

/home/cudb/automatedBackupStorage/43/255/BACKUP-2014-05-14_20-11-43.255.3.tar

/home/cudb/automatedBackupStorage/43/255/BACKUP-2014-05-14_20-11-43.255.4.tar

Checking /home/cudb/automatedBackupStorage on local node
--------------------------------


        Local node: ... EMPTY

Checking /home/cudb/systemDataBackup on nodes
--------------------------------


        44: ... EMPTY
        43: ... EMPTY

CREATE PART
--------------------------------


/home/cudb/systemDataBackup
Listening for current PLDB and DSGs status reports (may take upto 2 minutes)
Starting backup...
Before calling pg_notification
BackupStart [INFO] - PS Notification trying stop notification was successfully sent.
PS Notification successfully sent to : 172.31.233.139
PS Notification failed to : NONE
PS not notified : NONE


cudb_backup ver(1.3.4)

BEGIN MANAGEMENT FOR LAST BACKUP
Backup 2014-05-14_20-11 finished successfully in :
PLDB in CUDB node 44
    NDB node PL0
    NDB node PL1
    NDB node PL2
    NDB node PL3
DSG#1 in CUDB node 43
    NDB node DS1_0
    NDB node DS1_1
DSG#2 in CUDB node 43
    NDB node DS2_0
    NDB node DS2_1
Attempting to de-block Provisioning Gateway. This may take up to a couple of minutes.

COLLECT PART
--------------------------------


        Copying backup piece from node 44 for dsg 0 for ndb node id 3 ... OK
        Copying backup piece from node 44 for dsg 0 for ndb node id 4 ... OK
        Copying backup piece from node 44 for dsg 0 for ndb node id 5 ... OK
        Copying backup piece from node 44 for dsg 0 for ndb node id 6 ... OK
        Copying backup piece from node 43 for dsg 1 for ndb node id 3 ... OK
        Copying backup piece from node 43 for dsg 1 for ndb node id 4 ... OK
        Copying backup piece from node 43 for dsg 2 for ndb node id 3 ... OK
        Copying backup piece from node 43 for dsg 2 for ndb node id 4 ... OK


Please copy /home/cudb/automatedBackupStorage to a safe location to be able to
restore the backup pieces when needed.

ERROR SUMMARY
--------------------------------

None.

CUDB unit data restore utilizes backup files generated by unit data backup to restore the PLDB or a specific DSG cluster.

After restoration, it is necessary to store SQL procedures as described in Recreating Stored Procedures After Data Restore.

Unit data restore commands also allow cluster backups to be restored on clusters which contain a different number of database nodes. This can happen in case of configuring a system on hybrid hardware, having less PLDB data nodes in GEP5 nodes or if one of the involved clusters is crippled. In these cases, the following scenarios apply:

• If the cluster at the backup source has more blade or VM servers taking part in the backup and restore than the cluster targeted for restore (that is also called target cluster), the files for the missing blade or VM server(s) must be transferred to one of the working servers in the target cluster.

• If the cluster at the backup source has less blade or VM servers taking part in the backup and restore than the target cluster, no files must be transferred to the target blade or VM server equivalent to the missing server at the backup source.

Figure 2 shows the file distribution in case of a restoring the backup of a 4-database-node cluster to a 3-database-node cluster.

Figure 2   File Distribution for Restoring a 4-Database-Node Backup in a 3-Database-Node Cluster

Figure 3 shows the file distribution in case a 3-database-node cluster backup is restored on a 4-database-node cluster.

Figure 3   File Distribution for Restoring a 3-Database-Node Backup in a 4-Database-Node Cluster

Compared to data restores taking place between clusters where the number of blades or VMs is the same, the data restore procedure takes more time if the number of blades or VMs is different in the backup source and restore destination clusters.

Stored procedures must be created:

• after system data restore.

• after unit data restore, in case it was performed without the –restored-stored-procedures option.

To restore the stored procedures and application counter procedures on PLDB, enter the following command:

shell> cudbManageStore -p -o restorestoredprocedures

To restore the stored procedures and application counter procedures on a DS, enter the following command:

shell> cudbManageStore -d <dsId> -o restorestoredprocedures

The output is the restoration of the PLDB or a specific DSG cluster in the CUDB node.

Group data restore is basically a set of unit data restore procedures performed on all replicas of the affected data partition (PLDB or DSG). See Unit Data Restore for more information.

Following a successful group data restore, the replication channel must be synchronized properly.

Group data restore affects the information stored in all replicas of the affected the data partition (PLDB or DSG) targeted for restore.

Data stored in other partitions are unaffected by group data restore.

This type of restore affects the information stored in all database units of the CUDB system.

System data restore with the cudbSystemDataBackupAndRestore command is available in two different methods. The main difference between these methods is the location of the backup files:

• System data can be restored from a single source.

• System data can also be restored from distributed sources.

The difference is related to the system data backup procedure which is available both with and without automatic file collection.

The following sections describe how to restore system data with both methods. For the purposes of description, an example CUDB system is used, containing three CUDB nodes with triple geographical redundancy configuration. The system contains three DSGs.

The nodes, DSGs and the DSG replicas selected as the source replica during the system data backup creation are indicated in Example 2.

Example 2   Example CUDB System and Backup Fileset

PLDB in CUDB node 81    PLDB source replica was the replica that resides on Node 81
NDB node PL0            <-
NDB node PL1            <-
NDB node PL2            <-
NDB node PL3            <-
DSG#1 in CUDB node 81
NDB node DS1_0
NDB node DS1_1
DSG#2 in CUDB node 81   DSG2 source replica was the replica that resides on Node 81
NDB node DS2_0          <-
NDB node DS2_1          <-
DSG#3 in CUDB node 81
NDB node DS3_0
NDB node DS3_1

PLDB in CUDB node 82
NDB node PL0
NDB node PL1
NDB node PL2
NDB node PL3
DSG#1 in CUDB node 82   DSG1 source replica was the replica that resides on Node 82
NDB node DS3_0          <-
NDB node DS3_1          <-
DSG#2 in CUDB node 82
NDB node DS2_0
NDB node DS2_1
DSG#3 in CUDB node 82
NDB node DS1_0
NDB node DS1_1

PLDB in CUDB node 83
NDB node PL0
NDB node PL1
NDB node PL2
NDB node PL3
DSG#1 in CUDB node 83
NDB node DS2_0
NDB node DS2_1
DSG#2 in CUDB node 83
NDB node DS1_0
NDB node DS1_1
DSG#3 in CUDB node 83   DSG3 source replica was the replica that resides on Node 83
NDB node DS3_0          <-
NDB node DS3_1          <-

Before loading a backup to a CUDB node, prepare the system data restore by performing the steps of Restoring from a Single Source or Restoring from a Distributed Source (depending on how the source backup files are stored).

During the system restore process, each backup file is transferred automatically to all the CUDB nodes where the specific DSG - referred by the backup file - has a replica. The files are copied to the following NFS directory on the nodes:

/home/cudb/systemDataBackup/

After the copy is finished, the files are extracted and transferred automatically to the local directories of the respective CUDB blades or VMs (see Data Restore for more information). By default, the local directory is the following:

/local/cudb/mysql/ndbd/backup/BACKUP

After the restore is performed, the slave PLDB and DSG replicas are kick-started, that is, forced to connect and synchronize with the master replica.

Perform the steps of Restoring from a Single Source or Restoring from a Distributed Source (depending on how the source backup files are stored) to perform system data restore.

A system data backup can contain misalignments between the data in the PLDB backup and the data in the DSG backup. Therefore, a system data restore requires reconciliation for all DSGs. Refer to the section on requesting reconciliation manually in CUDB System Administrator Guide for more information.

In case any failure occurs during the restore phase, the system sends an error message and raises the Storage Engine, Restore Fault In DS or Storage Engine, Restore Fault In PLDB alarms.

Sample outputs of the CUDB system restore process are provided below both for single source (see Output of Restoring from a Single Source) and distributed source (see Output of Restoring from a Distributed Source) restore processes.

The combined data backup and restore is used to restore a slave PLDB or DSG replica by using a fresh backup taken on the corresponding master replica.

In case any failure occurs during the backup phase, the system raises the Storage Engine, Backup Fault In DS or Storage Engine, Backup Fault In PLDB alarms.

On one hand, the output is a new backup taken on the corresponding master replica:

/local/cudb/mysql/ndbd/backup/BACKUP/BACKUP-YYYY-MM-DD_HH-mm

On the other hand, the output is the restoration of the slave replica in the CUDB node that contains the replica to be restored.