Storage Engine, Data Inconsistency between Replicas Found in PLDB, Minor
Ericsson Centralized User Database

Contents

1Introduction
1.1Alarm Description
1.2Prerequisites

2
Procedure

Glossary

Reference List

1   Introduction

This instruction concerns alarm handling for the Storage Engine, Data Inconsistency between Replicas Found in PLDB, Minor alarm.

1.1   Alarm Description

This alarm is raised as a result of issues found when running a Consistency Check on a Processing Layer Database (PLDB) slave replica by means of the cudbConsistencyMgr command. For further information about the command, refer to CUDB Node Commands and Parameters, Reference [1]. For more information about Consistency Check, refer to CUDB Consistency Check, Reference [2].

The alarm is issued in the following situation:

The possible alarm causes and the corresponding fault reasons, fault locations, and impacts are described in Table 1.

Table 1    Alarm Causes

Alarm Cause

Description

Fault Reason

Fault Location

Impact

Divergences between the slave replica and its master replica.

The number of divergences between the slave replica and its master replica found when running a consistency check is lower than the threshold.

Wrong backup restored on the slave replica.

Affected PLDB replica.

If this slave replica becomes the master replica, there might be a service impact for the subscribers affected by the data inconsistency.

Slave replica addressed by faulty maintenance operation.

The alarm attributes are listed and explained in Table 2.

Table 2    Alarm Attributes

Attribute Name

Attribute Value

Auto Cease

No

Module

STORAGE-ENGINE

Error Code

21

Timestamp First

Date and time when the alarm was raised for the first time.

Repeated Counter

Number which indicates how many times the alarm was raised.

Timestamp Last

Date and time of the most recent alarm raised.

Resource ID

.1.3.6.1.4.1.193.169.1.1.21

Alarm Model Description

Data inconsistency between replicas found, Storage Engine.

Alarm Active Description

Storage Engine (PLDB): Data inconsistency between replicas found, minor (task <TASKID>) .

ITU Alarm Event Type

processingErrorAlarm (4)

ITU Alarm Probable Cause

databaseInconsistency (160)

ITU Alarm Perceived Severity

(5) - Minor

Originating Source IP

Node ID where the alarm was raised.

Sequence Number

Number which indicates the order in which alarms were raised.

In Table 2, the indicated variables are as follows:

1.2   Prerequisites

This section provides information on the documents, tools, and conditions that apply to the procedure.

1.2.1   Documents

Before starting this procedure, ensure that you have read the following documents:

1.2.2   Tools

Not applicable.

1.2.3   Conditions

Not applicable.

2   Procedure

When this alarm is raised, perform the following steps:

  1. Locate and identify the Lightweight Directory Access Protocol (LDAP) tree log based on the Originating Source IP and <TASKID> parameters in the alarm as follows:
    1. Find the CUDB node with Originating Source IP.
    2. Search for the /local/cudb_ddci/replica_check/cudbDsuDiff_op_TASKID*.xml file on both System Controllers (SCs) of the node.
  2. Align the slave replica to its master in the PLDB as follows:
    • If this node is still a PLDB slave node, and the node with Originating Source IP is still the master node of the PLDB, perform a combined unit data backup and restore on the PLDB against this CUDB node.
      Note:  
      When performing a PLDB restore, the cudbUnitDataBackupAndRestore command does not work, just exits with an error message, if any of the DS clusters in the node is the master replica of its data partition. Therefore, it is required to move the DSG masterships manually before performing the PLDB restore.

      Also, if the PLDB has more than one slave, then consider ordering consistency check for the rest of slaves after restoring the backup.


    • If this node (which was the checked slave at the time of the check) has become a PLDB master node since raising the alarm, perform a combined unit data backup and restore on the PLDB against the CUDB node with Originating Source IP.
      Note:  
      Do not perform combined backup and restore if that has been done since the mastership change of the PLDB.

      Also, when performing a PLDB restore, the cudbUnitDataBackupAndRestore command does not work, just exits with an error message, if any of the DS clusters in the node is the master replica of its data partition. Therefore, it is required to move the DSG masterships manually before performing the PLDB restore.

      Finally, if the PLDB has more than one slave, then consider ordering consistency check for the rest of slaves after restoring the backup.


    • In any other cases, ignore the results. Consider that the PLDB may not be consistent and further checks—involving the new master—are needed to verify it.
  3. Analyze the LDAP tree log file as described below to find the data impacted by the inconsistency:
    • For impacted entries in the ou=identities,<rootDn> branch, validate the subscribers with those public identities, or consider reprovisioning them. Refer to the application Front End (FE) documentation for the procedure to validate or reprovision subscribers.
    • For impacted entries in the ou=mscCommonData,<rootDn> or ou=servCommonData,<rootDn> branches, validate or reprovision the impacted entry. Refer to the application FE documentation for the procedure to validate or reprovision common data.
    • For impacted entries in the ou=admin,<rootDn> branch, validate that the LDAP user data is correct, or delete, then add again the impacted LDAP user by following the procedure described in CUDB System Administrator Guide, Reference [6].

    Refer to CUDB LDAP Interwork Description, Reference [4] for more information on the CUDB Main Directory Information Tree (DIT).

  4. If the LDAP tree log contains an internal error, notify the next level of support. To interpret the contents of the LDAP tree log, refer to CUDB Consistency Check, Reference [2].
  5. Clear the alarm manually as described in CUDB Node Fault Management Configuration Guide, Reference [3].

To find out where the master replicas are, refer to CUDB System Administrator Guide, Reference [6]. For further information about the combined unit data backup and restore procedure, refer to CUDB Backup and Restore Procedures, Reference [5].


Glossary

For the terms, definitions, acronyms, and abbreviations used in this document, refer to CUDB Glossary of Terms and Acronyms, Reference [7].


Reference List

CUDB Documents
[1] CUDB Node Commands and Parameters.
[2] CUDB Consistency Check.
[3] CUDB Node Fault Management Configuration Guide.
[4] CUDB LDAP Interwork Description.
[5] CUDB Backup and Restore Procedures.
[6] CUDB System Administrator Guide.
[7] CUDB Glossary of Terms and Acronyms.
Other Ericsson Documents
[8] System Safety Information.
[9] Personal Health and Safety Information.
Legal | © Ericsson AB 2018