Details of data linkage recovery using unload log files

The following figure shows the recovery procedure using unload log files.

Figure 9-9 Recovery procedure using unload log files

[Figure]

#: The initialization procedure for extraction includes the extraction definition preprocessing. After initialization has been completed, use the hdeprep command to preprocess the extraction definition. If the extraction definition is not preprocessed, a recovery error will occur.

The following provides the details of the recovery procedure using unload log files.

For the examples of command execution in the Description and example of command execution column, note the following:

Command prompt $ indicates a Datareplicator administrator's action, and # indicates a superuser's action (for the Windows Datareplicator, the user who installed the Datareplicator).
The su command is a UNIX superuser command; it is not supported by the Windows Datareplicator.

The vi command indicates the UNIX vi editor. In the Windows Datareplicator, use an editor such as Notepad (notepad.exe).

Table 9-13 Details of the recovery procedure using unload log files

Item	Target	Description and example of command execution	Input and output	Check item
Step 1: Back up the import error information file	Back-end server subject to error recovery	Save the import error information file because it contains extraction transaction information about the last information that has been imported. Example of command execution: $ cp $HDSPATH/errfile1 $HDSPATH/errfile1.sav $ cp $HDSPATH/errfile2 $HDSPATH/errfile2.sav	--	--
Step 2: Initialize for extraction	Manager node	Execute initialization for extraction. This includes the action preprocess. Example of command execution: $ hdestart -i $ hdeprep -f extfile	--	--
Step 3: Create a linkage recovery backup file (applicable when the system log information is unloaded to the HiRDB file system)^#1	Back-end server subject to error recovery	Create the linkage recovery backup files in order of generations. Example of command execution: $pdfbkup HiRDB-file-system-area-name/HiRDB-file-name backup-file-name	Input file: HiRDB file system containing the unload log files Output file: Linkage recovery backup file	--
Step 4: Divide the unload log files with the logmrg command^#2	Back-end server subject to error recovery	Execute the logmrg command. Example of command execution: $ logmrg back-end-server-name divided-blocks-count output-target-directory input-source-directory {unload-log-file-name\|linkage-recovery-backup-file} [{unload-log-file-name\|linkage-recovery-backup-file}] Note: Make sure that the unload log files or linkage recovery backup files are specified in chronological order, beginning with the oldest file.	Input file: Unload log file or linkage recovery backup file Output file: output-target-directory/unload-log-file-name_sequence-number.unlog $HDEPATH/caplogparm_BES-name	Make sure that a file named $HDEPATH/ caplogparm_BES-name has been created and its creation date is the same as the logmrg command execution date.
Step 5: Specify operands in the recovery information file^#3	Back-end server subject to error recovery	To the file named $HDEPATH/caplogparm_BES-name, add the operands indicating the recovery start location and whether recovery of HiRDB's data linkage-cancelled range is required. Example of command execution: $ vi $HDEPATH/caplogparm_BES-name	Input file: $HDEPATH/caplogparm_BES-name Output file: $HDEPATH/caplogparm_BES-name	--
Step 6: Set up the data linkage recovery facility 1	Node containing the back-end server subject to error recovery	Swap programs for the data linkage recovery facility 1 (transaction retrieval phase). Example of command execution: $ su # setup_tool1 (set_tool1 in Windows Vista and Windows Server 2008)	--	Make sure that the message Setup for <hdecapture_tool1> complete is displayed.
Step 7: Execute the data linkage recovery facility 1	Manager node	Start the source Datareplicator's extraction facility. In this case, make sure that only the extraction facility is started. Example of command execution: $ hdestart -e	Input files: $HDEPATH/hde_toolenv output-target-directory/unload-log-file-name_sequence-number.unlog $HDEPATH/caplogparm_BES-name Output files: tranlist_BES-name res_file_BES-name (applicable only when there is an incomplete transaction)	Use the hdestate command to monitor the status until the node containing the back-end server subject to error recovery or the source Datareplicator has terminated. After termination, perform the following procedure: Make sure that the error information file contains no errors. Make sure that there is no file named res_file_BES under the directory specified in the TOOL_OUTPUT_DIR environment variable. If only the node containing the back-end server subject to error recovery has been terminated, execute the hdestop command to terminate the source Datareplicator.
Step 8: Set up the data linkage recovery facility 2^{#4, #5}	Node containing the back-end server subject to error recovery	Swap programs for the data linkage recovery facility 2 (extraction queue creation phase). Example of command execution: $ su # setup_tool2 (set_tool2 in Windows Vista and Windows Server 2008)	--	Make sure that the message Setup for <hdecapture_tool2> complete is displayed.
Step 9: Execute the data linkage recovery facility 2	Manager node	Start the source Datareplicator's extraction facility. In this case, make sure that only the extraction facility is started. Example of command execution: $ hdestart -e	Input files: $HDEPATH/hde_toolenv output-target-directory/unload-log-file-name_sequence-number.unlog $HDEPATH/caplogparm_BES-name tranlist_BES-name Output file: Extraction information queue file (placing the data to be recovered in queue)	Use the hdestate command to monitor the status until the node containing the back-end server subject to error recovery or the source Datareplicator has terminated. After termination, perform the following procedure: Make sure that the error information file contains no errors. If only the node containing the back-end server subject to error recovery has been terminated, execute the hdestop command to terminate the source Datareplicator.
Step 10: Back up the recovery information file^#5	Back-end server subject to error recovery	Rename the file $HDEPATH/caplogparm_BES-name. Example of command execution: $ mv $HDEPATH/caplogparm_BES-name $HDEPATH/caplogparm_BES-name.sav	--	This step protects from a malfunction if the user forgets to uninstall the facility.
Step 11: Cancel the setup of data linkage recovery facility 2	Node containing the back-end server subject to error recovery	Execute the unsetup_tool command. Example of command execution: $ su # unsetup_tool (unset_tool in Windows Vista and Windows Server 2008)	--	Make sure that the message Unsetup for <hdecapture_tool> complete is displayed.
Step 12: Initialize for import	Manager node	Initialize for import processing. Example of command execution: $ hdsstart -i	--	--
Step 13: Send the extraction data obtained by the data linkage recovery facility	Manager node	Start the source Datareplicator's transmission facility. In this case, make sure that only the transmission facility is started. Example of command execution: $ hdestart -s $ hdestate	--	After starting the transmission facility, use the hdestate command to monitor when the transmission queue file's read offset reaches the write offset. When the read offset has reached the write offset, execute the hdestop command to terminate the source Datareplicator.
Step 14: Stop transmission and import processing	Manager node	Terminate the source Datareplicator's transmission process and the import process. Example of command execution: $ hdestop $ hdsstop	--	--

#1: To check the date of HiRDB files, execute the following command:; pdlogcat_s -i backup-source-HiRDB-file|grep "First use"; The backup-source-HiRDB-file is the HiRDB file from which the linkage recovery backup file was created. Specify the file in the format HiRDB-file-system-area/HiRDB-file-name.

#2: To check the date of an unload log file, execute the following command:; pdlogcat_s -i unload-log-file-name; In the command execution results, the date displayed in the row First use is the date the update log was actually stored in the corresponding unload log file. Make sure that the unload log files specified in the logmrg command are in chronological order of this date.

#3

Add the parameters at the end of the file according to the following rules:

Operand name	Classifi-cation	Setting	Description
RCVR_START	Earlier than 06-02	Optional	You can specify a maximum of 4,096 RCVR_START operands. Format extraction-transaction-information extraction-transaction-information ~ <24 hexadecimal characters> Specifies the extraction transaction information that indicates the start location for error recovery. For details about how to obtain the extraction transaction information, see 9.6.4(1) Obtaining the extraction transaction information (recovery start location).	If this operand is omitted, Datareplicator assumes that the beginning of the first unload log file that is read is the start location for error recovery.
06-02 or later	Optional	You can specify a maximum of 4,096 RCVR_START operands. Format {target-identifier-number\|}[,extraction-transaction-information] target-identifier-number* Specifies the target identifier number specified in the extraction system definition. If an asterisk () is specified as the target identifier number, all transmission targets are assumed. If only the target identifier number is specified in this operand, Datareplicator assumes the beginning of the input unload log file as the recovery start location. If an asterisk () is specified as the target identifier number in this operand and this operand is specified again, an error results. Specifying the same target identifier number more than once also results in an error. extraction-transaction-information ~ <24 hexadecimal characters> For details about how to obtain the extraction transaction information, see 9.6.4(1) Obtaining the extraction transaction information (recovery start location).	For each target, specify the transmission target identifier subject to error recovery and the extraction transaction information indicating the location of error recovery. If this operand is omitted, Datareplicator assumes that all transmission target identifiers are to be recovered and that the beginning of the input unload log file is the recovery start location.
RCVR_RPLSTOP	Optional	Y: HiRDB's linkage-cancelled range is subject to recovery. N or a value other than Y: HiRDB's linkage-cancelled range is not subject to recovery.	If the error recovery range includes HiRDB's linkage-cancelled range, specify whether this linkage-cancelled range is to be subject to recovery. When this operand is omitted, N is assumed.

Operand name

Classifi-cation

Setting

Description

RCVR_START

Earlier than 06-02

Optional

You can specify a maximum of 4,096 RCVR_START operands.

Format

extraction-transaction-information

extraction-transaction-information
~ <24 hexadecimal characters>
Specifies the extraction transaction information that indicates the start location for error recovery. For details about how to obtain the extraction transaction information, see 9.6.4(1) Obtaining the extraction transaction information (recovery start location).

If this operand is omitted, Datareplicator assumes that the beginning of the first unload log file that is read is the start location for error recovery.

06-02 or later

Optional

You can specify a maximum of 4,096 RCVR_START operands.

Format

{target-identifier-number|*}[,extraction-transaction-information]

target-identifier-number
Specifies the target identifier number specified in the extraction system definition. If an asterisk (*) is specified as the target identifier number, all transmission targets are assumed. If only the target identifier number is specified in this operand, Datareplicator assumes the beginning of the input unload log file as the recovery start location.
If an asterisk (*) is specified as the target identifier number in this operand and this operand is specified again, an error results. Specifying the same target identifier number more than once also results in an error.

extraction-transaction-information
~ <24 hexadecimal characters>
For details about how to obtain the extraction transaction information, see 9.6.4(1) Obtaining the extraction transaction information (recovery start location).

For each target, specify the transmission target identifier subject to error recovery and the extraction transaction information indicating the location of error recovery.
If this operand is omitted, Datareplicator assumes that all transmission target identifiers are to be recovered and that the beginning of the input unload log file is the recovery start location.

RCVR_RPLSTOP

Optional

Y:: HiRDB's linkage-cancelled range is subject to recovery.

N or a value other than Y:: HiRDB's linkage-cancelled range is not subject to recovery.

If the error recovery range includes HiRDB's linkage-cancelled range, specify whether this linkage-cancelled range is to be subject to recovery.
When this operand is omitted, N is assumed.

Notes about specifying the RCVR_START operand: You can recover the input unload log files only when the mapping key has not been updated (if the mapping key is updated, nonconformity of data linkage might occur due to duplicated updating during recovery). Additionally, to avoid duplicated update errors at the target, you must set SQLCODE for the missing key error (100) and duplicate key error (-803) in the skip_sqlcode operand in the import environment definition prior to the recovery.

Example (added parameters are underlined)

INPUT = /HiRDBDS/hirdbb/HDE/work/unldlog2_1.unlog,10,UNLDLOG
INPUT = /HiRDBDS/hirdbb/HDE/work/unldlog2_2.unlog,10,UNLDLOG
INPUT = /HiRDBDS/hirdbb/HDE/work/unldlog2_3.unlog,10,UNLDLOG
BLOCKBUF = 22200
RCVR_START = SND01,3BA6E5800000000000000015
RCVR_RPLSTOP = Y

#4

If there is a file named res_file_BES-name, check the file contents and take appropriate action according to the following (this is a text file):

Output information	Description	Action
N.G (start nothing)	The unload log information contains data that is missing the start of a transaction.	Add the immediately preceding generation of unload log file, and then re-execute the procedure starting with step 3. During re-execution, either delete the file named res_file_BES-name or save the file under a different name.
N.G (end nothing)	The unload log information contains data that is missing the end of a transaction.	Make sure that referencing shutdown is in effect on all RDAREAs that contain the source table. When referencing shutdown is in effect: Ignore this output and resume the subsequent recovery procedure. When referencing shutdown is not in effect: Apply referencing shutdown, add the unload log files up to the current point, then re-execute the procedure starting with step 3. During re-execution, either delete the file named res_file_BES-name or save the file under a different name.
N.G (rplstop found) N.G (start nothing,rplstop) N.G (end nothing,rplstop)	There is a transaction within HiRDB's data linkage-cancelled range.	This output information means that HiRDB's data linkage-cancelled range is not subject to recovery. If there is no problem excluding HiRDB's data linkage-cancelled range from recovery processing, ignore this message and resume the subsequent recovery procedure. To include HiRDB's data linkage-cancelled range in the recovery processing, specify RCVR_RPLSTOP=Y, and then re-execute the procedure starting with step 6.

Output information

Description

Action

N.G (start nothing)

The unload log information contains data that is missing the start of a transaction.

Add the immediately preceding generation of unload log file, and then re-execute the procedure starting with step 3.
During re-execution, either delete the file named res_file_BES-name or save the file under a different name.

N.G (end nothing)

The unload log information contains data that is missing the end of a transaction.

Make sure that referencing shutdown is in effect on all RDAREAs that contain the source table.

When referencing shutdown is in effect:: Ignore this output and resume the subsequent recovery procedure.

When referencing shutdown is not in effect:: Apply referencing shutdown, add the unload log files up to the current point, then re-execute the procedure starting with step 3.

During re-execution, either delete the file named res_file_BES-name or save the file under a different name.

N.G (rplstop found)
N.G (start nothing,rplstop)
N.G (end nothing,rplstop)

There is a transaction within HiRDB's data linkage-cancelled range.

This output information means that HiRDB's data linkage-cancelled range is not subject to recovery. If there is no problem excluding HiRDB's data linkage-cancelled range from recovery processing, ignore this message and resume the subsequent recovery procedure.
To include HiRDB's data linkage-cancelled range in the recovery processing, specify RCVR_RPLSTOP=Y, and then re-execute the procedure starting with step 6.

#5: If the node containing the back-end server subject to error recovery also contains other back-end servers, the recovery tool is also executed on those back-end servers. In this case, an error such as missing files (required for recovery) might be detected on back-end servers that are not subject to error recovery (see footnote 3 above). Ignore such errors.

If any errors occur during data linkage recovery via unload log files, Datareplicator outputs the KFRB05009-E message to the error information file for all errors related to data linkage recovery via unload log files. Identify the nature of any error based on the information that is displayed in the function: section of the message. The following table explains the function: section.

Table 9-14 Contents of the function: section in the KFRB05009-E message

Information in the function: section	Description	Action
analyze error	An analysis error occurred in the $HDEPATH/caplogparm_BES-name file.	Check to see if there is a file named $HDEPATH/caplogparm_BES-name. If there is no such file, re-execute the procedure starting with step 3. If the file exists, the operand added to the corresponding file is invalid. Correct the operand, and then re-execute the procedure starting with step 7.^#
env open_error	An open error occurred in the environment variable definition file (hde_toolenv).	Check to see if the environment variable definition file has been created under $HDEPATH. If the file has not been created, create it, and then re-execute the procedure starting with step 7.^#
inttrn error	Initialization of the recovery tool resulted in an error.	This error occurs only when there is not enough memory. Terminate another program to increase the available memory, and then re-execute the procedure starting with step 7.^#
open_error	An open error occurred in an unload log segment file.	The unload log segment files might not have been created correctly by the logmrg command. Re-execute the procedure starting with step 3.^#
read_error	A read error occurred in an unload log segment file.	The unload log segment files might not have been created correctly by the logmrg command. Re-execute the procedure starting with step 3.^#
trnout error	An output error occurred in the transactions list.	A space shortage might have occurred in the output target directory for the transactions list. Change the output target directory, and then re-execute the procedure starting with step 7.^#
trnget error	An input error occurred in the transactions list.	The transactions list has not been created correctly. Execute step 11^#, and then re-execute the procedure starting with step 6.^#
blk_invalid	An invalid log block was detected in the unload log file.	This is an internal conflict error. Contact the developer.
start point err	Specified extraction transaction information is invalid.	Invalid extraction transaction information was specified in the RCVR_START operand, or some required unload log files were not specified for input. If the specified extraction transaction information is invalid, correct it, and then re-execute the procedure starting with step 7;^# if required unload log files are missing, add the files, and then re-execute the procedure starting with step 3.^#
seq_invalid	The order of log blocks is invalid in the unload log files	The order of the unload log files or linkage recovery backup files specified during execution of the logmrg command might be invalid (not sorted in ascending order of the log output date). Check the order of the input unload log files or linkage recovery backup files, and then re-execute the procedure starting with step 3.^#
param len error	Length of the directory path name specified in the TOOL_OUTPUT_DIR environment variable is invalid.	Correct the value, and then re-execute the procedure starting with step 7.^#
remain trnentry	The number of transactions in the transactions list created by the data linkage recovery facility 1 does not match the number of transactions recovered by the data linkage recovery facility 2.	This is an internal conflict error. Contact the developer.

#: See (1) Recovery procedure using unload log files.

9.6.5 Details of data linkage recovery using unload log files

(1) Recovery procedure using unload log files

(2) Handling errors during data linkage recovery via unload log files