The source Datareplicator uses a system log I/O buffer to read system log information from the system log file. The source Datareplicator takes from the system log information it has just read only the update information for the table subject to extraction processing, as specified in the extraction definition, and stores this information in an extraction information queue file. This is the use the source Datareplicator makes of the system log I/O buffer. The following figure provides an overview of the extraction procedure.

Figure 4-31 Overview of the extraction procedure

(a) System log I/O buffer

When the source Datareplicator starts, it allocates a system log I/O buffer in local memory. This section discusses how you estimate and specify the size of the system log I/O buffer.

Estimating the size of the system log I/O buffer

• The system log I/O buffer is used to extract system log information from the system log file. As the size of the system log I/O buffer increases, the number of read/write operations decreases, because more update information can be extracted from the system log file at one time. However, if the source Datareplicator terminates abnormally, restart processing requires more time because more update information must be extracted again during the restart.
• The size of the system log I/O buffer must be at least the value specified in pd_log_max_data_size for the source HiRDB.
• If the source HiRDB is a parallel server, you must estimate a system log I/O buffer size for each back-end server.

Specifying the size of the system log I/O buffer

• You use the logiosize operand in the extraction environment definition to specify the size of the system log I/O buffer. Datareplicator extracts update information from the system log file based on this size.

(b) Extraction information queue I/O buffer for extraction

When the source Datareplicator starts, it allocates extraction information queue I/O buffers in local memory. This section discusses how you estimate and specify the size of the extraction information queue I/O buffer for extraction.

Estimating the size of the extraction information queue I/O buffer for extraction

• The source Datareplicator uses the extraction information queue I/O buffer for extraction to store extracted update information in an extraction information queue file. As the size of the extraction information queue I/O buffer for extraction increases, the number of read/write operations decreases, because more update information can be stored in the extraction information queue file at one time. However, if the source Datareplicator terminates abnormally, restart processing requires more time because more update information must be extracted again during the restart.
• The size of the extraction information queue I/O buffer for extraction must be at least 1 less than the value specified in the queuesize operand in the extraction environment definition.
• Avoid a specification that results in a remainder from the formula (queuesize operand value in the extraction environment definition - 1) specified value. If there is a remainder, the size of the space in the extraction information queue file equivalent to the remainder will not be used.
• If the source HiRDB is a parallel server, you must estimate an extraction information queue I/O buffer size for each back-end server.
• An extraction process uses one extraction information queue I/O buffer.

Specifying the size of the extraction information queue I/O buffer for extraction

• You use the quiosize operand in the extraction environment definition to specify the size of the extraction information queue I/O buffer for extraction. Datareplicator uses this size for allocating the extraction information queue I/O buffer for extraction and the extraction information queue I/O buffers for transmission. For details about the extraction information queue I/O buffers for transmission, see 4.6.4 Designing the transmission procedure.

The source Datareplicator uses the extraction master process to monitor for errors in the extraction node master process, extraction process, and transmission process (transmission master process). Error detection is faster if you use a short error monitoring interval. However, if the error monitoring interval is too short, the workload required for communications with the server becomes high.

You use the watchintvl operand in the extraction system definition to specify the error monitoring interval.

If the source HiRDB is a parallel server, the source Datareplicator executes extraction processing for each back-end server. In such a case, the source Datareplicator's extraction environment definition can include the environment common to all back-end servers and the environment for a specific back-end server.

You define the common environment under commondef and a specific environment under besdef(server-name) in the extraction environment definition. If you specify the same operand in both commondef and besdef(server-name), the specification in besdef is effective for the server specified in besdef.