The files discussed as follows are used during import processing.

(a) Import information queue files

The target Datareplicator uses these files to store the extraction definition and update information that is sent from the source system.

Datareplicator stores the received update information sequentially in one of the import information queue files. When this import information queue file becomes full, Datareplicator uses another import information queue file. This is called swapping, and it enables the target Datareplicator to store a large amount of update information and to import a large amount of information in the batch mode.

When all import information queue files become full, Datareplicator re-uses the first file. However, if import of the update information has not been completed for the file that is to be used next, swapping cannot take place. In this case, communication with the source system is broken and update information storage processing is suspended until the next transmission interval. If all update information in the file to be used during import processing in the next transmission interval has been read, Datareplicator resumes storing update information.

For details about the transmission interval, see 4.6.4 Designing the transmission procedure. For details about the transmission interval specified with XDM/DS, see the VOS3 XDM/DS manual.

The following figure shows the procedure for storing data in the import information queue files.

Figure 3-21 Procedure for storing data in the import information queue files

(b) Import-time status files

These files store information such as the reception/import status required for recovery in the event of an error and the extraction definition status in the source system.

(c) Import master status file

This file stores the execution results during initial startup.

(d) Import-time error information files

If reception or import processing results in an error, Datareplicator outputs error details to an error information file. Datareplicator can also output the same information to the syslog file. The syslogout operand in the import system definition specifies whether error information is to be output to the syslog file.

(e) Import-time activity trace files

Datareplicator uses the activity trace files to collect Datareplicator's activity status. These files contain information about Datareplicator's operation and performance. The import-time activity trace files include the import trace files.

To use the activity trace files, specify the int_trc_lvl and int_trc_filesz operands in the import system definition. You can edit the obtained activity trace files with the hdstrcedit command. For details about the hdstrcedit command, see its command syntax in Chapter 7. Command Syntax.

(f) Unimported information files

An SQL statement issued for import processing that results in an error is output to an unimported information file. After import processing is completed, you can check the unimported information files, and then re-execute affected SQL statements and recover the errors.

Datareplicator creates two unimported information files per source system. When the one in use becomes full, Datareplicator starts using the other one. This is called swapping. When swapping occurs, Datareplicator starts output of SQL statements from the beginning of the file, regardless of the file status.

(g) Import system definition file

This file defines the target Datareplicator's overall operating environment, such as the method for establishing connection with the source system and the target HiRDB's access method.

(h) Import environment definition files

These files define operating environments for import processing, such as the names of the import information queue files and the import status files.

(i) Import definition files

These files define detailed information about import processing, such as the correspondence between the table and columns subject to extraction and the table and columns subject to import and the import groups.

(j) Command log files

These files store a record of the dates and times Datareplicator's commands are executed.

(k) Update information definition file

When the source database to be linked using a SAM file is PDM2 E2, the update information definition file defines information about import processing, such as the table and columns subject to extraction, column information, and column redefinition information in the SAM file.

(l) SAM file

When a SAM file is used for data linkage, this is the PDM2 E2's or RDB1 E2's update information extraction SAM file transferred by the file transfer program.

In the case of PDM2 E2, you use the statistical activity analysis utility (PDMJANL) to create SAM files. When you execute this utility, you must either specify EXRANGE=SWAP or omit EXRANGE in the EXTRACT statement. If you execute the utility with any other specification, termination information might not be output to the SAM file, in which case data linkage will fail. In the case of RDB1 E2, you use the update information extraction facility to create an information extraction SAM file. Use a variable-length blocked dataset for the SAM file (fixed-length blocked datasets are not supported for data linkage).

For details about PDM2 E2's statistical activity analysis utility (PDMJANL), see the VOS3 PDM2, PDM2 E2 Utility manual or the VOS1 PDM2 E2 Utility manual.

(m) Unextracted data storage file

When a SAM file is used to link data with the source database, Datareplicator uses the unextracted data storage file for output of update data for a table that is not specified in the extraction statement in the update information definition file or for output of update data information that cannot be imported.

(n) import suppression list file

To use import suppress to skip errors that occur during import processing, create an import suppression list file. For details about using import suppression to skip errors, see 3.3.10(1) Using import suppression to skip errors.

The following figure shows the organization of processes during import processing.

Figure 3-22 Organization of processes during import processing

(a) Import command process

The import command process processes the target Datareplicator's command and issues an instruction to the import master process.

(b) Import master process

The import master process controls the entire target Datareplicator.

(c) Import communication master process

The import communication master process receives a connection request from the source system.

(d) Reception process

The reception process receives update information from the source system and stores it in an import information queue file.

(e) Import definition server process

The import definition server process controls the import process.

(f) Import process

The import process controls the import SQL process.

(g) Import SQL process

The import SQL process creates SQL statements on the basis of the update information and issues them to the target database.

(h) Import UOC process

The import UOC process issues SQL statements to the source database to process update information using a UOC routine.

(i) Update information input process

When a SAM file is used for data linkage, Datareplicator uses the update information input process to store the SAM file transferred from the mainframe database (PDM2 E2 or RDB1 E2) into the update information queue file using the hdssamqin command.

(j) Activity trace collection process

The activity trace collection process collects activity trace information.

(k) Synchronization managing process

The synchronization managing process manages global transactions when the import transaction synchronization facility is used.