The import method that assigns one import process and one import SQL process is called the transaction-based import method. This method reads one transaction at a time from the import information queue file and imports them in order into HiRDB. The following figure shows the organization of processes for the transaction-based import method.

Figure 3-23 Organization of processes for the transaction-based import method

The import method that creates an import group for one or more tables subject to import processing and imports data for one group at a time is called the table-based import method.

If the amount of update data is the same in each table subject to import processing, an improvement in import processing performance can be expected. However, the number of active processes increases, resulting in an increase in memory usage. When you use this import method, take into account the available memory capacity.

The table-based import method is broken down into the following types:

• Table-based partitioning method
• Key range-based partitioning method
• Hash partitioning method

The following figure shows the organization of processes for the table-based import method.

Figure 3-24 Organization of processes for the table-based import method

(a) Table-based partitioning method

The table-based partitioning method creates an import group for one or more tables that are subject to import processing. With this method, Datareplicator assigns one import process and one import SQL process to each import group. Each import process reads in order the update information in the import information queue file and issues SQL statements only to the tables assigned to the import group.

You can expect an improvement in throughput if the target HiRDB is a parallel server and if the tables are not row-partitioned among multiple servers.

(b) Key range-based partitioning method

The key range-based partitioning method creates one import group for one table that is subject to import processing and specifies key range partitioning conditions within the import group. With this method, Datareplicator assigns one import process and as many import SQL processes to the import group as there are key range partitions. The import process reads in order the update information in the import information queue file, and only the import SQL process that satisfies a specified condition issues an SQL statement.

You can expect an improvement in throughput when a target HiRDB is a parallel server, one table is row-partitioned among multiple units, and the key range partitioning conditions are defined by the target Datareplicator in the same manner as with the table's row-partitioning.

If data linkage does not require much workload for the target HiRDB, such as when there is only a small amount of data to be imported at one time or when the table subject to import processing is not large, you might be able to improve the performance of import processing by not using the key range partitioning method because range checking can be skipped.

If HiRDB is configured as described below, use of the key range-based partitioning method might not improve performance because of an increase in the communications load between front-end server and back-end server.

• The front-end server and back-end server are located at different machines.
• The HiRDB table is hash-partitioned.

Additionally, because the conditions for key range partitioning can be either match or range specifications, if consecutive values are sent to a column used as the partitioning key, processing becomes concentrated at the specified front-end server.

If key range partitioning does not improve performance for these reasons, use the hash partitioning method.

(c) Hash partitioning method

The hash partitioning method creates one import group for one table that is subject to import processing. With this method, Datareplicator assigns one import process and as many import SQL processes to the import group as there are hash partitions. The import process reads in order the update information in the import information queue file, and only the import SQL process that satisfies a specified condition issues an SQL statement.

If key range partitioning does not improve performance, use the hash partitioning method. The hash partitioning method can partition consecutive values that are concentrated at one location when the key range partitioning method is used. The hash partitioning method is especially effective when data is imported into a HiRDB/Parallel Server that uses the multi-FES facility.