9.3.1 Designing system log files

When system log files need to be swapped, HiRDB will terminate abnormally if there are no swappable target system log files. To prevent this, HiRDB has a facility for monitoring the free area for system log files (monitoring free area for system log files facility). This facility operates when the percentage of available free area for the system log files reaches a warning value. Select one of the following two levels:

Level 1:
Output the KFPS01162-W warning message when the percentage of free area for system log files reaches the warning level.
Level 2:
When the percentage of free area for the system log files reaches the warning level, suppress all further scheduling of new transactions, terminate forcibly all transactions in the server, and output the KFPS01160-E message. This controls the output volume of the system logs.

If level 2 is selected, all transactions in the server are terminated forcibly when there becomes insufficient free space in the system log files. Because of the severity of this action, the system log files should be designed quite carefully.

The following explains some of the design considerations for system log files.

Organization of this subsection
(1) Design considerations
(2) Design for improved reliability
(3) Facility for parallel output of system logs (AIX 5L only)
(4) Record length of a system log file
(5) Defining the system log files

(1) Design considerations

  1. System log files are not required for a utility special unit.
  2. Specify the same record length and number of records for all system log files.
  3. The number of system log files that can be created is 6 to 200 groups.
    If HiRDB is restarting after an abnormal termination due to insufficient space for the system log files, the number of system log files to be added must be the same as the number that were already created. For example, if 50 groups of 50 system log files had been created, each of the maximum size (2 gigabyte), then 50 groups of 50 system log files of the maximum size should now be added. Therefore, it is recommended that system log files always be created in units of 100 groups.
  4. The maximum total size of the system log files is 400 gigabytes.
  5. To reduce the number of unload operations, it is advisable to create many large system log files.
  6. A file that is involved in many input/output operations (such as a log unload file) should not be created on the same disk that contains HiRDB directory.
  7. The amount of space required for one system log file must satisfy the condition shown following.
    Size of one system log file (bytes) [Figure][Figure](a + 368) [Figure] c [Figure][Figure]c [Figure]b [Figure]d
    a: Value of pd_log_max_data_size operand
    b: Value of pd_log_sdinterval operand
    c: Value of pd_log_rec_leng operand
    d: Value of pd_spd_assurance_count operand
  8. The total amount of space required for system log files (if dual system log files are used, the total amount of space for one version) must satisfy the condition shown in Formula 1 and 2 below.
    Formula 1
    Total size of all system log files (bytes) [Figure][Figure] a[Figure] c[Figure][Figure] 3 [Figure]b
    a: Output system log size
    For details about determining this value, see 18.1.1(2) Determining the size of system log information.
    b: Record length of system log files specified in the pd_log_rec_leng operand
    c: Substitute the following values:
    If pd_log_rec_leng = 1024: 1000
    If pd_log_rec_leng = 2048: 2000
    If pd_log_rec_leng = 4096: 4000
    Formula 2
    Total size of all system log files (bytes) [Figure] 3 [Figure] a[Figure] (b + 1)
    a:
    Size of system log information that may be output at the corresponding server while executing the database updating transaction with the longest execution time.
    For details about the formula for estimating the size of system log information, see 18.1 Determining the size of system log files.
    b: Value of pd_spd_assurance_count operand
    Number of guaranteed-valid generations for synchronization point dump files.
(a) Effects on operations of the number of generations of system log files

If the total size of the system log files is unchanged, the size of each generation will depend on how many generations of system log files are being maintained. Table 9-3 shows the effects of the number of generations of system log files on operations. The total size of the system log files is unchanged.

Table 9-3 Effects on operations of the number of generations of system log files

Comparison itemSystem log file configuration
Small number of generationsLarge number of generations
Size of each generation of system log filesBecomes larger.Becomes smaller.
Swap intervalBecause the size of each generation of system log files becomes larger, the swap interval becomes longer.Because the size of each generation of system log files becomes smaller, the swap interval becomes shorter.
Unload frequencyBecause the swap interval becomes longer, the unload frequency becomes lower.Because the swap interval becomes shorter, the unload frequency becomes higher.
Effects on the system log size when something such as a disk failure makes several generations of system log files unusable
  • Because the size of each generation of system log files becomes larger, the log volume used for database recovery in the event of a disk failure increases, and the time required for database recovery increases.
  • If the decrease in the system log volume is large, the effects of the decrease in system log volume will have increasing effects on HiRDB operations.
  • Because the size of each generation of system log files becomes smaller, the log volume used for database recovery in the event of a disk failure decreases, and the time required for database recovery decreases.
  • If the decrease in the system log volume is small, the effects of the decrease in system log volume will have decreasing effects on HiRDB operations.

In normal operations, the lower the number of generations of system log files, the more advantageous the swapping interval and the unload frequency will become. However, if there is a failure, the effects on operations will be reduced with a larger number of log file generations.

(2) Design for improved reliability

(a) System log file duplexing

When system log file duplexing is used, HiRDB acquires the same system log information in both versions. In the event of an error on one of the versions, the system log can be read from the other version, thereby improving system reliability. When dual system log files are used, they must be used under the management of HiRDB rather than using a mirror disk. When using dual system log files, create the files for each system on a separate hard disk.

To use dual system log files, specify the following operands in the server definition:

(b) Single operation of system log files

Single operation of system log files is employed when dual system log files are used.

In the event of an error in a system log file, processing can continue using the normal version of the system log file without having to terminate the HiRDB unit abnormally even if neither system has a usable system log file. This is called single operation of system log files. To perform single operation of system log files, specify pd_log_singleoperation=Y in the server definition.

As opposed to single operation of system log files, continuing processing using both versions of system log files (normal processing mode) is called double operation of system log files.

(c) Automatic opening of system log files

If there is no overwrite-enabled system log file at the time of a HiRDB restart, but a reserved file is available, then HiRDB continues processing by opening the reserved file and placing it in overwrite-enabled status. This is called automatic opening of system log files.

To perform automatic opening of system log files, specify pd_log_rerun_reserved_file_open=Y in the server definition.

(3) Facility for parallel output of system logs (AIX 5L only)

When dual system log files are used, a facility for simultaneous output of system log information to both files using the Asynchronous I/O Subsystem,* which is an AIX 5L function, is called the facility for parallel output of system logs. By processing the output of system log information to both files in parallel, the time required for log output can be reduced.

* The Asynchronous I/O Subsystem must be installed and enabled in advance. If the Asynchronous I/O Subsystem is not enabled, HiRDB cannot start, resulting in abnormal termination. For AIX 5L V5.2 or later, there are legacy and POSIX versions of the Asynchronous I/O Subsystem; you should enable the legacy version of the Asynchronous I/O Subsystem, because that is the version that HiRDB uses. For details about the Asset Information Manager, see the AIX 5L documentation.
For details about the Asset Information Manager settings for using the facility for parallel output of system logs, see (d) Tuning the Asynchronous I/O Subsystem.
(a) Recommended configuration

Although you can define for each server whether or not the facility for parallel output of system logs is to be used, we recommend that you apply this facility to all servers. We also recommend that you place the primary and secondary files on separate devices in order to further reduce the time required for output of log information.

(b) Definition method

In the server definition, specify pd_log_dual_write_method=parallel.

(c) Notes
  1. If the Asynchronous I/O Subsystem is not enabled, HiRDB cannot start, resulting in abnormal termination. Either enable the Asynchronous I/O Subsystem, omit the pd_log_dual_write_method operand, or specify serial, and then restart HiRDB. If the Asynchronous I/O Subsystem parameter is set to STATE to be configured at system restart = available, HiRDB starts automatically.
  2. The facility for parallel output of system logs is not applicable to system log files placed in regular files. If you add system log files, place them in character special files.
  3. The facility for parallel output of system logs is applied only when both primary and secondary current files are placed in character special files and system log information can be output to those current files (they are not in closed, reserved, or error status). Parallel output of system logs does not take place, regardless of the system definition, if the current file satisfies either of the following conditions:
    • The primary or secondary current file is placed in a regular file.
    • The primary or secondary current file is in a status such that no log information can be output to it.
  4. When the servers using the facility for parallel output of system logs are run on multiple server machines, such as when the system switchover facility is used, and the Asynchronous I/O Subsystem is not enabled at any of those server machines, startup of a standby unit or system switchover fails. Make sure that Asset Information Manager is enabled at all server machines.
(d) Tuning the Asynchronous I/O Subsystem

To use the facility for parallel output of system logs, you must set the following parameters after installing Asset Information Manager:

Parameter nameRecommended value
STATE to be configured at system restartavailable
STATE of FastPathenable*

* This is the default value; there is no need to change it. If the value is changed, performance is degraded.

No other parameters require tuning.

(4) Record length of a system log file

Use the pdloginit command's -l option to specify the record length of a system log file. You can select 1024, 2048, or 4096 as the record length. By changing the record length, you may be able to reduce the size of system log file.

(a) When constructing a new HiRDB

When constructing a new HiRDB, you should select 1024 as the record length. To do this, specify a value of 1024 in the pd_log_rec_leng operand in the server definition.

(b) When already running a HiRDB

Change the record length of system log files on the basis of the average block length that is output to the system log file (OUTPUT BLOCK LENGTH), which is obtained from the statistical information about system activity by the statistics analysis utility (pdstedit). You should change the record length as follows:

Notes about changing the record length:
  • If records are short, the number of input/output operations in the file increases for large amounts of data; however, the amount of free space is reduced due to rounding up to the next HiRDB record unit, resulting in higher file utilization efficiency.
  • If records are long, the number of input/output operations in the file decreases for large amounts of data; however, the amount of free space is increased due to rounding up to the next HiRDB record unit, resulting in lower file utilization efficiency.
How to change the record length:
For details about how to change the record length of system log files, see the manual HiRDB Version 8 System Operation Guide.

(5) Defining the system log files

The pdlogadfg and pdlogadpf operands are used to define the correspondence between file groups and the created system log files.