Hitachi

JP1 Version 12 JP1/Automatic Job Management System 3 System Design (Configuration) Guide


F.1 Overview of database backups

You can back up all the tables in a JP1/AJS3 database (embedded database) to a file in case a failure occurs. If an error occurs in the scheduler database, the backup file can be used to restore the database to its state at the time the backup was taken.

When the embedded database is used, JP1/AJS3 creates system log files containing historical information about updates to the scheduler database. Using the database backup file and the information logged to these system log files, you can restore the scheduler database to its most recent state.

Creating backup files and restoring the database differ according to how the JP1/AJS3 system is operated. The following backup and recovery methods can be used:

For more information about the two backup and restore methods, see 23.1.1 Examining the embedded-database operating environment and operating method in the JP1/Automatic Job Management System 3 Configuration Guide.

For an overview of the backup enhancement function, see 5.2.1(5) Backing up and restoring an embedded database.

This section describes the operation that uses unload log files and operation that does not use the system log.

Organization of this subsection

(1) Restoring the scheduler database from unload log files

The following describes how to restore the scheduler database using the unload log files created automatically by the embedded database in addition to the database backup file.

The following figure shows how to recover from a database error using unload log files.

Figure F‒1: Recovery after an error (using unload log files)

[Figure]

(a) Duplicating the system files

In a configuration where the system files used by the embedded database are not duplicated, the embedded database stops when an error occurs in a system file.

If the system files are duplicated, and a disk error or other problem occurs in one set of system files, the system files can be restored the moment the error occurs and in the state they are in right before the error occurs. Note that file duplication requires more disk space than for unduplicated system files.

(b) Using unload log files

■ Automatic log unload function

The embedded database swaps the system log file to an alternate system log file at the following times:

  • When the current system log file is full

  • When the ajsembdbbackup command is executed

  • When the ajsembdboplog command is executed with the -w option specified

  • When the embedded database is restarted (only if it was stopped normally during the previous run)

When the current system log file is swapped out, it waits to be unloaded#1 and cannot be re-assigned. To make this system log file available again, it must be unloaded.#2 The embedded database automatically unloads the waiting system log files to a specified directory. The unloaded files are called unload log files, and the function that unloads them is called the automatic log unload function. These unload log files are used in addition to the database backup file to restore the database after a failure.

#1

In this state, the system log file contains historical update information, which is needed for a recovery, and cannot be overwritten. The file cannot be swapped in as an alternate system log file. It must be unloaded first.

#2

Unload here means to back up the data in the system log file.

■ Unload log file size

The size of the unload log files depends on the scale you set when configuring the environment for the embedded database. The table below lists the size of one unload log file for each of the specifiable scales. Refer to the values in the table when estimating the disk space required for unload log files.

Table F‒1: Size of an output unload log file

Scale specified when configuring the environment for the embedded database

Size of one unload log file

Large scale

Approx. 1,200 megabytes

Medium scale

Approx. 230 megabytes

Small scale

Approx. 30 megabytes

If you have increased the maximum size of a system log file using the ajsembdbaddlog command, compare the size specified in the -s option of the command with the size shown in Table F-1. The larger of the two values will be the maximum size of an unload log file. If you have increased the system log file size using the function for automatically expanding the system log, the maximum size of an unload log file will be three times the size given in Table F-1.

In either case, use the larger value to estimate the disk space required for unload log files.

Note that the size of an unload log file could vary depending on the output timing.

■ Estimating when the disk containing the unload log directory will become full

Unless they are deleted or moved to another disk, unload log files continue to be created in the same directory. The number of files keeps growing as JP1/AJS3 continues running, eventually filling the disk in which the unload log directory resides. When the disk is full, the automatic log unload function stops, resulting in Issue caused by the automatic log unload function stopping described below.

For this reason, you need to estimate when the disk is likely to become full. Back up the embedded database before that point, and delete unload log files created before the backup was taken or move them to another disk. If backing up the database before the disk is full is not feasible, temporarily move the unload log files to another disk to free up space on the disk containing the unload log directory. For details about how to delete or move unload log files, see B.2(3) Managing backup files and unload log files in the JP1/Automatic Job Management System 3 Administration Guide.

The rest of this subsection describes how to estimate at what point the disk with the unload log directory is likely to become full.

The following table provides guidance on how much information can be output to one system log file.

Table F‒2: Maximum amount of information output to one system log file

Scale specified when configuring the environment for the embedded database

Number of jobs or jobnets executed per day

Large scale

Approx. 50,000

Medium scale

Approx. 9,600

Small scale

Approx. 1,200

Note

These values vary depending on the operations that are performed on jobs and jobnets, and how often units are created, redefined, deleted, and so on. Adjust the estimated value according to your system operation.

Using the values in the above table and the size of one unload log file as listed in Table F-1, you can estimate how many days it will take for the disk to become full from the time operation starts.

An estimation example is shown below:

Conditions:

Scale of the embedded-database environment: Large

Disk space required for the unload log directory: 10 gigabytes

Calculation:

10 gigabytes / 1,200 megabytes = 8 days

Under these conditions, you can predict that the disk will be full, at the earliest, at the end of operation on the 8th day from the start of operation.

■ Issue caused by the automatic log unload function stopping

When the automatic log unload function stops, the embedded database no longer unloads system log files. If files are not unloaded, the number of system log files waiting to be unloaded continues to increase. If there is no system log file available when the current system log file is ready to be swapped out, the embedded database terminates abnormally.

For details about what causes the automatic log unload function to stop, see Table F-3.

■ Monitoring the operating status of the automatic log unload function

If the automatic log unload function stops, Issue caused by the automatic log unload function stopping described above occurs. The operating status of the automatic log unload function therefore needs to be monitored at regular intervals.

You can use either of the following two methods to monitor whether the function is active:

Monitoring using a message

If the automatic log unload function stops, message KFPS01150-E is output to the Windows event log or to syslog in UNIX. Determine whether the automatic log unload function is active according to whether this message appears.

Monitoring using a command

You can check whether the automatic log unload function is active by executing the ajsembdboplog command with the -s option specified, as in the following example. Here, the environment for the embedded database whose ID is _JF0 has already been set up:

ajsembdboplog -s -id _JF0
 
HOSTNAME : host_name (180252)
SERVER_NAME:ajs2
AUTO_LOG_UNLOAD  NOW_UNLOAD_LOG_GROUP  CREATE_DIR
          ACTIVE                    ****  K:/logback
CURRENT LOG GENERATION INFO.
LOG_GROUP GEN_NO.  SERVER_RUN_ID RUN_ID   UNLOAD_FILE_NAME
     log1        1   43c4ad0d      43c4acf3 ajs2_43c4ad0d0001_log1

In the execution result, the string (underlined portion) shown for AUTO_LOG_UNLOAD indicates the operating status of the automatic log unload function. The function is active if ACTIVE appears, but has stopped if STOP appears.

If you find that the automatic log unload function has stopped, take the action described in Table F-3, and then execute the ajsembdboplog command with the -r option specified, as in the following example:

ajsembdboplog -r -id _JF0

This command restarts the automatic log unload function.

■ Reasons for the automatic log unload function to stop

The following table lists the reasons why the automatic log unload function might stop, and describes what action to take in each case.

Table F‒3: Reasons for the automatic log unload function to stop, and corrective actions

Reason

Action

An error occurred on the disk that contains the unload log directory.

Back up the embedded database in case the unload log files are lost. After correcting the disk error, execute the ajsembdboplog command with the -r option specified, as follows:

ajsembdboplog -r

This command restarts the automatic log unload function.

The disk that contains the unload log directory is full.

For details about what action to take when the disk is full, see B.2(3) Managing backup files and unload log files in the JP1/Automatic Job Management System 3 Administration Guide. After taking the indicated action, execute the ajsembdboplog command with the -r option specified, as follows:

ajsembdboplog -r

This command restarts the automatic log unload function.

The unload log directory is unavailable for either of the following reasons:

  • Incorrect permissions are set.

  • No such directory exists.

After removing the cause of the error, execute the ajsembdboplog command with the -r option specified, as follows:

ajsembdboplog -r

This command restarts the automatic log unload function.

The ajsembdboplog command was executed with the -t option specified.

Execute the ajsembdboplog command with the -r option specified, as follows:

ajsembdboplog -r

This command restarts the automatic log unload function.

■ Recovery when the embedded database terminates abnormally because there is no alternate system log file

If the embedded database terminates abnormally because there is no system log file to swap in, take the following steps:

  1. Stop the scheduler service and all other services that access the database.

  2. Unload the system log.

    Execute the ajsembdboplog command to unload the system log file that is waiting to be unloaded. For the output size per file, see Table F-1.

  3. Start the embedded database.

    Execute the ajsembdbstart command to start the embedded database. The format of the command differs in Windows and UNIX, and according to the status of the embedded database.

    • In Windows:

      Execute the ajsembdbstart command without specifying any options other than the -id option.

    • In UNIX:

      The format of the ajsembdbstart command depends on the status of the embedded database. To check the database status, execute the ajsembdbstatus command. In the following execution example, the environment for the embedded database whose ID is _JF0 has already been set up:

      ajsembdbstatus -s ust -id _JF0

      HOSTNAME : host_name(144852)

      SYSTEMID : ajs2

      UNITID : unt1

      ENTRYHOST : host_name

      PAIRHOST :

      UNIT-STAT FES-STAT SETUP-STAT

      STOP ******** SETUP

      In the execution result, the string (underlined portion) shown for UNIT-STAT indicates the status of the embedded database. How you execute the ajsembdbstart command is determined by this information. If UNIT-STAT is STOP, execute the ajsembdbstart command with only the -id option specified. If UNIT-STAT is PAUSE, execute the ajsembdbstart command with the -id and -R options specified.

  4. Start the services that you stopped at step 1.

    Hot-start or warm-start the JP1/AJS3 services. Before you do so, check whether the status of the scheduler database is consistent with the actual job execution status. As the status of the scheduler database can only be preserved until just before the embedded database terminated abnormally, there might be inconsistencies with other control information. If it is too difficult to determine whether the scheduler database is consistent with the actual job execution status, cold-start the JP1/AJS3 services and register the jobnets for execution.

(c) Cautionary notes

Note the following points when using unload logs.

■ When configuring the environment

  • Duplication of system files requires more disk space than for unduplicated system files. For the amount of disk space required, see 23.1 Preparation for using an embedded database in the JP1/Automatic Job Management System 3 Configuration Guide.

  • Unload log files cannot be used together with the backup enhancement function. For an overview of the backup enhancement function, see 5.2.5 Backing up and recovering an embedded database by using the backup enhancement function.

■ When using unload logs

  • Unless they are deleted or moved to another disk, unload log files continue to be created in the same directory. The number of files keeps growing as JP1/AJS3 continues running, putting pressure on the disk that contains the unload log directory. Once you back up the database, you can delete unload log files created prior to the time at which you took the backup. For details about how to delete or move unload log files, see B.2(3) Managing backup files and unload log files in the JP1/Automatic Job Management System 3 Administration Guide.

  • When the automatic log unload function stops, the embedded database no longer unloads system log files, so the number of system log files waiting to be unloaded continues to increase. If there is no system log file available when the current system log file is ready to be swapped out, the embedded database terminates abnormally. You must therefore monitor whether the automatic log unload function is active. For the procedure, see Monitoring the operating status of the automatic log unload function in (b) Using unload log files.

  • If you take backup while the JP1/AJS3 service is active, conflict between the ajsembdbbackup command and the job execution process will result in degraded performance in both processes. Take backup at a time when the least amount of jobs are being executed.

■ When restoring the database

  • If the scheduler database is backed up while the JP1/AJS3 service is active, in addition to the backup file, you will require unload log files output since the backup was taken to perform a restoration. If the unload log files have been deleted, back up the database again because you will not be able to restore it using only the backup file created while the JP1/AJS3 service was active.

  • When restoring the scheduler database using unload log files, you will need all the unload log files output since the backup was taken (that is, all those created since the ajsembdbbackup command was executed).

  • When restoring the scheduler database using unload log files, we recommend that you specify the -ld option in the ajsembdbrstr command. If you use the -l option, you must specify in date order all the unload log files needed to restore the database, starting from the oldest. If you specify the files in the wrong order or miss any file, the ajsembdbrstr command terminates with an error. For the command syntax, see ajsembdbrstr in 3. Commands Used for Normal Operations in the manual JP1/Automatic Job Management System 3 Command Reference.

(2) Restoring the scheduler database without using the system log

The following describes how to restore the scheduler database from a backup file only, without using any system log files. This is the simplest method, requiring no unload log files. The following figure shows how to recover from a database error without using the system log.

Figure F‒2: Recovery after an error (without using the system log)

[Figure]

With this method, you do not need to monitor the system log files. However, as the system log is not used at recovery, any updates since the backup file was created cannot be restored.

(a) Cautionary notes

Note the following points when not using the system log at recovery.

■ When configuring the environment

Since the system log is not used, the database cannot be restored from system log files. However, because it is used by the embedded database, sufficient disk space must still be allocated to store the system log. For the required disk space, see 23.1 Preparation for using an embedded database in the JP1/Automatic Job Management System 3 Configuration Guide.

■ When backing up the database

The scheduler database cannot be backed up to a file while JP1/AJS3 services are active. Take backup at a time when the JP1/AJS3 services can be stopped. For details about how to back up the scheduler database, see B.1(3) Procedure for creating a backup file in the JP1/Automatic Job Management System 3 Administration Guide.

■ When restoring the database

If an error occurs, the scheduler database cannot be restored to its state immediately before the error. Because the system log is not used, the database can only be restored to the point in time at which the backup was taken. For the recovery procedure, see B.1(4) Procedures for restoring the database if an error occurs in the JP1/Automatic Job Management System 3 Administration Guide.