Hitachi

JP1 Version 12 JP1/Automatic Job Management System 3 System Design (Configuration) Guide


7.1.3 Database for QUEUE jobs and submit jobs

JP1/AJS3 uses an ISAM database as the database for QUEUE jobs and submit jobs. This database manages definition information about agents, queues, and execution-locked resources defined by the jpqimport command or by dynamic reconfiguration. It also manages job information and job statuses from registration through to completion. The following table gives details about the tables in the database for QUEUE jobs and submit jobs.

Table 7‒1: Database tables used in the execution environment for QUEUE jobs and submit jobs

Table name

Contents

Row length#

(bytes)

Variable-length rows (bytes)

Number of keys

JPQSYSINFO

System control information

28

N/A

1

JPQAGTINFO

Agent definition information

524

N/A

2

JPQSCHINFO

Scheduling information (number of concurrent jobs)

12

N/A

1

JPQQUEINFO

Queue definition info

280

N/A

2

JPQCONINFO

Queue/agent connection information

12

N/A

1

JPQRESINFO

Execution-locked resources definition information

68

N/A

2

JPQJOBINFO

Job definition information

5,860

N/A

2

JPQJOBCTRL

Job control information

656

N/A

2

JPQENVINFO

Environment variable information

516

more than 4

1

JPQTRANSINFO

Transfer file information

1,036

N/A

1

JPQNFYINFO

Status notification information (such as the client name, destination, port ID)

540

N/A

1

Legend:

N/A: Not applicable

#

The row length given for a variable-length table is the total size of fields other than the variable-length portion.

By default, the database for QUEUE jobs and submit jobs uses synchronous write mode. This mode guarantees the sequence in which data is written to the database, thereby increasing reliability. For details about setting the write mode for updating ISAM files, see 20.5 Setting up the job execution environment in the JP1/Automatic Job Management System 3 Configuration Guide.

Cautionary note

Do not create any direct symbolic links to the ISAM files in the database for QUEUE jobs and submit jobs. Symbolic links are destroyed when ISAM files are moved or recreated during database reorganization, resulting in new ISAM files being created in the database directory.

Because the files in the database directory are managed by the directory in which they are stored, specify the database directory if you need to create a symbolic link.

Consider the following when using QUEUE jobs and submit jobs:

Organization of this subsection

(1) Optional settings for the ISAM database

Set the following options as required:

(a) Synchronous write mode

Data can be written to the ISAM database in either synchronous or asynchronous write mode. Asynchronous write mode enhances performance, but data files and key files might become inconsistent if a failover occurs. To protect data during a failover when using a cluster system, we recommend that you set synchronous write as the ISAM write mode. Synchronous write mode maintains data consistency by following the order in which the data is written by JP1/AJS3, so that inconsistencies do not arise among the database tables. However, even in synchronous write mode, files can still be corrupted by a power outage while the OS is writing to the disk.

If you are using the ISAM database and file consistency is the only issue, you can fix any inconsistencies by reorganizing the database. However, depending on the timing, it might not be possible to recover from a disk error or file error caused by a power outage, whichever write mode is set. Bearing these points in mind, consider backing up the execution environment definition for QUEUE jobs and submit jobs from time to time.

(b) Automatic reorganization of the ISAM database

The ISAM database features an automatic reorganization function. You can choose whether to automatically reorganize the database of the execution environment for QUEUE jobs and submit jobs when the JP1/AJS3 service starts. If you do not enable this function, processing performance will gradually deteriorate over time, due to the growing unavailable areas and fragmentation of variable-length tables in the database. Enable the ISAM automatic reorganization function for this reason. For the procedure, see 7.1.2(2) Setting up the function for automatically reorganizing ISAM files (Windows host) or 7.1.2(3) Setting up the function for automatically reorganizing ISAM files (UNIX host).

If you are using a cluster system, we recommend that you enable automatic reorganization of the ISAM database so that recovery can be performed when JP1/AJS3 starts. In the event of a failover (switching of work task processing from the active server to a standby server), the database will need to be reorganized to restore the data and index pages if inconsistencies arise due to the timing of the failover. However, because files are reorganized one by one, as the number of execution results kept for QUEUE jobs and submit jobs continues to grow, the database can take a long time to reorganize and JP1/AJS3 services might take longer to start. Repeated failovers are typical of a cluster system, so you must adjust the start timeout value set in the cluster software. The time required for ISAM reorganization is proportional to the actual number of records. If adjusting the cluster software's start timeout is not possible, perform settings to prevent ISAM files from becoming excessively large (for example, change the number of days that job information is kept) so as to shorten the reorganization time.

Disable automatic reorganization of the ISAM database if you want JP1/AJS3 services to start quickly at all times and you plan to reorganize the ISAM database manually.

Important

To reorganize the ISAM database, disk space equivalent to about twice the size of the largest ISAM file in the execution environment for QUEUE jobs and submit jobs must be available on the drive containing the work directory for the physical host or logical host (work directory for execution of the manager process, specified in the WorkPath environment settings parameter). In addition, free space of about the same size as the largest of the ISAM files to be reorganized is required on the drive containing the database directory of the execution environment for QUEUE jobs and submit jobs.

(2) Troubleshooting the ISAM database

The ISAM database needs to be recreated if an error occurs. For details about how to perform a status check and recreate the ISAM database, see 2.11 Troubleshooting problems related to invalid ISAM files in the manual JP1/Automatic Job Management System 3 Troubleshooting.

(3) Maintaining the ISAM database

The following describes how to run a JP1/AJS3 system continuously over a long period using the JP1/AJS3 maintenance mode. Maintenance mode allows you to reorganize the ISAM database without interrupting active work tasks.

To perform maintenance in maintenance mode, execute the jajs_maintain command at a time when few work tasks are being executed.

An example of jajs_maintain execution is as follows:

jajs_maintain -m manager

For the command syntax, see jajs_maintain in 4. Commands Used for Special Operation in the manual JP1/Automatic Job Management System 3 Command Reference.

See also (c) Notes on maintenance mode.

The maintenance mode provided by JP1/AJS3 temporarily disables some of the system's functionality.

Because its functionality is partially disabled, the system runs in a degenerated state, but still continues operation. If the functionality disabled by maintenance mode is accessed while the system is degenerated, the function is executed after the system is restored without resulting in an error.

The purpose and types of maintenance mode are described next.

(a) Purpose of maintenance mode

After placing JP1/AJS3 in maintenance mode, you can reorganize the ISAM database.

Suppose you are executing 10,000 jobs per day with the ISAM database, and need to perform maintenance about once a year. With twice as many jobs (20,000 per day), the reorganization frequency is doubled (once every six months). The greater the number of jobs being executed, the more often maintenance is required, with a corresponding increase in administrative time and costs.

In previous versions that did not offer a maintenance mode, you had to stop the system to reorganize the database, and start it again afterwards. Now, using the JP1/AJS3 maintenance mode, you can perform maintenance simply by placing the system temporarily in a degenerated state. Also, the jajs_maintain command can be scheduled for automatic execution by JP1/AJS3, allowing long-term system operation with maintenance performed transparently. For details about the jajs_maintain command, see jajs_maintain in 4. Commands Used for Special Operation in the manual JP1/Automatic Job Management System 3 Command Reference.

You can also perform maintenance by gradually stopping just the manager functionality of JP1/AJS3 - Manager. In this mode, JP1/AJS3 - Manager denies requests to the manager process of the job execution control for QUEUE jobs and submit jobs, including job execution requests from other hosts and requests executed by the jpqjobsub command. Information about active jobs is not lost, however, because changes in job statuses can still be retained by the execution control agent for QUEUE jobs and submit jobs. When performing maintenance in this manner, choose a time when just a few jobs are being executed because the amount of memory used in accumulating unsent job status notifications between the manager and agent increases in proportion to the number of jobs.

Although all processes that access the database stop during maintenance, the execution control for QUEUE jobs and submit jobs starts again as soon as the reorganization is completed.

(b) Estimating how often to perform maintenance

With the database for QUEUE jobs and submit jobs, job execution performance drops by about 20% after approximately four million jobs have been executed with the JP1/AJS3 service running continuously.

If you are executing 10,000 jobs per day, you will need to reorganize the ISAM files about once a year. With twice as many jobs (20,000 per day), the reorganization frequency is doubled (once every six months). Using these figures as a guideline, monitor unused areas in the files, and estimate when maintenance will be required.

Note that periodically reorganizing the database is unnecessary in maintenance mode if you use the function for automatically reorganizing ISAM files when the JP1/AJS3 service starts.

(c) Notes on maintenance mode

Some JP1/AJS3 functions are disabled while the system is in a degenerated state in maintenance mode. When using maintenance mode, bear in mind your operating environment and determine the best time for performing maintenance.

■ Maintenance mode and restrictions on user operations

The following table shows the restrictions on user operations while JP1/AJS3 is in maintenance mode.

Table 7‒2: Maintenance mode and restrictions on user operations

User operation

Whether the operation can be performed

Execution of a QUEUE job from JP1/AJS3 - View#1

No

Execution of commands used at setup

Yes

Execution of commands for customizing the environment

No#1

Execution of commands for controlling JP1/AJS3

No#2

Execution of commands for operating on units

Yes

Execution of commands for executing jobs

Yes

The command that operate embedded database

Yes

The command used for the time of an error

Yes

The command used by special operation

No#3

Legend:

Yes: Can be performed.

No: Cannot be performed.

Note

The "User operation" in this table corresponds to classification of the command indicated on 1.5 Commands in the manual JP1/Automatic Job Management System 3 Command Reference.

#1

The command besides the jpqdbcond command can be used. When execute a QUEUE job during degeneration, a job will be start failure. Message KAVU0218-E is shown to the Execution result details dialogue box, which is JP1/AJS3 - View at this time.

#2

It can be used besides jajs_stop command, jajs_stop.cluster command, jajs_spmd_stop command and jajs_killall.cluster command.

#3

The commands can be used besides the command used by the QUEUE job and the submit job execution environment. When execute a QUEUE job during degeneration, a job outputs message KAVU4500-W and message KAVU0218-E in integrated trace log, and ends abnormally.

Do not attempt the operations marked No in the above table.

For details about the commands for performing user operations, see 1.5 Commands in the manual JP1/Automatic Job Management System 3 Command Reference.

■ Maintenance mode and restrictions on linkage with another host or system

The following table shows the restrictions on operations that link with JP1/AJS3 on another host or with another system while JP1/AJS3 is in maintenance mode.

Table 7‒3: Maintenance mode and restrictions on linkage with another host or system

Operation that links with another host or system

Whether the operation can be performed

Execution of a QUEUE job by JP1/AJS3 on another host#

No

Job execution by another system (JP1/NQSEXEC or JP1/OJE, for example)

No

Execution of a remote jobnet transferred to another host#

No

Execution of a remote jobnet transferred from another host

No

Legend:

No: Cannot be performed.

#

Do not place JP1/AJS3 in maintenance mode on the local host from the time the remote jobnet starts until it ends.

Because the remote host or system cannot recognize that the linked host is in maintenance mode, the job enters the Ended abnormally status. Make sure you choose an appropriate time to perform maintenance.

(4) Notes on the database (ISAM files) for QUEUE jobs and submit jobs