7.1.1 Queue/agent configuration

To execute QUEUE jobs and submit jobs in a manager/agent configuration, you must register the agent hosts that execute jobs with the manager host.

Organization of this subsection

(1) Queue/agent configuration
(2) About load distribution
(3) Running multiple execution hosts on the same computer
(4) Notes on the execution environment required for QUEUE jobs and submit jobs

(1) Queue/agent configuration

When you register an agent host with the manager host, an internal queue (default queue) is created. The default queue has the same name as the agent host.

To execute a QUEUE job or submit job, the user specifies the queue name or agent host name, and the job is registered in that queue. Jobs are queued in order of registration and are sequentially transferred to the connected agent host.

The following figure shows an example of a queue/agent configuration.

Figure 7‒1: Example of a queue/agent configuration

To register an agent host with the manager host, use the jpqimport command or jpqagtadd command. For the command syntax, see 4. Commands Used for Special Operation in the manual JP1/Automatic Job Management System 3 Command Reference.

Multiple agents can be connected to a queue. By connecting multiple agents to a queue, and setting the priority of each agent and the maximum number of jobs it can execute concurrently, you can distribute the job-processing load.

The following figure shows an example of load distribution with multiple agents connected to a queue.

Figure 7‒2: Example of queue/agent load distribution

To change the maximum number of jobs that an agent host can execute concurrently, use the jpqagtalt command. To connect multiple agents to a queue and set their priorities, use the jpqagtlink command. For the syntax of each command, see 4. Commands Used for Special Operation in the manual JP1/Automatic Job Management System 3 Command Reference.

For a description of load distribution based on setting the maximum concurrent jobs and agent priorities, see (2) About load distribution.

To Page Top

(2) About load distribution

The following explains the concept of load distribution in JP1/AJS3.

We will assume the following for the examples below:

There are three agent hosts A, B, and C, each of which can execute up to three jobs at the same time (maximum number of concurrently executable jobs = 3).
All three agents are connected to queue1, allowing the processing load to be distributed.
Agent host A has the highest priority, while agent hosts B and C have the same priority.

Six submit jobs are queued for execution at the same time.

Figure 7‒3: Illustration of load distribution

The order of execution of the submit jobs (job1 to job6) is determined as follows:

Determine the execution agent for job1.

Ratio: number of active jobs / maximum number of concurrently executable jobs:
- Agent host A: 0 / 3
- Agent host B: 0 / 3
- Agent host C: 0 / 3
The processing load is evenly distributed among all the agent hosts. Therefore, job1 is executed by agent host A, based on the order in which the agents are defined for queue1.
Determine the execution agent for job2.

The ratio (active jobs / maximum concurrent jobs) is now:
- Agent host A: 1 / 3
- Agent host B: 0 / 3
- Agent host C: 0 / 3
Although agent hosts B and C have a lighter load, job2 is assigned to agent host A, based on the priorities of the agents defined for queue1.
Determine the execution agent for job3.

The ratio (active jobs / maximum concurrent jobs) is now:
- Agent host A: 2 / 3
- Agent host B: 0 / 3
- Agent host C: 0 / 3
Although agent hosts B and C have a lighter load, host A executes job3, based on the priorities of the agents defined for queue1.
Determine the execution agent for job4.

The ratio (active jobs / maximum concurrent jobs) is now:
- Agent host A: 3 / 3
- Agent host B: 0 / 3
- Agent host C: 0 / 3
Agent host A has reached its maximum number of concurrently executable jobs. Therefore, agent host B executes job4, based on the order in which the agents are defined for queue1.
Determine the execution agent for job5.

The ratio (active jobs / maximum concurrent jobs) is now:
- Agent host A: 3 / 3
- Agent host B: 1 / 3
- Agent host C: 0 / 3
Agent host C has the lightest load. Therefore, agent host C executes job5.
Determine the execution agent for job6.

The ratio (active jobs / maximum concurrent jobs) is now:
- Agent host A: 3 / 3
- Agent host B: 1 / 3
- Agent host C: 1 / 3
Agent hosts B and C have a lighter load than A, and their load distribution is even. Therefore, agent host B executes job6, based on the order in which the agents are defined for queue1.

To Page Top

(3) Running multiple execution hosts on the same computer

In JP1/AJS3, you can set the maximum number of jobs that can be executed concurrently by each of the managed agents. By defining multiple agent host names as aliases on the same computer (one IP address), and registering different kinds of jobs with each agent alias, you can distribute work tasks within one computer. For example, hostB and hostC can be defined as aliases of a real host (hostA), allowing work tasks to be distributed on one computer as in the following figure. Note that queueless jobs cannot be executed using an alias for the agent host name.

Figure 7‒4: Example of distributing work tasks

To realize this kind of configuration, add the agent host name aliases to the configuration definition of the job execution control. Also specify the agent host name aliases in the hosts file so that the IP address can be resolved from each alias. For details about adding agent host names, see (b) Configuring the job execution control for QUEUE jobs and submit jobs.

When you define agent aliases in the jp1hosts information or jp1hosts2 information of JP1/Base, JP1 can have its own hosts information without changing the hosts file or other OS settings.

(a) Defining jp1hosts information or jp1hosts2 information

For details about how to define jp1hosts information or jp1hosts2 information, see the relevant section in the JP1/Base User's Guide.

Important: When you have registered jp1hosts information or jp1hosts2 information, JP1/AJS3 no longer searches the hosts file and the DNS server for the host names and IP addresses you entered in the jp1hosts information or jp1hosts2 information. Therefore, you must define both the real host name and aliases for an IP address in the jp1hosts information or jp1hosts2 information. Place the real host name before the aliases.

(b) Configuring the job execution control for QUEUE jobs and submit jobs

To add agent aliases, follow the steps below.

For the command syntax, see 4. Commands Used for Special Operation in the manual JP1/Automatic Job Management System 3 Command Reference.

Add agent definitions dynamically.

To add an agent definition, execute the jpqagtadd command with the -ah option specified. In the -ah option, specify an agent host name defined as an alias in the jp1hosts information or jp1hosts2 information. In the -cp option, specify the maximum number of jobs that can be executed concurrently by the agent, bearing in mind any jobs that will place a lock on the resource.

For example, suppose hostA executes routine jobs, and you want to execute one job exclusively on hostB, which is defined as an agent alias. Set the maximum number of concurrently executable jobs as follows:
```
jpqagtadd [-mh manager-host-name] -ah hostA -cp 0:00-0:00=5
jpqagtadd [-mh manager-host-name] -ah hostB -cp 0:00-0:00=1
```
With these settings, hostA can execute a maximum of five jobs concurrently, while hostB can execute only one registered job at any one time.
Open the queue.

Because the entrance to the queue is closed when you add an agent definition, you must open the queue using the jpqqueopen command.

For example, to open the entrance to hostA's queue, execute the command as follows:
```
jpqqueopen [-mh manager-host-name] -ah hostA -en
```

To Page Top

(4) Notes on the execution environment required for QUEUE jobs and submit jobs

Note the following when planning the execution environment for QUEUE jobs and submit jobs.

(a) Maximum number of concurrently executable jobs (QUEUE jobs and submit jobs)

In JP1/AJS3, you can set the maximum number of jobs that an agent host can execute concurrently. When this maximum is reached, subsequent jobs are placed in the wait state. By setting a maximum number of concurrently executable jobs, you can restrict the number of jobs being executed and distribute the load during particular time periods, or among multiple agents when more than one agent host is connected to a queue.

The maximum number of concurrent jobs must be specified per execution agent. You cannot set a maximum per queue.

■ Notes on setting a maximum number of concurrently executable jobs

The default setting (maximum of five concurrent jobs, 24 hours a day) will suffice if only short jobs are executed, and no more than a few jobs will be registered for execution simultaneously. However, if you execute jobs with long execution times or register many jobs at the same time, the number of concurrent jobs will quickly reach the maximum, and remain there for extended periods of time. This might affect the execution of subsequent jobs.

When the maximum number of concurrently executable jobs is reached on an agent host, subsequent jobs wait in the queuing status until the active jobs finish. In such cases, jobs will take longer than expected to execute. Take the job execution time and the number of jobs executed per unit time into consideration when you set the maximum number of concurrently executable jobs.

■ Checking whether the maximum number of concurrent jobs has been reached when jobs remain in queuing status

When the number of active jobs on an agent host reaches the maximum number of concurrently executable jobs, subsequent jobs might remain in the queuing status for a long time before being executed. To check whether the maximum number of concurrently executable jobs has been reached, execute the jpqagtshow command and compare the number of active jobs (EXECUTING) against the maximum number of concurrently executable jobs (CUREXECHGNUM). For details about the jpqagtshow command, see jpqagtshow in 4. Commands Used for Special Operation in the manual JP1/Automatic Job Management System 3 Command Reference.

By setting up JP1/AJS3 to output the following message to the integrated trace log when the maximum number of concurrent jobs is reached and no more jobs can be executed, you can check whether this is the reason for execution delays:

KAVU4310-I  The number of executing jobs reaches the
concurrently-executable-job-limit (maximum-number-of-concurrently-executable-jobs) at the agent (agent-host-name) (host name: host-name, job number: job-number)

For details about how to specify this setting, see 6.2.13 Outputting a message that reports that the maximum number of concurrently executable jobs has been reached in the JP1/Automatic Job Management System 3 Configuration Guide (in Windows) or 15.2.13 Outputting a message that reports that the maximum number of concurrently executable jobs has been reached in the JP1/Automatic Job Management System 3 Configuration Guide (in UNIX).

(b) Notes on the execution order of submit jobs

Submit jobs registered by the jpqjobsub command are executed in the order in which they are submitted. However, if any of the following situations applies when a job is sent to the execution agent host, it might not be executed in the proper order:

The agent usage rate at the execution agent host has reached the limit.
The execution agent host has failed or stopped.
A job that accesses the same execution-locked resource as the job being transferred has just ended.

By setting relationship lines for a job or jobnet, you can prevent the job execution order from being changed. Consider using relationship lines if the jobs executed in your system can be grouped in separate jobnets.

If jobnet operation is not feasible, enable the setting that prevents any change to the submit job execution order. This will ensure that, until the above situation is resolved, the job registered first will always be taken from the queue first, thereby preventing any change to the execution order.

For details about this setting, see 7.1.11 Setting for preventing changes in the submit job execution order.

Agent usage rate here refers to the ratio of active jobs to the maximum number of concurrent jobs at an agent host. For details about the agent usage rate, see 10.4.1(6) Distributing jobs among execution hosts (agent hosts) in the manual JP1/Automatic Job Management System 3 Overview.

Supplementary note: Jobs submitted with an execution start time specified in the -st option of the jpqjobsub command might sometimes fail to start by the specified time, depending on factors such as the availability of the agent host to which the job is to be sent. Such jobs remain in the queuing status, along with other jobs waiting for execution, until the agent host is restored. Once the agent host becomes available again, the queued jobs are transferred, but in the order they were submitted, not in order of their execution start times.

(c) Action when too many jobs are enumerated by the jpqendjobshow or jpqjobshow command

When execution of the jpqendjobshow command (enumeration of completed jobs) or jpqjobshow command (enumeration of queued jobs) produces a long list of jobs in the standard output file, the command and other requests might fail or take a long time to execute.

Consider the following to prevent such problems.

When using the jpqendjobshow command:

To reduce the amount of data, specify how many items of job information to acquire in the -rn option, or specify the time range of the acquired job information in the -sa and -sp options.

When using the jpqjobshow command:

To reduce the amount of data, specify the owner user name of the jobs to be acquired in the -u option, or specify the status of the jobs to be acquired in the -s option. If the problem still persists, change the communication timeout value. The default is 10 minutes.

For details about how to change the communication timeout value, see 6.2.14 Changing the timeout value for the job execution control manager process in the JP1/Automatic Job Management System 3 Configuration Guide (for Windows systems) or 15.2.14 Changing the timeout value for the job execution control manager process in the JP1/Automatic Job Management System 3 Configuration Guide (for UNIX systems).

When enumerating job information, the job execution control manager (jpqman) consumes memory in proportion to the number of listed items. With a large number of items in the list, the jpqjobshow command or jpqendjobshow command sometimes fails and message KAVU4280-E is output to the integrated trace log. If this happens, you might need to perform a setting to reduce memory usage by the job execution control manager when enumerating job information.

For details about this setting, see 7.1.12 Setting to reduce memory usage by the job execution control manager (jpqman) when enumerating job information.

To Page Top