7.10.1 Notes on the standard output file and standard error output file
The following cautionary notes relate to the standard output file and standard error output file.
- Organization of this subsection
(1) When defining the standard output file and standard error output file
Note the following when defining the standard output files and standard error output file output at job execution (of PC and Unix jobs including queueless jobs, flexible jobs and HTTP connection jobs):
-
By default, jobs executed from jobnets acquire the content of the standard error output to serve as their execution result. Their execution results are not acquired from the standard output. You can view the contents of the standard error output in the Execution Result Details dialog box of the JP1/AJS3 - View window. This window also displays the error messages output to the standard error output by JP1/AJS3.
-
To output the contents of the standard output and standard error output to files of your choosing, specify the file names in the Standard output and Standard error fields when defining the job.
-
To also have the contents of the standard output displayed in the Execution Result Details dialog box of the JP1/AJS3 - View window, specify the same file name in the Standard output and Standard error fields when defining the job.
-
If you specify the same file name in the Standard output and Standard error fields, use the same Append setting for both files. If you specify the option to create a new file for one file and the option to add data to an existing file for the other, a parameter error occurs and the job fails to register for execution.
-
If you specify the same name as the script file name or the execution file name as the environment variable file name, the standard input file name, the standard output file name, or the standard error output file name in the Define Details - [unit-name] window, the job might terminate abnormally. Do not use the same file name as the script file or the execution file for any of these files.
-
If you specify the same standard output file name or standard error output file name for jobs that are executed concurrently, the older output results in the standard output or the standard error output will be overwritten by the new output results. Also, when multiple jobs that use the standard output or standard error output run concurrently, the older output results will be overwritten by the new output results.
Specify a different standard output file name or standard error output file name for each of the jobs to be executed concurrently.
-
If you specify an inaccessible network file name as the standard output file name or the standard error output file name, the job might fail to start or end abnormally. Specify a correct network file name.
-
To specify the name of a network file as the standard output file name or standard error output file name, you must have the following permission for the network file:
- In Windows
-
The account for the JP1/AJS3 service (or the JP1/AJS3 Queueless Agent service for queueless jobs) must have Create, Read, and Modify permissions for the file.
- In UNIX
-
The account specified as the OS user who executes the job must have Create, Read, and Modify permissions for the file.
-
Specifying a file name in the Standard output field might make the jobnet run slower than it would if no file name were specified.
- Supplementary note
-
By default, submit jobs executed using the jpqjobsub command do not save data to standard output or standard error output. Specify the desired file name in an option of the jpqjobsub command.
(2) When outputting large quantities of data to the standard output or standard error output file
In JP1/AJS3, when a job ends (meaning a standard job, action job, or custom job, but not queueless jobs), the standard output file and standard error output files are transferred from the agent host to the manager host. If you execute a job that outputs large quantities of data (several megabytes or more) to the standard output or standard error output file, transferring and analyzing the data can place a high demand on system resources. The additional CPU and memory resources such operations require can delay job execution, and even slow down the entire system. This problem affects the manager host and the agent host.
If you specify the option to append data, output data accumulates with each executed job. This markedly increases the size of the transferred file. In this situation as well, transferring and analyzing the data can place a high demand on system resources, which can cause jobs to terminate abnormally or delay the transfer of data from the agent host to the manager host. In this case, we recommend that you disable the append option. Alternatively, you can periodically back up and delete the standard output and standard error output data files.
In JP1/AJS3, you can set an upper limit on the sizes of the standard output file and standard error output file so that excess data is discarded or a warning message is output when the upper limit has been reached. In this way, you can control the file sizes so that the processing of only a few jobs will have little effect on overall system performance. For details about how to set a maximum file size for the standard output or standard error output file, see 6.2.7 Placing restrictions on file reception in the JP1/Automatic Job Management System 3 Configuration Guide (for Windows systems) or 15.2.7 Placing restrictions on file reception in the JP1/Automatic Job Management System 3 Configuration Guide (for UNIX systems).
In addition, you can also specify the settings on the agent host so that excess data is discarded or a warning message is output when the upper limit has been reached. In this way, you can also control the file sizes so that the processing of only a few jobs will have little effect on overall system performance. For details about how to specify the settings in Windows, see 6.2.27 Placing restrictions on file transmission in the JP1/Automatic Job Management System 3 Configuration Guide. For details about how to specify the settings in UNIX, see 15.2.24 Placing restrictions on file transmission in the JP1/Automatic Job Management System 3 Configuration Guide.
If you have not imposed a limit on the sizes of the standard output file and standard error output file, and a job is not operating correctly, take appropriate action as follows:
-
When a large amount of data is output to the standard output
For a PC job or Unix job, the standard output file is transferred only when the file name is explicitly specified in the job definition. If a large amount of data is transferred as the standard output file, the job might end abnormally. If the job ends abnormally, do not specify anything as the standard output file name, and redirect the standard output from a batch file or shell script within the job.
If you do not specify a standard output file name in the job definition, the standard output file is not transferred to the manager host.
-
When a large amount of data is output to the standard error output
Specify a NULL device as the standard error output file name in the job definition. If the job is to be executed in UNIX, specify /dev/null; if the job is to be executed in Windows, specify NUL. If a standard error output file name is explicitly specified in the job definition, use a batch file or shell script to redirect the standard error output.
When you specify these settings, the standard error output file is not transferred to the manager host, and you will be unable to view the contents of the file in the Execution Result Details dialog box of JP1/AJS3 - View.
- Supplementary notes
-
-
If the problems described above occur, a file named A_JPQ*_job-number might remain in the job execution environment work directory of agent host. The agent host job execution environment work directory is a folder specified for the WorkPath environment setting parameter of [{JP1_DEFAULT|logical-host-name}\JP1NBQAGENT\Process]. For details about the WorkPath environment setting parameter, see 20.5.2(37) WorkPath (for agent process) in the JP1/Automatic Job Management System 3 Configuration Guide.
You can safely delete files whose file name contains the job number of a job that ended abnormally.
-
When you use submit jobs that output large amounts of data to the standard output or standard error output, the amount of data in the temporary files created in the work directory of the job execution environment of the manager host (M_JPQSTDE_job-number or M_JPQSTDE_job-number) also increases. Usually, when the setting determining how many days to retain job information is exceeded for a job, temporary files for that job are deleted when job information deletion processing is performed. However, if the disk becomes full due to the temporary files, you can delete them manually after the job terminates. If the disk often becomes full, consider reducing the number of days job information is retained, or redirecting the standard output or standard error output from a script file in the job.
Note that once you delete a temporary file, you can no longer use the jpqjobget command to view the standard output and standard error output files. The manager host job execution environment work directory is a folder specified for the WorkPath environment setting parameter of [{JP1_DEFAULT|logical-host-name}\JP1NBQMANAGER\Process]. For details about the WorkPath environment setting parameter, see 20.5.2(1) WorkPath (for manager process) in the JP1/Automatic Job Management System 3 Configuration Guide.
-
The following tables show examples of specifying redirection as Parameter in the detailed definition of a job.
|
Job definition |
Items to be specified |
Specification |
|---|---|---|
|
The names of the standard output file and standard error output file are different, and the executable file is an exe file. |
Executable file: test.exe Parameters: None Standard output file: out.txt Standard error output file: err.txt |
Executable file: cmd.exe Parameters: /C test.exe >out.txt 2>err.txt Standard output file: None Standard error output file: None |
|
The names of the standard output file and standard error output file are different, and the executable file is an bat file. |
Executable file: test.bat Parameters: None Standard output file: out.txt Standard error output file: err.txt |
Executable file: test.bat Parameters: >out.txt 2>err.txt Standard output file: None Standard error output file: None |
|
The names of the standard output file and standard error output file are the same. |
Executable file: test.bat Parameters: None Standard output file: out.txt Standard error output file: out.txt |
Executable file: test.bat Parameters: >out.txt 2>&1 Standard output file: None Standard error output file: None |
|
Job definition |
Items to be specified |
Specification |
|---|---|---|
|
The names of the standard output file and standard error output file are different. |
Script file: test.sh Parameters: None Standard output file: out.txt Standard error output file: err.txt |
Script file: test.sh Parameters: >out.txt 2>err.txt Standard output file: None Standard error output file: None |
|
The names of the standard output file and standard error output file are the same. |
Script file: test.sh Parameters: None Standard output file: out.txt Standard error output file: out.txt |
Script file: test.sh Parameters: >out.txt 2>&1 Standard output file: None Standard error output file: None |
(3) About restrictions placed on reception and transmission of the standard output file and standard error output file
If you execute a job that outputs a large amount of data (for example, a few megabytes or more) to the standard output file or standard error output file, file data analysis or transfer will result in a heavy load. As a result, the CPU usage or the amount of memory used by JP1/AJS3 increases. This might cause job execution to delay, adversely affecting the overall system performance. To prevent this, in JP1/AJS3 11-00 or later, by default, the manager host places restrictions on file reception, and the agent host places restrictions on file transmission.
File reception and transmission restrictions in JP1/AJS3 work by truncating sent or received data at a certain file size, and outputs a message indicating that the upper limit was reached. The maximum size of a file received on the manager host is 5 MB (5,242,880 bytes), and the maximum size of a file sent from the agent is 3 MB (3,145,728 bytes).
You can change the action taken by the system when the maximum file size is exceeded, and the maximum file size, on the manager host. For details about how to change the settings in Window, see 6.2.7 Placing restrictions on file reception in the JP1/Automatic Job Management System 3 Configuration Guide. For details about how to change the settings in UNIX, see 15.2.7 Placing restrictions on file reception in the JP1/Automatic Job Management System 3 Configuration Guide. You can change the action taken by the system when a maximum file size is exceeded, and the maximum file sizes, on the agent host. For details about how to change the settings in Windows, see 6.2.27 Placing restrictions on file transmission in the JP1/Automatic Job Management System 3 Configuration Guide. For details about how to change the settings in UNIX, see 15.2.24 Placing restrictions on file transmission in the JP1/Automatic Job Management System 3 Configuration Guide.
- Supplementary note
-
If you upgrade-install JP1/AJS3 on the manager host, the new installation will inherit the same restrictions on file reception that were set in the environment setting parameters of the previous installation. If you upgrade-install JP1/AJS3 on the agent host, JP1/AJS3 outputs a message indicating that the maximum file size is reached but does not truncate large data at the maximum file size.
(4) Other notes
-
You can specify only text files as the output destination for standard output and standard error output data.
-
Make sure that machine-dependent characters are not output in the standard output and standard error output data. JP1/AJS3 - View cannot display these characters correctly.
-
Standard output data and standard error output data is saved to temporary files on the agent host, which are transferred to the manager host when the job has terminated. These files are created with the following names under the work directories in the job execution environment on both the manager and agent hosts.
- Manager host (submit jobs only)
-
M_JPQSTDE_job-number
M_JPQSTDO_job-number
- Agent host
-
A_JPQSTDE_*_job-number
A_JPQSTDO_*_job-number
The following describes when the files are deleted from each host.
- Manager host:
-
Temporary files for QUEUE jobs and submit jobs are deleted when job information deletion processing is performed. For other jobs, the temporary files are deleted when the job ends.
- Agent host:
-
The temporary files are deleted automatically when the job ends.
For queueless jobs, only the standard error output data is stored in a temporary file on the agent host. The data in this file is transferred to the manager host when the queueless job terminates. The file is deleted automatically after the queueless job terminates or when the queueless agent service starts.
The standard output data and standard error output data for user programs executed by flexible jobs are always forwarded to the manager host. You can check the names of forwarded files by executing the ajsshow command by specifying %so for the -i option.
-
When you display the Execution Result Details dialog box of JP1/AJS3 - View after executing a job for which a standard error output file name is specified and Append is selected, the displayed information differs depending on the destination service of the job.
-
When Standard is specified as the Exec. Service:
The content of the file specified in Standard error is displayed.
-
When Queueless Agent is specified as the Exec. Service:
The content output to standard error output at job execution is displayed.
-