7.6.2 Notes on the Monitoring Files job

(1) Events monitored by the Monitoring Files job

The following table lists the events that can be monitored.

#1

If a file with the specified name already exists when monitoring starts, the condition is satisfied when the old file is deleted and a new file is created.

You can use the start monitoring option to specify whether the condition is satisfied if a file with the specified name already exists when monitoring starts. For details about the start monitoring option, see (3) Options of the Monitoring Files job.

#2

If the monitoring condition is satisfied, the system performs a close check to make sure the monitored file is not being accessed by any process other than the Monitoring files job.

If another process is accessing the file, the condition is judged to be unsatisfied, and another close check is performed when the next monitoring interval occurs. If the file is not being accessed by a process other than the Monitoring files job, the condition is judged to have been satisfied. The close check prevents the job from assuming that the condition is satisfied before the transmission (for example, copying) of a monitored file has been completed.

#3

If a file with the specified name does not exist when monitoring starts, the condition is satisfied when a new file with the specified name is created and then deleted.

#4

If the file with the specified name does not exist when monitoring starts, the condition is satisfied when a new file with the specified name is created and the size of the file or the last write time is changed. Simply creating a file does not satisfy the condition.

You can specify multiple conditions simultaneously. For example, specify Delete and Final time write if you want to execute the succeeding job when the file is deleted or updated. However, you cannot specify Change size and Final time write simultaneously.

The following figures show the basic operation of the Monitoring Files job.

To Page Top

(2) Specifying file names

To specify a file name, you can use an absolute path or wildcard characters consisting of an absolute path and a wildcard (*). The following table lists examples of using wildcards to specify file names.

Note

The file name examples in the above table are for UNIX. When you specify a file name for Windows, it appears in the format c:\jp1\*. An error occurs at execution of the Monitoring Files job in the following cases:

• A wildcard is specified in a directory name.

• A relative path is specified.

• The specified file name already exists as a directory (for example, when you specify file name /jp1/aaa, but directory /jp1/aaa already exists).

• The parent directory of the file name including the wildcard does not exist (for example, when you specify file name /jp1/*, but directory /jp1/ does not exist).

To Page Top

(3) Options of the Monitoring Files job

You can specify the following two options for the Monitoring Files job:

• Start monitoring option

• Monitoring Files job status passing option

Details of each option are given below.

To Page Top

(4) Notes on defining the Monitoring Files job

Note the following when you define the Monitoring Files job:

• When you execute a Monitoring Files job with a monitoring interval of 1 to 9 seconds, succeeding jobs are not executed immediately (in 1 to 9 seconds) when a file update (event) occurs. When you execute many Monitoring Files jobs, it could take some time before succeeding jobs are executed. In such a case, set a monitoring interval that provides sufficient leeway.

  For details about the formula for estimating the length of a monitoring interval for the Monitoring Files job, see 3.1.6 Monitoring interval set when using the Monitoring Files job in the JP1/Automatic Job Management System 3 System Design (Configuration) Guide.

• A Monitoring Files job might slow down or the CPU usage of the file monitoring process might go up if a generic name, containing a wildcard (*), is specified for that job as the monitoring-target file name, and if many files exist which match that name. When you use a wildcard character to specify the names of monitoring target files, consider ways to minimize the number of files that match the specified monitoring target file names. We also recommend that you delete unnecessary files to reduce the number of files. Alternatively, use full names to specify the names of monitoring target files.

• As the monitoring targets of the Monitoring Files job, do not specify files in an environment where disks are mounted or unmounted when jobs are executed. If you specify such files as monitoring targets, monitoring might not operate normally or might detect the creation or deletion of files at unexpected times.

• In Windows, the Monitoring Files job can monitor files that do not exceed 2 gigabytes (2,147,483,647 bytes). To monitor larger files (large file), you must set up the environment accordingly. For details about how to perform the settings, see 6.3.16 Enabling monitoring of a large file in the JP1/Automatic Job Management System 3 Configuration Guide. Note that in the UNIX version of JP1/AJS3 11-10 or later, large files can be monitored regardless of whether monitoring of large files is enabled.

  If the monitoring of large files is enabled by specifying the relevant settings in Windows, or by upgrading JP1/AJS3 from a version earlier than 11-10 to version 11-10 or later in UNIX, the following notes apply:

  • When a user program such as a Unix job references or uses the status passing information attribute FLSIZE from a Monitoring Files job, a value that exceeds 2 gigabytes might be passed to the user program. This might cause an overflow in the process that handles file sizes in the user program, and the program will need to be amended or other remedies found.

• The name of a file monitored by the Monitoring Files job must conform to the LANG environment variable used for running JP1/AJS3. If you monitor a file whose name uses any other character set, operation is unpredictable.

• If the file being monitored by the Monitoring Files job is also the subject of another event job or log file monitoring by the JP1/Base event service functionality, the target file is considered to be open and the monitoring condition is not satisfied. Therefore, do not set the files as the monitoring target of the Monitoring Files job if they are the target of another event job or subject to log file monitoring by the JP1/Base event service.

• Do not specify as monitoring targets the syslog, device special files, and other system files of the OS. The Monitoring Files job might be unable to identify each of the many unspecified processes accessing the files, processes that access the file frequently, and processes that use the file. Also, do not specify a RAW file as a monitoring target because the Monitoring Files job cannot identify a process that is updating or using such a file.

• If you specify files whose names include network paths^# as the monitoring targets of a file monitoring job, set Y for the NetworkFilewatch environment setting parameter. For details about the NetworkFilewatch environment setting parameter, see 20.6.2(30) NetworkFilewatch in the JP1/Automatic Job Management System 3 Configuration Guide.

  If the names of monitoring-target files include network paths when Y is not set for the NetworkFilewatch environment setting parameter, the file monitoring job might perform unintended operations, depending on the network conditions.

  If the file name you want to specify refers to a file on the local host, specify a file name that does not include a network path.

  If the file name you want to specify refers to a file on another host, monitor the file in either of the following ways:

  • Install JP1/AJS3 on the host on which the file to be monitored exists, and then monitor the file on that host locally.

  • Transfer the monitoring-target file by using FTP or another protocol to a host on which JP1/AJS3 is installed, and then monitor the file.

  #

  Examples of files whose names include network paths are as follows:

  Windows: Files that can be viewed on a network drive and UNC

  UNIX: Files that can be viewed when mounting an NFS

• If Y is not set for the NetworkFilewatch environment setting parameter when file monitoring jobs are run on UNIX hosts, do not monitor files for which operations (create, update, and view) might be performed by other hosts over the network. If a file is accessed from another host over a network, a condition related to the file might be met while the file is being accessed or might be met more than once by performing only one operation.

• The Monitoring Files job cannot correctly monitor a file that is frequently opened and closed to append data. If such a file is closed when the job performs monitoring, the job might assume that file updating has finished and an event might occur. If you want to monitor such a file, do not use the Monitoring Files job directly. Instead, prepare another file that indicates when file updating has finished, and use the job to monitor the file you prepared.

• Note the following when you are using names containing wildcards (*) to specify the names of files to be monitored:

  • Monitoring Files events occurring within a given monitoring interval cannot be used to monitor files based on the order in which they were created, updated, or deleted. The event order option prevents the order of Monitoring Files events from being altered by factors affecting the communication status. It does not change how the Monitoring Files job behaves.

• Note the following regarding satisfying the monitoring condition for the Monitoring Files job:

  • When you specify Delete as a monitoring condition and the monitoring target file does not exist when monitoring starts, the monitoring condition is satisfied after the file is created and then deleted.

  • When Change size or Final time write, is specified as a monitoring condition and the monitoring target file does not exist when monitoring starts, the monitoring condition is satisfied after the file is created, and then the file size is changed or the last write time is changed.

    Change size and Final time write monitoring conditions are satisfied after a write is actually made to the target disk. Simply editing a file by using a text editor does not satisfy the monitoring condition.

• If you select the state Do not inherit for the Monitoring Files job status passing option, the information kept by the Monitoring Files job will be lost if the JP1/AJS3 service stops while the Monitoring Files job is active. When you restart the JP1/AJS3 service and begin file monitoring again, the previous file monitoring information will not be inherited.

• The Monitoring Files job checks files at a specified monitoring interval. When an asterisk (*) is specified, the job checks all the monitored files. If a file update that matches the monitoring condition takes place more than once for one of the monitored files between monitoring times, the Monitoring Files job detects only the last update. Also, if a single file update is detected by a large number of jobs, the next monitoring time might be delayed because events will be issued for all the jobs. If a file update that matches the monitoring condition takes place more than once before the delayed monitoring time arrives, the Monitoring Files job detects only the last update.

  If you specify a short monitoring interval and there are a large number of jobs to monitor, monitoring might take longer than the monitoring interval.

• When the Monitoring Files job detects that the status of a monitoring target file has changed, the job performs a close check to see if any process has the monitoring target file open. If the Monitoring Files job determines that the file is not open, the job issues an event. If the monitoring target file is open, the job does not issue an event, and enters Now monitoring status again. The Monitoring Files job, after the monitoring interval, checks if any process has the monitoring target file open again. In short, as long as the monitoring target file is open, the Monitoring Files job does not issue an event even if it detects that the status of the monitoring target file has changed. Note that if you monitor files on an NFS server in a Windows environment or monitor files via a network in a UNIX environment, it is impossible to check whether a file has been opened by a process of another host. Therefore, an event is issued even when a file being monitored is opened by a process of another host.

• If you select the state Inherit for the Monitoring Files job status passing option, and execute a saved JP1/AJS3 unit or use the environment under the installation directory for a backup copy of JP1/AJS3, you need to have cold-started the JP1/AJS3 service. If you execute a Monitoring Files job of the backup JP1/AJS3 without cold-starting the JP1/AJS3 service, monitoring starts from the monitoring status in effect before the unit was saved.

• If an information file or folder is affected by some error that prevents files that store passing statuses from being created or written to, the message KAVT2034-W is output to the integrated trace log, and job execution continues. In this case, the file for inheriting the monitoring status is not created.

• If you select the state Inherit for the Monitoring Files job status passing option, execute a Monitoring Files job in a start condition, and then kill the JP1/AJS3 service, the status of the Monitoring Files job will be passed. However, if the JP1/AJS3 service is subjected to planned termination rather than killed, the status of the Monitoring Files job will not be passed.

• The following directories might contain a file that could be read or written by the file monitoring process at any time while the process is running. Therefore, do not specify a file in the following directories as a monitoring target:

  When running on the physical host

  For Windows, if the installation folder is the default installation folder or is in a folder protected by the system:

  %ALLUSERSPROFILE%\Hitachi\JP1\JP1_DEFAULT\JP1AJS2\jp1ajs2\sys

  The default location of %ALLUSERSPROFILE% is system-drive\ProgramData.

  A folder protected by the system is the path to a folder in any of the following:

  - system-drive\Windows

  - system-drive\Program Files

  - system-drive\Program Files (x86)

  For Windows, if the installation folder is other than the above:

  JP1/AJS3-installation-folder\jp1ajs2\sys

  For UNIX:

  /var/opt/jp1ajs2/sys

  When running on a logical host

  For Windows:

  shared-folder\jp1ajs2\sys

  For UNIX:

  shared-directory/jp1ajs2/sys

• A Monitoring Files job executed under Windows might detect updates to files other than the expected target files when both of the following conditions are met:

  1. The monitoring target files are specified using a 3-character extension, with a wildcard for the rest of the file name.

     Example:

     Monitoring target files specified as C:\Temp\*.txt.

  2. The monitoring target folder contains files with extensions that are four characters or longer, and the first three characters match the extension as specified in 1 above.

     Example:

     Suppose that the monitoring target files are specified as *.txt (to match the beginning of a string by using a wildcard), and the target folder contains the three files aaa.txt, aaa.txta, and aaa.txtab. When any one of these files is updated, the Monitoring Files job will assume that the monitoring conditions are satisfied. The table below shows examples of how different monitoring target files are selected according to how you specify the file extension.

  Table 7‒5: Detection of monitoring target files by file extension

  Monitoring target file name	Updated files
  	aaa.tx	aaa.txt	aaa.txta	aaa.txtab
  C:\Temp\*.tx	Y	N	N	N
  C:\Temp\*.txt	N	Y	Y	Y
  C:\Temp\*.txta	N	N	Y	N
  C:\Temp\*.txtab	N	N	N	Y

  Legend:

  Y: An event is issued when this file is updated.

  N: No event is issued when this file is updated.

• When you use a file transfer tool to perform an operation on a monitoring target file such as updating the file, the Monitoring Files job might assume that the Delete or Create monitoring condition is satisfied, even if the file was only overwritten. That is, if the tool performs file deletion processing internally, and the deletion timing happens to coincide with the monitoring interval, the Monitoring Files job assumes that the file has been deleted. To avoid such problems, we recommend that you use the Create and Final time write monitoring conditions even if the monitoring target files are always overwritten in your particular setup.

• If you change the Monitoring Files job status passing option from the state Do not inherit to Inherit during monitoring by a Monitoring Files job specified as a start condition, and then restart JP1/AJS3, the messages KAVT2031-E and KAVT2034-W are output to the integrated trace log. This indicates that the status of the job before the restart could not be inherited because the Monitoring Files job status passing option for the job was set to the state Do not inherit before you restarted JP1/AJS3. For this reason, do not change the option from the state Do not inherit to Inherit while a Monitoring Files job is active, as you will be unable to inherit the status from before JP1/AJS3 restarted.

• When you use a Monitoring Files job in Windows and specify *.* as the name of the monitoring target file, the job will also detect files without an extension.

  For example, if you specify C:\temp\*.* as the monitoring target file, an event will be issued when any of the following files under C:\temp is updated:.

  • abc

  • abc.txt

  • abc.txt.txt

• You cannot specify a symbolic link as a monitoring target file. However, provided that the file itself is not a symbolic link, you can include a symbolically linked directory in the name of the target file.

• Each time the monitoring interval arrives, a Monitoring Files job checks whether the status of the monitoring target file has changed. When a change is detected, a close check is performed regardless of the monitoring condition (Create, Change size, or Final time write) specified in the Monitoring Files job. In Windows, the check process attempts to open the monitoring target file in write mode. If the attempt to open the file is successful, the file is immediately closed and a determination is made that no other process has the file open. No other program can access the monitoring target file while it is open for the purposes of a close check. For this reason, when you create a user program that operates a monitored file from within an event job or a jobnet with a start condition, take measures to deal with potential file access problems by incorporating retry processing into the user program.

• In Windows, if the monitored files are read-only, the files cannot be opened in write mode, a close check cannot be performed, and no event will be issued even when a file change that matches the monitoring conditions is detected. If a monitored file is read-only, the message KAVT2036-W is output to the integrated trace log. Remove the read-only attribute from the monitored files so that the files can be opened in write mode.

• The Windows file system uses the following two file name types:

  • Long file names specified arbitrarily by users

  • Corresponding short file names generated automatically by Windows (8.3 filenames)

  The Monitoring Files job in JP1/AJS3 monitors both file name types, and therefore the Monitoring Files job will detect an event for a file whose short file name matches the monitoring target file. In this case, an event might appear to have been detected for the wrong file.

  If the Monitoring Files job appears to have detected an event for the wrong file, run the following command at the command prompt to output the short file names of files in the folder that contains the monitoring target file. Check the command output to see whether the file name specified as the monitoring target file matches the short file name of another file in the folder.

  dir full-path-of-folder-containing-monitoring-target-file /x

  To prevent the Monitoring Files job from identifying a short file name as its monitoring target, specify the monitoring target file name as follows:

  • When using wildcard characters to specify the monitoring target file, make sure that the base file name is at least 8 characters.

    The following table shows examples of defining the monitoring target file name to include and exclude short file names as monitoring targets.

    Table 7‒6: Examples of defining the name of the monitoring target file to include and exclude short file names as monitoring targets

    Monitoring target file group		Wildcard characters specified as monitoring target file name
    Long file name	Short file name
    C:\temp\ABCDEFGH001.log	C:\temp\ABCDEF~1.log	To exclude short file names as monitoring targets: C:\temp\ABCDEFGH*.log The base file name ABCDEFGH is 8 or more characters. To include short file names as monitoring targets: C:\temp\ABCDEF*.log The base file name ABCDEF is fewer than 8 characters.
    C:\temp\ABCDEFGH002.log	C:\temp\ABCDEF~2.log
    C:\temp\ABCDEFGH003.log	C:\temp\ABCDEF~3.log
    C:\temp\ABCDXXXXXXX.log	C:\temp\ABCDEF~4.log

  • When using a full name to specify the monitoring target file, do not specify a file name in short file name format.

    Here, a file name in short file name format refers to a long file name in the 8.3 format used by short file names.

    Supplementary note

    If you specify the name of a monitoring target file in short file-name format, make sure that the files in the same location as the monitoring target file are all named in short file-name format.

• In a Linux environment, if a network connection between a host that is running a file monitoring job and a file system (such as NFS) is closed, the job might sometimes remain running. If this occurs, you might become unable to monitor the states of files correctly.

  One solution to this problem is to set a timeout for the close check of the file monitoring job. If a timeout is set, a message can be output when the file monitoring job remains running after a disconnection. To output such a message, set the CloseCheckTimeout and CloseCheckWarnLogInterval environment setting parameters. For details about the CloseCheckTimeout environment setting parameter, see 20.6.2(32) CloseCheckTimeout in the JP1/Automatic Job Management System 3 Configuration Guide, for details about the CloseCheckWarnLogInterval environment setting parameter, see 20.6.2(33) CloseCheckWarnLogInterval in the JP1/Automatic Job Management System 3 Configuration Guide.

To Page Top