Job Management Partner 1/Automatic Job Management System 3 Troubleshooting

[Contents][Glossary][Index][Back][Next]

1.4.1 Collecting data for troubleshooting in Windows

Organization of this subsection
(1) Execute the data collection tool
(2) Obtain a dump file
(3) Obtain a dump file
(4) Check the status of processes
(5) Check the operation data
(6) Collect error information displayed on the screen
(7) Collect information about the embedded database

(1) Execute the data collection tool

The following describes the procedure for executing the data collection tool and the data that the tool collects.

(a) Procedure for executing the data collection tool

Execute the data collection tool as shown below. For details about how to set up the data collection tool, see 7. Collecting Log Data in the Job Management Partner 1/Automatic Job Management System 3 Configuration Guide 1.

The following shows an example of executing the data collection tool:

 
c:\>c:\usertools\_04.bat

By default, the results of executing the data collection tool are output to the following folders under %TEMP%\jp1ajs2\backlog. Back up these folders.

For physical hosts:
  1. JP1_DEFAULT\JP1_DEFAULT_1st
    The data for the first reports is output.
  2. JP1_DEFAULT\JP1_DEFAULT_2nd
    All other data is output.

For logical hosts:
  1. logical-host-name\logical-host-name_1st
    The data for the first reports is output.
  2. logical-host-name\logical-host-name_2nd
    All other data is output.

When you use the data collection tool in a cluster system, you can collect data by specifying a logical host name. The data collection tool also provides options for limiting the amount of data that is collected. This following describes the syntax of the data collection tool:

Format
 
_04.bat
     [-h logical-host-name]
     [-s]
     [-t]
     [-u]

Description
The data collection tool obtains maintenance information, such as the JP1/AJS3 logs and information about the OS.

Execution permission
Administrators

Arguments

-h logical-host-name
Specify the name of the target logical host.
The data collection tool collects the data on the physical host in addition to the data on the specified logical host.
If you do not specify a name, the data collection tool obtains the physical host logs.

-s
Specify this option if you do not want to collect information about the database used by JP1/AJS3 or information about the database used by JP1/AJS3 Console.
If you do not specify this option, the data collection tool collects information about these databases.

-t
Specify this option if you do not want to obtain the hosts and services files.

-u
Specify this option if you do not want to obtain user dumps.
You cannot obtain user dumps for Windows 7, Windows Server 2008, and Windows Vista.

Cautionary notes
  1. If you execute the data collection tool without the -s option specified during startup of the JP1/AJS3 service, the job might end abnormally. In such a case, JP1/AJS3 does not collect information about the ISAM database.
  2. To collect data about a logical host in a cluster system, execute the data collection tool on the host on which the logical host runs.
  3. The data collection tool does not compress the data if collects. To compress the data, use a data compression tool.
  4. If you have already created a file containing the execution results of batch files, JP1/AJS3 outputs a message asking for permission to overwrite the file. To overwrite the file, specify y. If you do not want to overwrite the file, specify n.
  5. While a batch file is being executed, JP1/AJS3 might display an error message reporting that a sharing violation has occurred for a file in the JP1/AJS3 - Manager-or-JP1/AJS3 - Agent-installation-folder\sys folder. This is not a problem.
  6. If you perform an operation on the collected data such as, moving it or deleting it immediately after the execution of the data collection tool has finished, a message reporting that the process cannot access a file because another process is currently using the file might appear. This message appears because a process that collects OS information internally is still being executed despite termination of the tool. If this message appears, wait a while and then retry the operation.
  7. If you execute the data collection tool while using Outlook, the following message might appear:
    The program is attempting to access email addresses in Outlook. Is this all right?
    The program indicated in this message is the machine configuration collection program (msinfo32) executed internally by the data collection tool. This program does not affect the operation of the data collection tool or Outlook.
    The data collection tool does not collect email addresses. If this message appears, click the No button.
  8. When you execute the data collection tool in Windows 7, Windows Server 2008, and Windows Vista, you must open the Command Prompt window as an administrator. To open the Command Prompt window in Windows, click Start, All Programs, and Accessories. Right-click Command Prompt and then click Run as administrator.
    If you execute the data collection tool without the Administrators permission, UAC in Windows 7, Windows Server 2008, and Windows Vista repeatedly displays a confirmation dialog box while you are running the data collection tool.
    If you turn UAC off, you will not need to open the Command Prompt window as an administrator.

Return values
0 Normal end
Value other than 0 Abnormal end

Output destination
%TEMP%\jp1ajs2\backlog
You can change the output destination during the setup of the data collection tool.

Messages
Message Description Action
Finished. Press any key. The process ended normally. Press any key. Wait for a user response.
Failed. Press any key. The process did not end normally. Press any key.
See the output OS messages, remove the cause of the error, and then re-execute the process.		 Wait for a user response.
File file-name is exist. Delete to continue? (y/n) A file created the last time this process was executed exists. Press y to delete the file and continue processing, or press n to stop processing. Wait for a user response.
Directory directory-name is exist. Delete to continue? (y/n) A directory created the last time this process was executed exists. Press y to delete the directory and continue processing, or press n to stop processing. Wait for a user response.
Error exist in option. An option is specified incorrectly. Specify the option correctly, and then re-execute the process. Complete the process.
The following logical hosts exist, other than the specified logical host. : logical-host-name A logical host environment other than the one specified for the -h option exists. If necessary, re-execute the process. After this message, the following message is output:
Finished. Press any key.

Example 1
The following command collects physical host materials:
 
_04.bat

Example 2
The following command collects physical host materials when the JP1/AJS3 service is running:
 
_04.bat -s

Example 3
The following command collects data about a logical host (cluster):
 
_04.bat -h cluster

(b) Data that can be collected by using the data collection tool

You can use the data collection tool (_04.bat) to collect the following types of data.

For physical hosts:
  • The data for first reports (%TEMP%\jp1ajs2\backlog\JP1_DEFAULT\JP1_DEFAULT_1st)

Name of folder or file containing collected data Description
\FILELIST.txt List of files under the JP1/AJS3 installation folder
\GETERROR.log Log information that is output by the data collection tool (_04.bat)
\HOSTS hosts file
\services services file
  1. \JP1AJS2.DAT
  2. \JP1AJS2C.DAT
  3. \JP1AJS2CONSOLE.DAT
  4. \JP1AJS2DA.DAT
  5. \JP1AJS2SE.DAT
  6. \JP1AJS2V.DAT
  7. \JP1AJS2WOA.DAT
  8. \JP1BASE.DAT
 Registry information about each product
\REGDATA.DAT JP1 registry information
\WIN.DAT Windows registry information
\jpomanevshow.txt Execution result of jpomanevshow.exe
\ajsqlstatus.txt Execution result of ajsqlstatus.exe
\ajsagtshow.txt Execution result of ajsagtshow.exe
\jajs_status.txt Execution result of jajs_status.exe
\OSINFO.txt OS version, Windows environment variables, IP configuration, network configuration, statistics
\PERMISSIONINFO.TXT Access permissions for JP1/AJS3 folders
\WINMSD.TXT Information, such as a system overview, collected by using the msinfo32 command and the tasks and services being executed.
For Windows 7, Windows Server 2008, and Windows Vista, all system information that is collected by using the msinfo32 command
  1. \ALLUSERSPROFILE\conf\JP1AJS2 (for Windows Server 2008)
  2. \conf\JP1AJS2 (for Windows Server 2003)
  3. \conf\jp1ajs2cm
  4. \conf\JP1AJS2DA
  5. \conf\JP1AJS2V (for Windows Server 2003 and Windows XP Professional)
  6. \ALLUSERSPROFILE\conf\JP1AJS2V (for Windows 7, Windows Server 2008, and Windows Vista)
  7. \conf\JP1AJS2WOA
 Folders containing the environment settings files
\drwtsn\Windows2003\drwtsn32.log (for Windows Server 2003) Dr. Watson logs#1
\EMBDB\_JF*#2\conf Embedded database definition file
\EMBDB\_JF*#2\spool Embedded database failure investigation file
\EMBDB\_JF*#2\etc Other information related to the embedded database that is needed for investigation
\HNTRLib2\spool Integrated trace log
  1. \ALLUSERSPROFILE\jobinf (for Windows Server 2008)
  2. \jobinf (for Windows Server 2003)
 Folders containing information about jobs
  1. \ALLUSERSPROFILE\JP1AJS2 (for Windows Server 2008)
  2. \JP1AJS2 (for Windows Server 2003)
  3. \JP1AJS2C
  4. \jp1ajs2cm
  5. \JP1AJS2DA (for Windows Server 2003 and Windows XP Professional)
  6. \ALLUSERSPROFILE\JP1AJS2DA (for Windows 7, Windows Server 2008, and Windows Vista)
  7. \JP1AJS2HC
  8. \JP1AJS2V (for Windows Server 2003 and Windows XP Professional)
  9. \ALLUSERSPROFILE\JP1AJS2V (for Windows 7, Windows Server 2008, and Windows Vista)
  10. \JP1BASE
 Folders containing log files
  1. \PATCHLOG\JP1JAJS2\PATCHLOG.txt
  2. \PATCHLOG\JP1JAJS2C\PATCHLOG.txt
  3. \PATCHLOG\JP1JAJS2console\PATCHLOG.txt
  4. \PATCHLOG\JP1JAJS2DA\PATCHLOG.txt
  5. \PATCHLOG\JP1JAJS2V\PATCHLOG.txt
  6. \PATCHLOG\JP1JAJS2WOA\PATCHLOG.txt
 Patch log files
\resource\JP1AJS2V Folder containing resource files
  1. \ALLUSERSPROFILE\sys\JP1AJS2 (for Windows Server 2008)
  2. \sys\JP1AJS2 (for Windows Server 2003)
  3. \sys\JP1AJS2WOA
  4. \sys\JP1AJS2V (for Windows Server 2003 and Windows XP Professional)
  5. \ALLUSERSPROFILE\sys\JP1AJS2V (for Windows 7, Windows Server 2008, and Windows Vista)
 Folders containing the system files
  1. \tmp\JP1AJS2DA (for Windows Server 2003 and Windows XP Professional)
  2. \ALLUSERSPROFILE\tmp\JP1AJS2DA (for Windows 7, Windows Server 2008, and Windows Vista)
 Folders containing temporary files for execution
  1. \ALLUSERSPROFILE\tmp\schedule\pd*.trc (for Windows Server 2008)
  2. \tmp\schedule\pd*.trc (for Windows Server 2003)
 Embedded database trace logs
\user_acl Folder containing user mapping and authentication settings files

#1
This data cannot be collected for Windows 7, Windows Server 2008, and Windows Vista.

#2
_JF* indicates an embedded database identifier (_JF0, _JF1, _JF2, and so on). A folder is created for each identifier.
  • The data for second reports (%TEMP%\jp1ajs2\backlog\JP1_DEFAULT\JP1_DEFAULT_2nd)

Name of folder or file containing collected data Description
\ADD Folder for additional collected data
  1. \cmdatabase
  2. \database
  3. \embdatabase\_JF*#1
 Database storage folders
\eventlog\AppEvent(Backup).env Application event log
\eventlog\SysEvent(Backup).env System event log
  1. \SHMDUMP\coreinfo-ISAM.shmdump
  2. \SHMDUMP\coreinfo-Scheduler.shmdump
  3. \SHMDUMP\ProgMon.shmdump
  4. \SHMDUMP\coreinfo-host.shmdump
 Shared memory information
  1. \USERDUMP\*.dmp
  2. \WTSNDUMP\*.dmp
 User dumps#2

#1
_JF* indicates an embedded database identifier (_JF0, _JF1, _JF2, and so on). A folder is created for each identifier.

#2
The data collection tool cannot collect this data for Windows 7, Windows Server 2008, and Windows Vista.

For logical hosts:
  • The data for first reports (%TEMP%\jp1ajs2\backlog\logical-host-name\logical-host-name_1st)

Name of folder or file containing collected data Description
\jpomanevshow.txt Execution result of jpomanevshow.exe
\ajsqlstatus.txt Execution result of ajsqlstatus.exe
\ajsagtshow.txt Execution result of ajsagtshow.exe
\jajs_status.txt Execution result of jajs_status.exe
\jobinf Folder containing information about jobs
\JP1AJS2_LOGICAL Folder containing log files
\JP1BASE Folder containing JP1/Base log files
\EMBDB\_JF*#\conf Embedded database definition file
\EMBDB\_JF*#\spool Embedded database failure investigation file
\EMBDB\_JF*#\etc Other information related to the embedded database that is needed for investigation
\sys\JP1AJS2 System folder
\tmp\schedule\pd*.trc Embedded database trace log
\user_acl Folder containing user mapping and authentication settings files

#
_JF* indicates an embedded database identifier (_JF0, _JF1, _JF2, and so on). A folder is created for each identifier.
  • The data for second reports (%TEMP%\jp1ajs2\backlog\logical-host-name\logical-host-name_2nd)

Name of folder or file containing collected data Description
  1. \cmdatabase
  2. \database
  3. \embdatabase\_JF*#
 Database storage folders
\SHMDUMP\ProgMon.shmdump Information about the shared memory for monitoring processing status
\SHMDUMP\coreinfo-host.shmdump Information about the shared memory for the system management function

#
_JF* indicates an embedded database identifier (_JF0, _JF1, _JF2, and so on). A folder is created for each identifier.

(2) Obtain a dump file

If a STOP error occurs in Windows, the contents of memory are dumped to a file. Obtain this dump file.

If a problem occurs, a crash dump is output. You can use the data collection tool to obtain the crash dump file.

To have a memory dump file and a crash dump file created when a problem occurs, you must specify settings for each file. For details about the settings for collecting log information, see 7. Collecting Log Data in the Job Management Partner 1/Automatic Job Management System 3 Configuration Guide 1. If the appropriate settings are specified, dump files are output to the specified paths.

(3) Obtain a dump file

If a problem occurs, you must obtain a memory dump file and a crash dump file. The procedure for doing so varies according to the OS.

n In Windows 7, Windows Server 2008, and Windows Vista

If a STOP error occurs in Windows, a memory dump is output. Obtain this dump file. To ensure that a memory dump is output when a problem occurs, you must specify settings. For details about the settings, see 7.1(3) Procedure for setting the outputting of a dump file in the Job Management Partner 1/Automatic Job Management System 3 Configuration Guide 1. If the appropriate settings are specified, a memory dump is output to the specified path.

When a problem occurs, a crash dump is also output. However, because Windows 7, Windows Server 2008, and Windows Vista do not provide Dr. Watson log files, you cannot use the data collection tool to collect the crash dump. To obtain information equivalent to the crash dump, collect user dumps and problem reports by performing the procedure described below.

Collecting a user dump
When a JP1/AJS3 process stops due to an application error, perform the operations below while the error dialog box is displayed.

Cautionary note
If you close the error dialog box, dumps will not be collected correctly and user dumps cannot be collected. If you accidentally close the error dialog box before you collect a dump (for example, by clicking the OK button), reproduce the problem to display the dialog box, and then perform the following procedure.
To collect a user dump:
  1. Start Task Manager.
    Use either of the following steps to start Task Manager:
    - Right-click open space on the taskbar, and then click Task Manager.
    - Hold down the CTRL and SHIFT keys, and then press the ESC key.
  2. In the Task Manager dialog box, click the Processes page.
  3. Right-click the name of the JP1/AJS3 process that stopped because of an application error and click Create Dump File.
  4. When a dialog box displaying the output destination path for the user dump appears, collect the user dump from the displayed location.

Obtaining a problem report
You can detect and find solutions for problems on a target host. If a JP1/AJS3 process stops because of an application error, perform the following procedure and obtain a problem report:
To obtain a problem report:
  1. In Windows, open the Run dialog box, enter wercon in the text box, and click the OK button.
    The Problem Reports and Solutions dialog box appears.
  2. In the left pane, click View problem history.
    A list of problems appears.
  3. Double-click the applicable problem.
    The contents of the problem report appear.
  4. Click Copy to clipboard.
  5. Copy the text into a text editor, and save the file.
    Use the problem report text file as data for investigating the failure.

n In Windows Server 2003 and Windows XP Professional

If a STOP error occurs in Windows, a memory dump is output. Obtain this dump file.

When a problem occurs, a crash dump is also output. You can use the data collection tool to obtain the dump file.

To ensure that a memory dump and a crash dump are output when a problem occurs, you must specify certain settings. For details about the settings for collecting dump files, see 7.1(3) Procedure for setting the outputting of a dump file in the Job Management Partner 1/Automatic Job Management System 3 Configuration Guide 1. If the appropriate settings are specified, dump files are output to the specified paths.

(4) Check the status of processes

In Windows, start Task Manager and open the Processes page. Check the operating status of processes.

For details about JP1/AJS3 processes, see B.2 Processes (for Windows).

(5) Check the operation data

For the problem that has occurred, check the operation data and record it. You need to check the following information:

  1. Details about the operation
  2. Time that the problem occurred
  3. Machine configuration (the version of each OS, host name, JP1/AJS3 - Manager and JP1/AJS3 - Agent configuration, and JP1/AJS3 Console Manager and JP1/AJS3 Console Agent configuration)
    For Windows Server 2003, you can use System Information to check the machine configuration.
  4. Whether the problem is reproducible
  5. Name of the user, if any, who logged in from JP1/AJS3 - View or JP1/AJS3 Console View

(6) Collect error information displayed on the screen

Make hard copies of the following:

  1. Windows on the screen when an application error has occurred
  2. Error message dialog box
    Displayed information when a details button, if any, is clicked
  3. Information in the Command Prompt window if the problem occurred during the execution of a command
    Before making a hard copy of the Command Prompt window, make the following preparations by using Console in Control Panel:

    Options page
    Select QuickEdit Mode.

    Layout page
    Enter 500 in Height of Screen Buffer Size.

(7) Collect information about the embedded database

You need the following information if an error occurs while you are using the embedded database:

The following describes how to collect the above information.

(a) Information needed to investigate the cause

To investigate the cause of an error, you mainly need to collect information about the OS and information about the embedded database. Use OS commands to collect information about the OS. Use embedded database commands to collect information about the embedded database.

If you use the support service to solve problems, you must submit the information listed in the table below. This table describes the type of information needed to investigate the cause of problems and how to collect information for each type of problem. Problems are classified into seven levels, of which level 1 has the highest priority.

Problems are grouped by type as follows:

  1. Performance
    The following processing or operation takes too much time:
    • Startup of the embedded database system (including normal startup, restart, and startup after action has been taken for a failure)
    • Stopping of the embedded database system (including normal stopping and forced stopping)
    • Execution of an operation command for the embedded database

  2. No response
    A response is not returned when the following processing or operation is performed:
    • Startup of the embedded database system (including normal startup, restart, and startup after action has been taken for a failure)
    • Stopping of the embedded database system (including normal stopping and forced stopping)
    • Execution of an operation command for the embedded database

  3. Abnormal end
    One of the following has occurred:
    • Abnormal termination of the embedded database system
    • Abnormal termination of an embedded database process
    • Abnormal termination of an operation command for the embedded database

      Table 1-16 Information needed to investigate the cause of failures and how to collect that information

      No. Component Information to be collected Collection method Performance No response Abnormal end
      1 OS Windows event log Use an OS function (command). 1 1 1
      2 CPU usage rate and device status Use the performance monitor. 3 4 3
      3 Embedded database Information about embedded database failures Obtain the files under the following folders and store them on a DAT or another storage device:
      • embedded-database-practical-directory\spool
      • embedded-database-practical-directory\tmp
      An error log file, a command log file, remote command information files, and node switching function information files are output to the above folders.      		 2 2 2
      4

      Error log file
      The error log is output to a file under embedded-database-practical-directory\spool\errlog.
      		 2 2 2
      5

      Command log file
      The command log is output to a file under embedded-database-practical-directory\spool\cmdlog.
      		 2 2 2
      6

      Remote command information files
      The information about remote commands is output to embedded-database-practical-directory\spool\pdrshs1 and pdrshs2.
      		 2 2 2
      7

      Node switching function information files
      Information about the node switching function is output to embedded-database-practical-directory\spool\pdshs1 and pdshs2.
      		 2 2 2
      8 Specification difference absorption library error information files The error information about the specification difference absorption library is output to embedded-database-practical-directory\UXPLDIR\SPOOL\uxpllog1 and uxpllog2. 2 2 2
      9 Embedded database system definitions Obtain the files under embedded-database-practical-directory\conf and store them on a DAT or another storage device. 4 5 4
      10 SQL trace file and error log file Obtain the output files and store them on a DAT or another storage device. A file name begins with pderr or pdsql. -- 6 5
      11 System log file Use the ajsembdboplog command to unload the system log. Obtain the unload log file and store it on a DAT or another storage device. 6 7 6

Legend:
--: The information need not be collected.

Note
The size of a file increases at a fixed rate if additional data continues to be stored in the file as a result of redirection. Because such a file takes up disk space, instead use bat files, which can be switched and reused after several generations.

(b) Information needed to re-create the embedded database environment

If a problem occurs during operation of the embedded database, you might need to re-create the environment that produced the problem to test whether the problem is reproducible or to investigate the cause of the problem. To enable this procedure, collect the following information needed to re-create the embedded database environment.

To collect the information needed to re-create the embedded database environment:

  1. Start the embedded database.

  2. Execute the ajsembdbrorg command with the -k unld option specified.

  3. Save conf under the embedded database practical directory in a folder of your choice.

  4. Record the environment variables related to the embedded database.

For details about how to use the ajsembdbrorg command and a description of the command, see 10.2.2 Reorganizing a database in the Job Management Partner 1/Automatic Job Management System 3 Administration Guide.

[Contents][Back][Next]

[Trademarks]

Copyright (C) 2009, 2010, Hitachi, Ltd.
Copyright (C) 2009, 2010, Hitachi Solutions, Ltd.