6.2 Data to be collected in the event of an error

This section discusses the data that should be collected in the event of an error. You must collect data when any of the following events occurs:

The table below lists the data that needs to be collected for each event.

Table 6-1 Data that needs to be collected for each event

No.DataEvent
Invalid timeoutAbnormal server shutdownServer process shutdown
1Standard output, standard error outputYYY
2Error report file----Y
3SDP server definition filesYYY
4Adaptor definition filesYYY
5Log filesYYY
6Trace filesY#1YY
7Adaptor trace informationY#2YY
8Tuple logsY#3YY
9Thread dumpYYY
10OS statusY----
11OS statisticsYYY
12Hitachi Web Server error log file#4YYY
Legend:
Y: Must be collected
--: Need not be collected
#1
You can collect this data after you have shut down the SDP server. For details about shutting down the SDP server in the event of an invalid timeout, see 6.4.1(5) A timeout occurred during query processing.
#2
You can collect this data after you have terminated the adaptors.
#3
You can collect this data by terminating query groups.
#4
You can collect the Hitachi Web Server error log file in the event of a Dashboard Server error.

The table below describes how to collect the data.

Table 6-2 How to collect the data

No.DataHow to collect
1Standard output, standard error outputMake a note of (or copy) the messages output to the standard output and standard error output
2Error report fileCopy the files under the following directory:
working-directory\
3SDP server definition filesCopy the files under the following directory:
working-directory\conf\
4Adaptor definition filesCopy the files under the following directory:
working-directory\conf\xml\
5Log filesCopy the files under the following directory:
working-directory\logs\
6Trace filesCopy the files under the following directory:
working-directory\trc\
7Adaptor trace informationCopy the files under the following directory:
working-directory\trc\adaptor\
8Tuple logsCopy the files under the following directory:
working-directory\trc\tuplelog\
9Thread dumpCollect data by executing the jheapprof command
10OS statusCollect data by executing the netstat command
11OS statisticsCollect performance monitor logs
12Hitachi Web Server error log fileCollect the Hitachi Web Server error log files under the following directory:
installation-directory\psb\CC\web\containers\uCSDPAF_Server\logs\

The details about how to collect each type of data are described below.

Organization of this section
(1) Standard output and standard error output
(2) Error report file
(3) SDP server definition files
(4) Adaptor definition files
(5) Log files
(6) Trace files
(7) Adaptor trace information
(8) Tuple logs
(9) Thread dump
(10) OS status
(11) OS statistics
(12) Hitachi Web Server error log file

(1) Standard output and standard error output

Make a note of (or copy) the messages output to the standard output and standard error output.

Note that all messages related to startup and termination of the SDP server and adaptors are output to the standard output or the standard error output. Some of these messages are also output to log files, but others are not.

(2) Error report file

An error report file is output in the event of abnormal termination of JavaVM. You can use this file to determine the location and cause of a process shutdown.

The error report file is output directly under the working directory and has the following file name:

hs_err_pidprocess-ID.log

(3) SDP server definition files

The SDP server definition files include the following files:

Collect these files under the following directory:

working-directory\conf\

If the JavaVM options file for RMI connections is not stored under this directory, also collect that file.

(4) Adaptor definition files

The adaptor definition files include the following files:

Collect these files under the following directory:

working-directory\conf\xml\

(5) Log files

The log files include the following:

Collect all the files under the following directory:

working-directory\logs\

For details about the message logs and trace logs, see 6.3.1 Details of log files.

The Hitachi JavaVM log file is the log file to which Hitachi-specific JavaVM logs are output; it is obtained by using extended options added by Hitachi to the standard JavaVM. This file provides more troubleshooting information than the standard JavaVM.

(6) Trace files

The trace files include various trace files, such as API trace information.

Collect all the files under the following directory:

working-directory\trc\

For details about the API trace information, see 6.3.2 Details of API trace information.

(7) Adaptor trace information

Adaptor trace information is provided as trace files to which the adaptor processing status is output.

Collect the files under the following directory:

working-directory\trc\adaptor\

For details about the adaptor trace information, see 6.3.3 Details of adaptor trace information.

(8) Tuple logs

Tuple logs are the log files to which information about the input and output tuples is output.

Collect the files under the following directory:

working-directory\trc\tuplelog\

For details about the tuple logs, see 6.3.4 Details of tuple logs.

(9) Thread dump

A thread dump is a file to which information about the threads running inside Java processes is output.

You use the jheapprof command to collect a thread dump.

For details about how to collect a thread dump and details about the jheapprof command, see 6.3.5 Details of a thread dump.

(10) OS status

Collect as the OS status the network information and environment variables. To collect this information, execute the netstat command as follows:

netstat -e > netstat_e.txt
netstat -s > netstat_s.txt
netstat -an > netstat_an.txt

set > set.txt

(11) OS statistics

The OS statistics includes information that enables you to check the status, such as system loading and system performance. You use the OS's performance monitor to collect this information.

While the SDP server is executing, collect at a regular interval the performance monitor logs shown in the table below.

Table 6-3 Performance monitor logs

ObjectInstanceCounter
Processor_Total%Processor Time
%Privileged Time
%User Time
Memory--Cache Bytes
Cache Faults/sec
Page Faults/sec
Transition Faults/sec
Transition Faults/sec
Process_TotalHandle Count
Page Faults/sec
Private Bytes
Virtual Bytes
Working Set Bytes
java#%Processor Time
%Privileged Time
%User Time
Page Faults/sec
Thread Count
Private Bytes
Virtual Bytes
Working Set Bytes
Legend:
--: Not applicable
#
This information can be collected only if collection of performance logs has already been started in advance.

We recommend setting 60 seconds as the log collection interval; however, you should determine the appropriate interval on the basis of disk capacity. Setting the collection interval to a large value can reduce the adverse effects of OS statistics collection on performance, but the accuracy of the OS statistics might be compromised.

For details about how to specify the setting, see the OS documentation.

(12) Hitachi Web Server error log file

The Hitachi Web Server error log file is used to check and monitor the Web container operating status. Error, warning, and information messages are output to this file.

If you output stream data summary analysis results to a dashboard using the standard adaptors, collect the following log file in the event of a Dashboard Server error (the question mark (?) in the log file name represents a 1- or 2-digit integer):

installation-directory\psb\CC\web\containers\uCSDPAF_Server\logs\user_err?.log