Notices

Summary of amendments

Preface

1. Application Server Functionality

1.1 Classifications of functionality

1.1.1 Functionality as an application execution platform

1.1.2 Functionality for operating and maintaining the application execution platform

1.1.3 Functionality and corresponding manuals

1.2 Functionality corresponding to the purpose of the system

1.2.1 Functionality for system maintenance

1.2.2 JavaVM functionality of the product

1.2.3 Functionality for migrating from products of earlier versions

1.3 Description of the functionality described in this manual

1.3.1 Meaning of classifications

1.3.2 Example of tables describing classifications

1.4 Main functionality changes in Application Server 09-50

2. Troubleshooting

2.1 Organization of this chapter

2.2 Overview of troubleshooting

2.2.1 Overview of Troubleshooting

2.2.2 Flow of data acquisition when a trouble occurs

2.3 Acquiring the Data

2.3.1 Data That Can Be Acquired Automatically When a Problem Occurs

2.3.2 Collecting the Material Using Commands during Error Detection

2.3.3 Collecting the Snapshot Log

2.3.4 Location to store the acquired information

2.4 Types of Required Data

2.4.1 Trouble types and the required data

2.4.2 List of Required Data to Be Acquired

2.4.3 Correspondence Between Acquisition Methods and Investigation Methods

2.5 Troubleshooting and Recovery

2.5.1 If the Configuration Software Process (Logical Server) Terminates Abnormally

2.5.2 When an attempt to forcefully terminate a J2EE application fails

2.5.3 If a Problem Occurs When Using the Database Session Failover Function

2.5.4 When an error occurs in the EADs session failover functionality

2.5.5 If a problem occurs when using the Memory Session Failover Function

2.5.6 If a problem occurs in the JPA Application

2.5.7 If JavaVM Terminates abnormally

2.5.8 If Administration Agent is terminated forcibly when OutOfMemoryError occurs

2.5.9 If a Problem Occurs in the System Linked with JP1

2.5.10 If a problem occurs in 1-to-1 node switching system

2.5.11 If problems occurs in N-to-1 recovery systems

2.5.12 If a problem occurs in the node switching system for the host unit management model

2.5.13 If a Problem Occurs in the EJB Client

2.6 Precautions Related to Troubleshooting

2.6.1 Precautions Related to the System Log of an EJB Client Application

2.6.2 Precautions When Using CTM

2.6.3 Precautions when using PRF

2.6.4 JavaVM data-related considerations

3. Preparing for Troubleshooting

3.1 Organization of this chapter

3.2 Overview of data acquisition settings

3.2.1 Specifiable contents

3.2.2 Overview of data acquisition settings (Systems that execute J2EE applications)

3.2.3 Overview of data acquisition settings (Systems executing batch applications)

3.3 Execution environment settings

3.3.1 Data acquisition settings using failure detection time commands (Systems for executing J2EE applications)

3.3.2 Data acquisition settings using failure detection time commands (Systems for executing batch applications)

3.3.3 Settings for collecting snapshot logs (Systems for executing J2EE applications)

3.3.4 Settings for collecting snapshot log (Systems for executing batch applications)

3.3.5 Settings for acquiring the Management Server log

3.3.6 Settings for Acquiring the J2EE Server Log

3.3.7 Settings for Acquiring the Batch Server Log

3.3.8 Settings for Acquiring the SFO Server Log

3.3.9 Settings for Acquiring the Web Server Log

3.3.10 Settings for Acquiring the Redirector Log

3.3.11 Settings for acquiring the in-process HTTP server log

3.3.12 Settings for Acquiring the Cosminexus Manager Log

3.3.13 Settings for Acquiring the Resource Adapter Logs

3.3.14 Settings for Acquiring the Cosminexus TPBroker Log

3.3.15 Settings for collecting Cosminexus JMS Provider logs

3.3.16 Settings for Collecting the OS Statistical Information

3.3.17 Settings for Collecting a User Dump

3.3.18 Settings for Acquiring a Core Dump

3.3.19 Settings for Acquiring the JavaVM Material

4. Output Destinations and Output Methods of Data Required for Troubleshooting

4.1 Organization of this chapter

4.2 Types of data used for troubleshooting (When snapshot log is not used)

4.3 Application Server log (Systems for executing J2EE applications)

4.3.1 Acquiring the Cosminexus Component Container Logs

4.3.2 Acquiring the Cosminexus Performance Tracer Log

4.3.3 Acquiring the Cosminexus Component Transaction Monitor Log

4.3.4 Acquiring the log output in audit log

4.3.5 Acquiring the Application User Log

4.4 Application Server log (Systems for executing batch applications)

4.4.1 Acquiring the Cosminexus Component Container Logs (systems executing batch applications)

4.4.2 Acquiring the Application User Log (systems executing batch applications)

4.5 EJB Client Application System Log

4.5.1 Types of EJB Client Application System Logs

4.5.2 Output Destination of the EJB Client Application System Log

4.6 Trace based performance analysis

4.7 JavaVM thread dump

4.7.1 When using the management command

4.7.2 When using separate commands

4.7.3 When using JavaVM commands

4.7.4 Precautions to be taken when class-wise statistical information is output in the thread dump

4.8 JavaVM Garbage Collection Log

4.9 Memory Dump

4.9.1 Acquiring a User Dump (In Windows)

4.9.2 Acquiring J2EE Server Memory Dump

4.9.3 Acquiring the CORBA Naming Service Memory Dump

4.9.4 Notes on obtaining the memory dump

4.10 JavaVM log (JavaVM log file)

4.11 JavaVM Output Message Logs (Standard Output or Error Report File)

4.11.1 In Windows

4.11.2 In UNIX

4.12 OS Status Information and OS Logs

4.12.1 Acquiring the OS Status Information

4.12.2 Acquiring OS Logs

4.13 OS Statistical Information

4.13.1 In Windows

4.13.2 In UNIX

4.14 Application Server definition information

4.15 Contents of J2EE server or batch server working directory

4.16 Application Server Resource Setting Information

4.17 Web Server Logs

4.18 JavaVM stack trace information

4.19 Event log of the Explicit Heap Memory Management functionality

4.20 Information on the execution of the Component Container Administrator setup command (In UNIX)

5. Problem Analysis

5.1 Organization of this chapter

5.2 Application Server Log

5.2.1 Output Format and Output Items of the Hitachi Trace Common Library Format Log

5.2.2 Precautions to Be Taken When Referencing the Hitachi Trace Common Library Format Log

5.2.3 Output format and output items of access log of In-process HTTP Server

5.2.4 Output Format and Output Items of the Event Log (In Windows)

5.2.5 Output Format and Output Items of syslog (In UNIX)

5.3 EJB Client Application Log

5.4 Trace based performance analysis

5.5 JavaVM Thread Dump

5.6 JavaVM Garbage Collection Log

5.7 JavaVM log (JavaVM log file)

5.7.1 Options to output the JavaVM log file

5.7.2 Acquiring the extended verbosegc information

5.7.3 Contents of the code cache area-related log

5.8 Message log output by JavaVM (Standard output and error report file)

5.8.1 When a Signal Occurs

5.8.2 When C Heap Is Insufficient

5.8.3 When an Internal Error Occurs

5.8.4 When Thread Creation Fails

5.9 OS status information and OS log

5.10 JavaVM stack trace information

5.10.1 When the -XX:+HitachiLocalsInThrowable Option Is Specified

5.10.2 When the -XX:+HitachiLocalsInStackTrace Option Is Specified

5.11 Event log of Explicit Memory Management functionality

5.11.1 Output trigger of event log of the Explicit Heap Memory Management functionality

5.11.2 Confirmation method of event log of Explicit Heap Memory Management functionality

5.11.3 Contents output when output level is normal

5.11.4 Contents output when output level is verbose

5.11.5 Contents Output when Output Level is Debug

6. Troubleshooting Procedure

6.1 Organization of this chapter

6.2 List of main problems

6.3 Processes that output logs

6.4 Troubleshooting during setup

6.5 Troubleshooting during operations

6.6 Troubleshooting the server management commands

6.7 Examples of troubleshooting during operations

6.7.1 Troubleshooting when a process is down

6.7.2 Troubleshooting when a response is delayed

7. Performance Analysis by Using Trace based Performance Analysis

7.1 Organization of this chapter

7.2 Overview of the trace based performance analysis

7.2.1 Overview of the trace based performance analysis of Application Server

7.2.2 Overview of the trace based performance analysis of applications

7.3 Collecting the trace based performance analysis file by using Management Server

7.3.1 How to collect a trace based performance analysis file

7.3.2 Output Destination and Output Information of a trace based performance analysis file

7.3.3 Output information of the trace based performance analysis file (for the trace based performance analysis)

7.3.4 Output information of the trace based performance analysis file (for the user-extended trace based performance analysis)

7.4 Implementation for collection of root application information of trace based performance analysis

7.5 Settings of execution environment

7.5.1 Settings for using the trace based performance analysis

7.5.2 Settings for using the user-extended trace based performance analysis

7.5.3 Settings for the methods to be traced by the user-extended trace based performance analysis

7.6 Logs output when the user-extended trace based performance analysis is executed

7.7 Analysis operation of the processing performance by using the trace based performance analysis file

7.7.1 Overview of the Operation for Analyzing the Processing Performance

7.7.2 Analyzing the Response Time of a Web Server

7.7.3 Investigating the Processing Status of a Request in an Application Server

7.7.4 Investigating the Life Cycle of a Session

7.7.5 Identifying the Transaction in Which a Timeout Occurred

7.7.6 Identifying the Request for Which Timeout Occurred

7.7.7 Investigating the Log Using the Root Application Information

7.7.8 Identifying the Connection in Which an Error Occurred

7.7.9 Investigation about the location of the problem associated to the trace based performance analysis file and thread dump

7.8 Notes on using the user-extended trace based performance analysis

8. Trace Collection Points and PRF Trace Collection Levels of the Trace Based Performance Analysis

8.1 Organization of this chapter

8.2 Trace Get Point of trace based performance analysis and the PRF Trace Get Level

8.3 Trace collection points of a CTM

8.3.1 Trace collection points and PRF trace collection levels

8.3.2 Trace information that can be collected

8.4 Trace collection points of a redirector

8.4.1 Trace collection points and PRF trace collection levels

8.4.2 Trace information that can be collected

8.5 Trace collection points of a Web container (trace of request processing)

8.5.1 Trace Get Point and the PRF Trace Get Level

8.5.2 Trace information that can be collected

8.5.3 Trace collection points and PRF trace collection levels (when an in-process HTTP server is used)

8.5.4 Trace information that can be collected

8.6 Trace collection points of a Web container (session trace)

8.6.1 Trace Get Point and the PRF Trace Get Level (Session Trace)

8.6.2 Trace information that can be collected

8.7 Trace collection points of a Web container (filter trace)

8.7.1 Trace collection points of a Web container when the processing terminates normally (filter trace)

8.7.2 Trace collection points of a Web container when an exception occurs (filter trace)

8.8 Trace collection points of a Web container (trace of the database session failover functionality)

8.8.1 Trace collection points and trace information that can be collected during request processing for creating an HTTP session (Trace of the database session failover functionality)

8.8.2 Trace collection points and trace information that can be collected during request processing for updating an HTTP session (Trace of database session failover functionality)

8.8.3 Trace collection points and trace information that can be collected during request processing for disabling an HTTP session (Trace of database session failover functionality)

8.8.4 Trace collection points and trace information that can be collected during request processing for disabling an HTTP session through valid period monitoring (Trace of database session failover functionality)

8.9 Trace collection points of a Web container (Trace of the EADs session failover functionality)

8.9.1 Trace collection points and trace information that can be collected during request processing for creating an HTTP session (Trace of the EADs session failover functionality)

8.9.2 Trace collection points and trace information that can be collected during request processing for updating an HTTP session (Trace of the EADs session failover functionality)

8.9.3 Trace collection points and trace information that can be collected during request processing for disabling an HTTP session (Trace of the EADs session failover functionality)

8.9.4 Trace collection points and trace information that can be collected when an EADs client is invoked (Trace of the EADs session failover functionality)

8.10 Trace collection points of a Web container (trace of the memory session failover functionality)

8.10.1 Trace collection points and PRF trace collection levels

8.10.2 Trace information that can be collected

8.11 Trace collection points of an EJB container

8.11.1 In the case of a Session Bean or Entity Bean

8.11.2 In the Case of Message-driven Bean (EJB2.0)

8.11.3 In the case of a Message-driven Bean (EJB2.1 and later)

8.11.4 For Timer Service

8.11.5 When the Session Bean is invoked asynchronously

8.11.6 When method cancellation occurs

8.12 Trace collection points of a JNDI

8.12.1 Trace Get Point and the PRF Trace Get Level

8.12.2 Trace information that can be collected

8.13 Trace collection points of a JTA

8.13.1 When a CMT and TransactionManager are used

8.13.2 When UserTransaction is used

8.13.3 In the case of a transaction timeout

8.13.4 When using the asynchronous concurrent processing for threads

8.14 Trace collection points of a DB Connector and JCA container

8.14.1 Connection-related trace collection points and trace information that can be collected

8.14.2 Trace collection points and trace information that can be collected when a local transaction is used

8.14.3 Trace collection points and trace information that can be collected when a connection association is used

8.14.4 Trace collection points and trace information that can be collected when the automatic connection close functionality is used

8.14.5 Trace collection points and trace information that can be collected in the case of linkage with the DB Connector for Cosminexus RM

8.14.6 Trace collection points and trace information that can be collected when work management is used

8.15 Trace collection points of an RMI

8.15.1 Trace get point and the PRF trace get level

8.15.2 Trace information that can be collected

8.16 Trace collection points of an OTS

8.16.1 Trace Get Point and the PRF Trace Get Level

8.16.2 Trace information that can be collected

8.17 Trace collection points of standard output, standard error output, and user log

8.17.1 Trace collection points of standard output or standard error output

8.17.2 Trace collection points of the user log

8.18 Trace collection points of a DI

8.18.1 Trace Get Point and the PRF Trace Get Level

8.18.2 Trace information that can be collected

8.19 Trace collection points of the batch application execution functionality

8.19.1 Trace Get Point and the PRF Trace Get Level

8.19.2 Trace information that can be collected

8.20 Trace collection points in a JPA

8.20.1 Trace collection points and trace information that can be collected when the persistent context of application management is used

8.20.2 Trace collection points and trace information that can be collected when the persistent context of container management is used

8.21 Trace collection points of the Cosminexus JPA provider

8.21.1 Trace collection points and trace information that can be collected during the acquisition or release processing of EntityManagerFactory

8.21.2 Trace collection points and trace information that can be collected during the acquisition processing of EntityManager

8.21.3 Trace collection points and trace information that can be collected during the operation of EntityManager

8.21.4 Trace collection points and trace information that can be collected during the release processing of EntityManager

8.21.5 Trace collection points and trace information that can be collected during the operation of Query

8.21.6 Trace collection points and trace information that can be collected during the operation of EntityTransaction

8.21.7 Trace collection points and trace information that can be collected in the case of callback method to the user

8.21.8 Trace collection points and trace information that can be collected during binary conversion of the entity class

8.21.9 Trace collection points and trace information that can be collected during transaction linkage with the transaction manager

8.21.10 Trace collection points and trace information that can be collected during the connection operation of the DB Connector

8.22 Trace collection points of the TP1 inbound integrated function

8.22.1 Trace collection points and PRF trace collection levels

8.22.2 Trace information that can be collected

8.23 Trace collection points of Cosminexus JMS Provider

8.23.1 Trace collection points of the JMS ConnectionFactory interface and the trace information that can be collected

8.23.2 Trace collection points of the JMS Connection interface and the trace information that can be collected

8.23.3 Trace collection points of the JMS session interface and the trace information that can be collected

8.23.4 Trace collection points of the JMS messages, producer, consumer, and queue browser and the trace information that can be collected

8.23.5 Trace collection points of CJMSP Broker when connecting to the CJMSP resource adapter and the trace information that can be collected

8.23.6 Trace collection points of the transaction management in the CJMSP resource adapter and trace information that can be collected

8.23.7 Trace collection points when Message-driven Bean is deployed from the CJMSP resource adapter and the trace information that can be collected

8.24 Trace collection points of JavaMail

8.24.1 Trace collection points of JavaMail transmission and the trace information that you can collect

8.24.2 Trace collection points on JavaMail receipt and the trace information that you can collect

8.25 Trace collection points of JSF

8.25.1 Trace collection points in the processing between JSF and Web container and the trace information that can be collected

8.25.2 Trace collection points in the processing between JSF and user applications

8.26 Trace collection points of CDI

8.26.1 Trace collection points of CDI and the trace information that can be collected

8.27 Trace collection points when a J2EE server is started or terminated

8.27.1 Trace Get Point and the PRF Trace Get Level

8.27.2 Trace information that can be collected

8.28 Trace collection points of an application

8.28.1 Trace collection points and PRF trace collection levels

8.28.2 Trace information that can be collected

9. Product JavaVM Functionality

9.1 Organization of this chapter

9.2 Overview of the product JavaVM functionality

9.3 Class-wise statistical functionality

9.3.1 Overview of the class-wise statistical functionality

9.3.2 Functionality that requires the class-wise statistical functionality

9.3.3 Outputting Statistic Information for Each Class

9.3.4 Precautions to output the class-wise statistical information

9.4 Instance statistical functionality

9.4.1 Overview of the instance statistical functionality

9.4.2 Class-wise statistical information output by the instance statistical functionality

9.5 STATIC member statistical functionality

9.5.1 Overview of the STATIC member statistical functionality

9.5.2 Class-wise statistical information output by the STATIC member statistical functionality

9.6 Reference-related information output functionality

9.6.1 Overview of the reference-related information output functionality

9.6.2 Class-wise statistical information output by the reference-related information output functionality

9.6.3 Class-wise statistical information output by the static field-based reference relationship output functionality

9.6.4 Notes for the output of the static field-based reference relationships

9.7 Pre-statistical garbage collection selection functionality

9.7.1 Overview of the pre-statistical garbage collection selection functionality

9.7.2 Guidelines for selecting the garbage collection

9.8 Unused objects statistical functionality in the Tenured area

9.8.1 Overview of the unused objects statistical functionality in the Tenured area

9.8.2 Class-wise statistical information output by the unused objects statistical functionality in the Tenured area

9.8.3 Notes for executing the unused objects statistical functionality in the Tenured area

9.9 Base object list output functionality for Tenured augmentation factors

9.9.1 Overview of the base object list output functionality for Tenured augmentation factors

9.9.2 Class-wise statistical information output by the base object list output functionality for Tenured augmentation factors

9.10 Class-wise statistical information analysis functionality

9.10.1 Overview of the class-wise statistical information analysis functionality

9.10.2 Output example of the class-wise statistical information analysis functionality

9.10.3 Notes for the class-wise statistical information analysis functionality

9.11 Tenuring distribution information output functionality of the Survivor area

9.11.1 Overview of the tenuring distribution information output functionality of the Survivor area

9.11.2 Output format and output example of the tenuring distribution information of the Survivor area

9.11.3 Settings for execution environment

9.11.4 Precautions when using tenuring distribution information output functionality of the Survivor area

9.12 hndlwrap functionality

9.12.1 Overview of the hndlwrap functionality

9.12.2 Notes for using the hndlwrap functionality

9.13 Functionality to set the upper limit of allocation size of C heap during JIT compilation

9.14 Functionality to set the upper limit of the number of threads

9.15 Notes on using the product JavaVM functionality (in UNIX)

10. Migrating from Application Server of Earlier Versions (In the J2EE Server Mode) (INTENTIONALLY DELETED)

10.1 (INTENTIONALLY DELETED)

11. Migrating to the Recommended Functionality

11.1 Migration from Microsoft IIS to Cosminexus HTTP Server

11.2 Migration to In-process Transaction Service

11.2.1 Migration flow

11.2.2 Migration of the operation

11.2.3 Migration of the acquisition method of the troubleshoot information

11.2.4 Program source code migration

11.3 Notes on migration to a database connection using HiRDB Type4 JDBC Driver

11.4 Migration to a database connection using Oracle JDBC Thin Driver from DABroker Library

Appendixes

A. List of Snapshot Logs to Be Collected

A.1 Overview of the list of the snapshot log to be collected

A.2 Cosminexus Component Container

A.3 Cosminexus Component Transaction Monitor

A.4 Cosminexus DABroker Library

A.5 Cosminexus Developer's Kit for Java

A.6 Cosminexus Performance Tracer

A.7 Cosminexus Web Services - Security

A.8 Cosminexus HTTP Server

A.9 Microsoft Internet Information Service

A.10 HCSC server

A.11 HCSC server (FTP receipt)

A.12 HCSC server (TP1 adapter)

A.13 HCSC server (File adapter)

A.14 HCSC server (Object Access adapter)

A.15 HCSC server (Message Queue adapter)

A.16 HCSC server (FTP adapter)

A.17 HCSC server (file operation adapter)

A.18 HCSC server (FTP inbound adapter)

A.19 HCSC server (mail adapter)

A.20 HCSC server (HTTP adapter)

A.21 Audit log

A.22 Other information

B. Identifying the Connection in which an Error Has Occurred When Connecting to a Database

B.1 Cosminexus Component Container

B.2 Cosminexus DABroker Library

B.3 HiRDB Client

B.4 HiRDB Server

B.5 Oracle Client

B.6 Oracle Server

C. Log Information Output for Each Functionality

C.1 Cosminexus JPA Provider operation log

D. Recovering Tables for a CMR When an Error Occurs

E. Main Functionality Changes in Each Version

E.1 Main functionality changes in 09-00

E.2 Main functionality changes in 08-70

E.3 Main functionality changes in 08-53

E.4 Main functionality changes in 08-50

E.5 Main functionality changes in 08-00

F. Glossary

Index

[Trademarks]

All Rights Reserved. Copyright (C) 2013, Hitachi, Ltd.