With the MCF trace facility, MCF obtains information, such as events and data sent or received, for each process. The MCF trace information is output to the trace area (trace buffer) in the shared memory. If the MCF trace disk output facility is specified in the MCF communication configuration definition and the trace area becomes full, the MCF trace information is output to an MCF trace file.

You might temporarily require the MCF trace information while the disk output facility is disabled. In this case, you can output the MCF trace information to an MCF trace file by using the MCF-trace-information acquisition start command (mcftstrtr) and the MCF-trace-information acquisition stop command (mcftstptr). For details about the commands, see the manual OpenTP1 Operation.

The MCF trace facility collects information about the functioning, before an error occurred, of control functions within a process and the control flow for each event.

When an OpenTP1 UAP (SUP, SPP, or MHP) terminates abnormally, trace information is output from the API called by the UAP. This is called a UAP trace, and the acquired information is referred to as UAP trace information. OpenTP1 acquires UAP trace information separately for each UAP process.

UAP trace information is stored as files named server-nameN.uat (where N represents a serial number appended to the core file of the UAP process) under the $DCDIR/spool/save/ directory. These files are known as UAP trace edit/output files.

UAP trace information can be output from a core file generated at abnormal termination of the UAP. To obtain the trace information, execute the uatdump command, specifying the required core file. The UAP trace information can be output to a specified file by redirecting the uatdump command's output destination.

The value specified in the uap_trace_max operand of the user service definition determines the maximum size of UAP trace information that can be stored in a file. When this value is reached, subsequent trace information is wrapped around to the start of the file.

UAP trace information is acquired separately for each abnormally terminated UAP process. When transaction processing called by an RPC on another node terminates abnormally, UAP trace information is acquired for each of the UAP processes executed on the two nodes.

For details about UAP traces, see description in the OpenTP1 Tester and UAP Trace User's Guide.

OpenTP1 has an RPC trace facility to obtain information about RPC service requests; this information is collected into a file. You can use an RPC management command rpcdump to dump the RPC trace information. Some of the uses of the RPC trace facility are:

• to find out which client UAP performed inter-process communication with which server UAP
• to find out the order in which a specified UAP executed services
• to analyze the flow of service requests up to the time an RPC error occurred

You can use the rpcdump command to edit the contents of an RPC trace file.

You can obtain an RPC trace file for each system service. To obtain the file, in the system service definition for each service specify that the trace is to be obtained. You can use the rpcmrg command to merge and print, in chronological order, the RPC trace files from several system services.

OpenTP1 has a performance verification trace to acquire trace information, such as an OpenTP1 identifier, for major events of services running in OpenTP1.

The advantages of conducting a performance verification trace are:

• tracing events across nodes and processes
• acquiring a trace for an internal event, rather than for an API, to verify which process causes a bottleneck in the performance

The following table lists and describes the system definitions that are related to performance verification trace.

Table 5-5 System definitions related to performance verification trace

Definition name	Format	Operand	Description
System common definition	set	prf_trace	Specifies whether performance verification trace information is to be acquired.
	set	trn_prf_trace_level	Sets the trace acquisition level
Performance verification trace definition	set	prf_file_size	Sets the size of trace file

For details about the definitions, see the manual OpenTP1 System Definition.

You can use the prfget command to retrieve a performance verification trace in binary format and then use the prfed command to output it in character format.

You can also use the dc_prf_utrace_put function to collect user-specific trace information in a trace file and the dc_prf_get_trace_num function to obtain the sequential number of the most recent trace acquired in the process.

Notes:

• When OpenTP1 is restarted in the normal way or by a hot standby method, no trace information is carried over.
• Trace acquisition performed by the performance verification trace facility is not subject to control by locks so that performance of online processing will not be affected. For this reason, if contention occurs during trace acquisition in a multiprocessor environment, some trace information may be missing or invalid trace information may be acquired. When trace information is edited using the prfed command, invalid information is displayed as error records.

OpenTP1 collects trace information about all events involving transactional linkage using the XA resource service, such as transaction requests from an application server and OpenTP1 transaction processing. This information is known as an XAR performance verification trace.

XAR performance verification traces are stored as files with the file name _xr_001, _xr_002, _xr_003, and so on, under the $DCDIR/spool/dcxarinf/ directory. These files are known as XAR trace information files for verification. The output directory and file names cannot be changed for the XAR performance verification trace information files. You can change the size and number of trace information files that can be acquired.

For details, see the description of the XAR performance verification trace definition in the manual OpenTP1 System Definition.

To collect XAR performance verification trace information:

1. Set Y in the prf_trace operand in the system common definition.
2. Specify the level of XAR performance trace information to acquire in the xar_prf_trace_level operand in the XA resource service definition.

The following table describes the XAR performance verification trace information that is collected depending on the value of the xar_prf_trace_level operand.

Table 5-6 Relationship between the value specified in the xar_prf_trace_level operand and acquired XAR performance verification trace information

Value of xar_prf_trace_level operand	Type of XAR trace information collected	Event ID	Trace data size (bytes)	Collection timing
00000001	Transaction request from an application server	0x4a00	128	Request to start a transaction branch	Immediately after call
		0x4a01	128		Immediately before return
		0x4a02	128	RPC execution request from within a transaction branch	Immediately after call
		0x4a03	128		Immediately before return
		0x4a04	128	Request to terminate a transaction branch	Immediately after call
		0x4a05	128		Immediately before return
		0x4a06	128	Request to prepare a transaction branch for commit	Immediately after call
		0x4a07	128		Immediately before return
		0x4a08	128	Request to commit a transaction branch	Immediately after call
		0x4a09	128		Immediately before return
		0x4a0a	128	Request to roll back a transaction branch	Immediately after call
		0x4a0b	128		Immediately before return
		0x4a0c	64	Request to report a transaction branch in Prepared or Heuristically Completed status	Immediately after call
		0x4a0d	64		Immediately before return
		0x4a0e	128	Request to discard a Heuristically Completed transaction branch	Immediately after call
		0x4a0f	128		Immediately before return
00000002	OpenTP1 transaction process	0x4b00	64	Start a transaction branch	Immediately before
		0x4b01	64		Immediately after
		0x4b02	64	Execute an RPC from within a transaction branch	Immediately before
		0x4b03	64		Immediately after
		0x4b04	64	Terminate a transaction branch	Immediately before
		0x4b05	64		Immediately after
		0x4b06	64	Prepare a transaction branch for commit	Immediately before
		0x4b07	64		Immediately after
		0x4b08	64	Commit a transaction branch	Immediately before
		0x4b09	64		Immediately after
		0x4b0a	64	Roll back a transaction branch	Immediately before
		0x4b0b	64		Immediately after
		0x4b0c	64	Report a transaction branch in Prepared or Heuristically Completed status	Immediately before
		0x4b0d	64		Immediately after
		0x4b0e	64	Discard a Heuristically Completed transaction branch	Immediately before
		0x4b0f	64		Immediately after

For details about the xar_prf_trace_level operand, see the description of the XA resource service definition in the manual OpenTP1 System Definition.

To collect XAR trace information files and output edited trace information, use the prfget and prfed commands. The acquisition and edit/output procedures are as follows.

Collecting XAR trace information for verification

To collect only the trace information that has not been acquired with the latest run ID, execute the following commands:

$DCDIR/bin/prfget -f _xr | $DCDIR/bin/prfed -d

To collect all trace information, execute the following commands:

$DCDIR/bin/prfget -a -f _xr | $DCDIR/bin/prfed -d

Editing and outputting XAR trace information for verification

To edit and output trace information from XAR trace information files for verification, execute the prfed command. Specify the options as required.

XAR trace information for verification is output in the same format as the performance verification trace. For details about the output format, see the description of the prfed command in the manual OpenTP1 Operation.

OpenTP1 collects trace information about various events related to journal buffering and journal output performed by the journal service. This information is called JNL performance verification trace.

JNL performance verification traces are stored as files with the file name _jl_001, _jl_002, _jl_003, etc., under the $DCDIR/spool/dcjnlinf/prfinf directory. These files are called JNL performance verification trace information files. The output destination and names of the JNL performance verification trace information files cannot be changed.

To collect JNL performance verification trace information:

1. Specify Y in the prf_trace operand in the system common definition.
2. Also specify in the system common definition the level of JNL performance verification trace information to be acquired. Make this specification in the jnl_prf_event_trace_level operand.

The table below shows the trace information that is collected depending on the specification of the jnl_prf_event_trace_level operand. For details about the timing of acquiring each event, see the description of the acquisition of performance verification trace information in the manual OpenTP1 Operation.

Table 5-7 Relationship between the jnl_prf_event_trace_level operand value and the trace information that is collected

jnl_prf_event_trace_level operand value	Event IDs of trace information
	0xc202, 0xc203, 0xc401, 0xc402	0xc001 to 0xc201, 0xc204 to 0xc400
00000000	N	N
00000001	Y	N
00000002	Y	Y
Other	Y	Y

Legend:

Y: Trace information is acquired.

N: Trace information is not acquired.

You use the prfget command to collect JNL performance verification trace information files and the prfed command to edit and output the files. The following describes how to collect the files.

How to collect JNL performance verification trace information files:

To collect only the trace information that has not been acquired with the most recent run ID, execute the following commands:

$DCDIR/bin/prfget -f _jl | $DCDIR/bin/prfed -d

To collect all trace information, execute the following commands:

$DCDIR/bin/prfget -a -f _jl | $DCDIR/bin/prfed -d

OpenTP1 collects trace information about locking involved in transaction processing. This information is called LCK performance verification trace.

LCK performance verification traces are stored as files with the file name _lk_001, _lk_002, _lk_003, etc., under the $DCDIR/spool/dclckinf/prf directory. These files are called LCK performance verification trace information files. The output destination and names of the LCK performance verification trace information files cannot be changed, but you can change the size and number of LCK performance verification trace information files that can be acquired. For details, see the description of the LCK performance verification trace definition in the manual OpenTP1 System Definition.

To collect LCK performance verification trace information:

1. Specify Y in the prf_trace operand in the system common definition.
2. In the lck_prf_trace_level operand in the lock service definition, specify the level of LCK performance verification trace information to acquire.

The table below shows the LCK performance verification trace information that is collected depending on the specification of the lck_prf_trace_level operand.

Table 5-8 Relationship between the lck_prf_trace_level operand value and the LCK performance verification trace information that is collected

lck_prf_trace_level operand value	LCK performance verification trace information	Event ID	Trace data size (bytes)	Collection timing
00000000	Trace information about locking is not acquired.	--	--	--
00000001	Trace information about locking is acquired.	0x6400	128	Locking of resources	Immediately after call
		0x6401	128		Immediately before return
		0x6410	128	Lock release wait	Immediately before start
		0x6411	128		Immediately after termination
		0x6420	128	Unlocking of all resources	Immediately after call
		0x6421	128		Immediately before return
		0x6430	128	Unlocking with a resource name specified	Immediately after call
		0x6431	128		Immediately before return

For details about the lck_prf_trace_level operand, see the description of the lock service definition in the manual OpenTP1 System Definition.

You use the prfget command to collect LCK performance verification trace information files and the prfed command to edit and output the files. The following describes how to collect and how to edit and output the files.

How to collect LCK performance verification trace information files:

To collect only the trace information that has not been acquired with the most recent run ID, execute the following commands:

$DCDIR/bin/prfget -f _lk | $DCDIR/bin/prfed -d

To collect all trace information, execute the following commands:

$DCDIR/bin/prfget -a -f _lk | $DCDIR/bin/prfed -d

How to edit and output LCK performance verification trace information files:

To edit and output trace information in LCK performance verification trace information files, execute the prfed command with options specified, as necessary.

The output format of LCK performance verification trace information is the same as for the performance verification trace. For details about the output format, see the prfed command in the manual OpenTP1 Operation.

OpenTP1 collects trace information (such as the MCF identifier) about the main events involving message transmission using TP1/Message Control. This information is called MCF performance verification trace.

The MCF performance verification trace provides the following advantages:

• You can output MCF-specific details, such as thread ID, logical terminal name, and API name.
• If you analyze the MCF performance verification trace by using the detailed information that was output as the key, you can verify the performance of a series of message transmission processing transactions and UAP.

The following table lists and describes the system definitions that are related to the MCF performance verification trace.

Table 5-9 System definitions related to the MCF performance verification trace

Definition name	Format	Operand	Description
User service definition	set	mcf_prf_trace	Specifies whether MCF performance verification trace information is acquired for each user server
MCF performance verification trace definition	set	prf_file_size	Size of a trace file for the MCF performance verification trace information
	set	prf_file_count	Number of trace file generations for the MCF performance verification trace information
Definition of system service information	set	mcf_prf_trace	Specifies whether MCF performance verification trace information is acquired for each MCF communication service
System service common information definition	set	mcf_prf_trace_level	Acquisition level of MCF performance verification trace information

For details about the definitions, see the manual OpenTP1 System Definition.

You can use the prfget command to retrieve an MCF performance verification trace in binary format and then use the prfed command to output it in character format.

Notes

• The default is that the MCF performance verification trace is not collected. To collect it, you must specify Y in the prf_trace operand or omit this operand in the system common definition and also specify 00000001 in the mcf_prf_trace_level operand in the system service common information definition.
• You can collect the MCF performance verification trace during message transmission (event IDs 0xa000 and 0xa001) only when you use one of the following protocol products:
  TP1/NET/TCP/IP
  TP1/NET/XMAP3
  TP1/NET/OSAS-NIF
  For events other than message transmission (event IDs other than 0xa000 and 0xa001), you can collect the MCF performance verification trace regardless of the type of communication protocol.
  

The XAR event trace acquires the type of the transaction request sent from an application server using the XA resource service as event trace information. The acquired information is called the XAR event trace information.

The following table lists the types of transaction requests sent from an application server, request codes, and their meaning.

Table 5-10 Types of transaction requests and request codes

Request type#	Request code	Meaning of request code
Start()	xar_start	Starts a transaction branch.
Call()	xar_call	Executes an RPC from within a transaction branch.
End()	xar_end	Ends a transaction branch.
Prepare()	xar_prepare	Prepares the commit for a transaction branch (first phase of a two-phase commit).
Commit()	xar_commit	Commits a transaction branch (second phase of a two-phase commit).
Rollback()	xar_rollback	Rolls back a transaction branch.
Recover()	xar_recover	Notifies a transaction branch in the Prepared status or the Heuristically Completed status.
Forget()	xar_forget	Discards a transaction branch in the Heuristically Completed status.

#

Transaction request types are internal functions of OpenTP1.

The XAR event trace information is acquired in the xarevtr1 or xarevtr2 file under the $DCDIR/spool/dcxarinf/trace/ directory. These files are called the XAR event trace information files. You cannot change the output destinations and files names of XAR event trace information files.

If there is an existing XAR event trace information file when you start the XA resource service, a backup file is created for the existing file and another, unused XAR event trace information file is prepared as the output destination. For example, if the xarevtr1 XAR event trace information file exists, the file is renamed to xarevtr1.bk1. Because this renaming causes the file name xarevtr1 to become available, a new XAR event trace information file is then created with the name of xarevtr1. Up to three generations of backup files are created. Backup files are created only when there is already an XAR event trace information file at the start of the XA resource service. During online processing, if the number of records written to one file exceeds the value of the maximum number of records that can be output to an XAR event trace information file, then no backup file is created. This maximum number is specified in the xar_eventtrace_record operand in the XA resource service definition. If one output file becomes full, the next file is used. If all the files have become full, the oldest one is overwritten with new records. Do not delete the XAR event trace information files while the system is online.

You can specify the output level of the XAR event trace information by using the xar_eventtrace_level operand in the XA resource service definition. By default, the output level is specified to acquire the XAR event trace only if an error occurs. You can acquire all the XAR event trace information if you want to. However, in that case, the online performance is affected. We recommend that you use the default output level except for debugging.

You can use the xarevtr command to edit and display the XAR event trace information acquired in the output file.

For details on how to specify the output level of the XAR event trace information and the number of output records, see the manual OpenTP1 System Definition. For details on how to edit and display the XAR event trace information, see the manual OpenTP1 Operation.

The TRN event trace is historical information about the XA functions issued in transaction branches, and about the events of transaction services (transaction management service, transaction recovery service, and resource manager monitoring service).

The TRN event trace is stored as files named _tr_nnn (nnn: 3-digit number starting at 001) in the $DCDIR/spool/dctrninf/trace/prf directory. These files are called TRN event trace information files. You cannot rename or move these files.

You can change the size of each TRN event trace information file and the maximum number of TRN event trace information files that can be stored. For details, see the TRN event trace definition in the manual OpenTP1 System Definition.

To collect the TRN event trace information:

1. Set Y in the prf_trace operand in the system common definition.
2. Use the trn_prf_event_trace_level operand in the transaction service definition to specify the level for collecting the TRN event trace information.
3. Use the trn_prf_event_trace_condition operand in the transaction service definition to specify the type of TRN event trace collection.

The following table lists the values that can be specified in the trn_prf_event_trace_condition operand and the types of TRN event trace collection specified by the values.

The table below shows the TRN event trace information that is collected depending on specification of the trn_prf_event_trace_condition operand.

Table 5-11 Relationship between the trn_prf_event_trace_condition operand value and the TRN event trace information that is collected

Value of trn_prf_event_trace_condition	TRN event trace information that is collected	Event ID	Trace data size (bytes)	Collection timing
xafunc	Trace information about XA functions	0x4500	320 (192 for the xa_open and xa_close functions)	During transaction processing#
trnservice	Trace information about the operating status of the transaction service	0x4501	192	When the transaction service starts, ends, or is recovered

#

For a transaction committed in two phases, the trace amount collected per transaction branch is 12 number of resource managers. However, the trace amount varies depending on the conditions, such as the XA interface object files linked to the user server and the transaction optimization settings.

For details about this operand, see the manual OpenTP1 System Definition.

To collect TRN event trace information files and output edited trace information, use the prfget and prfed commands. The acquisition and edit/output procedures are as follows.

Collecting TRN event trace information

To collect only the trace information that has not been acquired with the latest run ID, execute the following commands:

$DCDIR/bin/prfget -f _tr | $DCDIR/bin/prfed -d

To collect all the trace information, execute the following commands:

$DCDIR/bin/prfget -a -f _tr | $DCDIR/bin/prfed -d

Outputting TRN event trace information

To output TRN event trace information, execute the prfed command with the -d option specified. Specify other options as required.

TRN event trace information is output in the same format as the performance verification trace. For details about the output format, see the description of the prfed command in the manual OpenTP1 Operation.

The following shows an example of TRN event trace information output when xafunc is specified in the trn_prf_event_trace_condition operand.

PRF: Rec Node: trn1 Run-ID: 0x4046b806 Process: 26264      Trace: 10
Event: 0x4500 Time: 2004/01/01 12:34:56 678.123.000 Server-name: Sup
Rc:   0      Client: **** - ********** Server: **** Root: trn1 - ********
Svc-Grp: *************************** Svc: ****************************
Trn: 78d0trn100000001trn1trn100000001
    xa_commit                       (IN)  OpenTP1_TAM
    axid:010000000000000c00000010
         3738643074726e330000000174726e3374726e330000000100000000
  Internal code1:0x2007       Internal code2:   0
  Internal code3:0

OpenTP1 collects trace information about the various events generated by the name service, including communication invoked by the name service and the registration and deletion of service information from the cache. This information is known as an NAM event trace.

NAM event traces are stored as files with the file name _nm_001, _nm_002, _nm_003, and so on, under the $DCDIR/spool/dcnaminf/ directory. These files are known as NAM event trace information files. The output directory and file names cannot be changed.

To collect NAM event trace information:

1. Set Y in the prf_trace operand in the system common definition.
2. Specify the level of NAM event trace information to acquire in the nam_prf_trace_level operand in the system common definition.

The following table lists the values that can be specified in the nam_prf_trace_level operand and the types of NAM event trace information acquired for each level. For the collection timing for each event, see the description of collecting performance verification trace information in the manual OpenTP1 Operation.

Table 5-12 Relationship between the value specified in the nam_prf_trace_level operand and acquired NAM event trace information

Value of nam_prf_trace_level operand	Event ID of trace information
	0xf000 to 0xf029	0xf100 to 0xf109	0xf200 to 0xf218
00000000	N	N	N
00000001	N	Y	N
00000002	Y	N	N
00000003	Y	Y	N
00000004	N	N	Y
00000005	N	Y	Y
00000006	Y	N	Y
00000007	Y	Y	Y
Other value	Y	Y	N

Legend:

Y: Trace information is acquired.

N: Trace information is not acquired.

To collect NAM event trace information files and output edited trace information, use the prfget and prfed commands. The acquisition and edit/output procedures are as follows.

To collect NAM event trace information, execute the following commands:

$DCDIR/bin/prfget -a -f _nm | $DCDIR/bin/prfed -d

OpenTP1 collects information about process events such as process creation, destruction, activation, and termination. This information is known as a process service event trace.

Process service event traces are stored as files with the file name _pr_001, _pr_002, _pr_003, and so on, under the $DCDIR/spool/dcprcinf/ directory. These files are known as process service event trace information files. The output directory and file names cannot be changed.

To collect process service event traces, set Y in the prc_prf_trace operand in the process service definition.

For the collection timing for each event, see the description of collecting performance verification trace information in the manual OpenTP1 Operation.

To collect process service event trace information files and output edited trace information, use the prfget and prfed commands. The acquisition and edit/output procedures are as follows.

To collect process service event trace information, execute the following commands:

$DCDIR/bin/prfget -a -f _pr | $DCDIR/bin/prfed -d

OpenTP1 collects event information when the amount of time it takes to process an OpenTP1 file access request issued internally from a process under OpenTP1 control exceeds the value specified in the fil_prf_trace_delay_time operand in the system common definition. This is called a FIL event trace.

By analyzing the FIL event trace, you can check the delay status of processing related to OpenTP1 file access.

FIL event traces are stored as files with the file name _fl_001, _fl_002, _fl_003, etc., under the $DCDIR/spool/dcfilinf/ directory. These files are called FIL event trace information files. The output destination and names of the FIL event trace information files cannot be changed.

You use the prfget command to collect FIL event trace information files and the prfed command to edit and output the files. The following describes how to collect the files.

How to collect FIL event trace information files:

$DCDIR/bin/prfget -a -f _fl | $DCDIR/bin/prfed -d

Every time an OpenTP1 operation command is executed, information such as the command start time and command termination time is output to cmdlog1 and cmdlog2 in $DCDIR/spool/cmdlog.

You can open cmdlog1 and cmdlog2 with a text editor such as vi. From the logged command start time and command termination time, you can obtain, for example, the time required to execute the command (response time).