3.1.2 Memory monitoring examples

This subsection explains how to monitor memory performance.

Organization of this subsection

(1) Overview
(2) Monitoring methods

(1) Overview

You can monitor memory performance to detect physical memory shortages and incorrect process operations.

Memory consists of physical memory and a swap file, as illustrated below. However, because the causes of bottlenecks are not limited to a small amount of physical memory or a small swap file, the paging status, page faults, and other items related to efficient memory usage must be monitored as well.

The following figure illustrates the configuration of the memory space.

Figure 3‒2: Conceptual diagram of the memory space

Insufficient physical memory degrades overall system performance.

Memory areas not accessed by programs for a long time are saved to the swap file, and are loaded into physical memory on demand. Physical memory is efficiently used in this manner. Note, however, that swap file access is markedly slower than physical memory access. Therefore, frequent swapping or frequent page faults will considerably delay system processing.

Paging

Paging is the movement of code and data between physical memory and a paging file. Loading from the paging file into physical memory is referred to by the term page-in, and saving from physical memory to the paging file is referred to by the term page-out.
Page fault

A page fault is an attempt to access an area that does not exist in physical memory.

Because paging often occurs even in normal processing, measure performance when the system is operating stably before attempting to determine proper thresholds.

The Swap Outs alarm is provided by the monitoring template. If you want to perform more detailed monitoring, see the following table, which lists and describes the principal records and fields related to memory monitoring.

Table 3‒2: Principal fields related to memory monitoring
Record	Field	Description (example)
`PI`	`Total` `Physical` `Mem` `Mbytes`	The amount of physical memory.
	`Free` `Mem` `Mbytes`	The amount of available physical memory.
	`Alloc` `Mem` `Mbytes`	The amount of physical memory in use.
	`Alloc` `Mem` `%`	Physical memory usage.
	`Total` `Swap` `Mbytes`	Swap area size.
	`Free` `Swap` `Mbytes`	Free space in the swap area.
	`Alloc` `Swap` `Mbytes`	Swap area in use. If the value of this field continues to be at or above the threshold (the `Total` `Physical` `Mem` `Mbytes` field value of the `PI` record), a larger amount of memory might be required.
	`Alloc` `Swap` `%`	Swap area usage. If the value of this field continues to be at or above the threshold (determined by the system load status), the swap area might need to be expanded.
	`Page` `Scans/sec`	The number of page scans that occurred per second. If the value of this field continues to be at or above the threshold (150), memory might be a bottleneck.
	`Swapped-Out` `Pages/sec`	The number of swapped-out pages per second. If the value of this field continues to be at or above the threshold (200), memory might be a bottleneck.
	`Buffers` `Mem` `%`	The percentage of physical memory allocated to the file buffer. The field is unavailable in AIX, HP-UX, and Solaris.
	`Buffers` `Mem` `Mbytes`	The amount of physical memory allocated to the file buffer (MB). The field is unavailable in AIX, HP-UX, and Solaris.
	`Cache` `Mem` `%`	The percentage of physical memory allocated as cache memory. The field is unavailable in HP-UX and Solaris non-global zone environment.
	`Cache` `Mem` `Mbytes`	The amount of physical memory allocated as cache memory (MB). The field is unavailable in HP-UX and Solaris non-global zone environment.
	`Effective` `Free` `Mem` `%`	The percentage of physical memory available to applications. The field is unavailable in Solaris non-global zone environment.
	`Effective` `Free` `Mem` `Mbytes`	The amount of physical memory available to applications (MB). The field is unavailable in Solaris non-global zone environment.
	`Effective Used Mem %`	The percentage of physical memory actually allocated. The field is unavailable in Solaris non-global zone environment.
	`Effective Used Mem Mbytes`	The amount of physical memory actually allocated (MB). The field is unavailable in Solaris non-global zone environment.

The cause of a system memory shortage is not always physical memory itself. A problem with a program can also cause a shortage. By monitoring memory usage for each process, you can identify the cause of a shortage. If there is a process improperly occupying memory or if the amount of memory used by a process continues to increase steadily, the program running the process is likely to be defective.

The following table lists and describes the principal records and fields related to monitoring the memory usage of a specific process.

Table 3‒3: Principal fields related to the memory monitoring for each process
Record	Field	Description (example)
`PD_PDI`	`Real` `Mem` `Kbytes`	The amount of physical memory used by a process. If the value of this field is large, a specific process might be using a large amount of physical memory.
	`Virtual` `Mem` `Kbytes`	The amount of virtual memory used by a process. If the value of this field is large, a specific process might be using a large amount of virtual memory.
	`Swaps`	The number of times swapping occurred for a process. If the value of this field is large, you might need to look for a process that is causing a bottleneck due to frequent swapping.

To Page Top

(2) Monitoring methods

(a) Monitoring the memory usage status

You can use the usage status of virtual memory as a guideline for determining whether to increase physical memory.

Even when memory usage is temporarily high, if the high load status does not persist, performance degradation might be permissible. Therefore, monitoring the number of page scans and the number of swapped-out pages in addition to memory usage is recommended.

If the amount of used virtual memory (the Alloc Swap Mbytes field of the PI record) is larger than the total amount of physical memory (the Total Physical Mem Mbytes field of the PI record), more memory might be required.

For definition examples, see 3.2.2(2) Definition example other than for monitoring templates.

(b) Monitoring the number of page scans

You can use the Pagescans alarm provided by the monitoring template to monitor the number of page scans (Page Scans/sec field of the PI record).

Monitoring the swap-out status and memory usage in addition to the number of page scans is recommended.

If you find processes that are performing many page scans, you can take appropriate action. If no such processes exist, the system environment is inadequate for the processing. In this case, you might need to increase the amount of physical memory.

For details, see 3.2.2(1) Monitoring template.

(c) Monitoring the swap-out status

You can use the Swap Outs alarm provided by the monitoring template to monitor the swap-out processes (Swapped-Out Pages/sec field of the PI record).

Monitoring the number of page scans and the memory usage status in addition to the swap-out status is recommended.

If you find processes that are excessively swapping pages, take appropriate action. If no such processes exist, the system environment is inadequate for the processing. In this case, you might need to increase the amount of physical memory.

For details, see 3.2.2(1) Monitoring template.

(d) Monitoring memory usage by process

If you think there is a problem with a process after monitoring the memory usage status, the number of page scans, and the swap-out status, you must identify the process causing the problem.

If server activities have not increased, you can use a real-time report to monitor the memory usage of each process (by using the Real Mem Kbytes field of the PD_PDI record, for example) for a number of minutes. In the results displayed as a line graph, you can then check for a process whose memory usage is increasing steadily.

If you identify a process that is causing a memory leak or is excessively swapping pages, contact the vendor or take other appropriate action.

For definition examples, see 3.2.2(2) Definition example other than for monitoring templates.

(e) Checking the amount of memory that the system can actually use

In Linux, AIX, and Solaris, information already stored in memory is retained as long as possible. For this reason, when the Free Mem Mbytes field of the PI record is used to monitor the amount of available memory, the value of the field gradually approaches 0. However, the stored information can be released from memory anytime and does not prevent applications from using memory. The amount of memory that can be freed up anytime can be monitored by using the Buffers Mem Mbytes field (Linux) or Cache Mem Mbytes field (Linux, AIX, and Solaris) of the PI record. PFM - Agent for Platform calculates the amount of memory that is available in reality from these fields and outputs the result to the Effective Free Mem Mbytes field of the PI record. You can use this field to monitor the amount of memory that is actually available to the system.

In HP-UX, the same value is output to the Free Mem Mbytes and Effective Free Mem Mbytes fields. This value is the amount of memory that is available in reality.

To Page Top