7.2.1 Performance data collection and management
This subsection describes troubleshooting related to performance data collection and management procedures of Performance Management. For details about how to correct other types of errors, see the JP1/Performance Management User's Guide.
- Organization of this subsection
(1) No performance data is collected even when PFM - RM for Virtual Machine is started
The items to check and the actions to be taken are described below for each monitored virtual environment.
(a) For VMware
-
If you are monitoring information of a virtual machine running on VMware, check the following items:
-
Make sure that VMware Tools is running on the monitored virtual machine.
-
If you change the time on a physical server on which VMware is running. performance data may not be collected in some cases. In such cases, start the mgmt-vmware service.
-
-
If the Status field value of a PD record shows ERROR, take an action based on the value in the Reason field. The table below shows the actions to be taken.
Table 7‒2: Reason field value and actions to be taken Reason field value
Explanation
Action
Connection failed
An attempt to connect to the target virtual environment failed.
-
Check whether the host of the virtual environment is active.
-
Check whether the following items, which were specified during setup of the instance environment, are valid: #
- VM_Type
-
Check whether the following items, which were specified during setup of the monitoring target, are valid:#
- Target Host
- VM_Host
- Security
- Port
- UserID
- Password
-
Check whether the host name (Target Host or VM_Host), which was specified during setup of the monitoring target, can be resolved.
-
Check whether a valid certificate is installed in the PFM - RM for Virtual Machine host. When the default certificate of VMware is used in an environment in which a PFM - RM for Virtual Machine host cannot connect to the Windows Update site, check the Windows settings. For details, see 2.5.1 For VMware.
-
HTTPS connection must be permitted on the virtual environment side.
For details about how to set a communication method for a virtual environment, see the VMware documentation.
-
Make sure that the user ID that was specified for UserID during setup of the monitoring target has at least read-only permissions for VMware roles. For details about how to check permissions, see the VMware documentation.
Authorization failed
Authorization failed at the target virtual environment.
-
Check whether the host of the virtual environment is active.
-
Check whether the following items, which were specified during setup of the monitoring target, are valid: #
- UserID
- Password
- Domain
Timeout
Performance data collection did not end within the specified time.
-
Check whether the host of the virtual environment is active.
-
Check whether the following items, which were specified during setup of the instance environment, are valid: #
- VM_Type
-
Check whether the following items, which were specified during setup of the monitoring target, are valid: #
- Target Host
- VM_Host
- Security
- Port
- UserID
- Password
-
Check whether the host name (Target Host or VM_Host) which was specified during setup of the monitoring target, can be resolved.
-
Make sure that a certificate has been embedded. For PFM - RM for Virtual Machine, a certificate must be installed for each monitoring target. When the default certificate of VMware is used check the Windows settings. For details, see 2.5.1 For VMware.
-
If the cause cannot be determined, collect maintenance data and contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
Collection Error
A collection error occurred.
-
Check whether the following items that were specified during setup of the instance environment are correct:#
- HostUserID
- HostPassword
- HostDomain
-
If the cause cannot be determined, collect maintenance data and contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
- #
-
To check an item that has been set, execute the jpcconf command. Alternatively, on PFM - Web Console, view the Remote Monitor Configuration properties from the Remote Monitor Collector service of PFM - RM for Virtual Machine.
-
-
In all other cases, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
(b) For Hyper-V
-
If the Status field value of a PD record shows ERROR, take action as indicated by the value in the Reason field. The table below shows the actions to be taken.
Table 7‒3: Reason field value and actions to be taken Reason field value
Explanation
Action
Connection failed
Attempt to connect to the target virtual environment failed.
-
Check whether the host of the virtual environment is active.
-
Check whether the following items that were specified during setup of the instance environment are correct:#
- VM_Type
-
Check whether the following items, which were specified during setup of the monitoring target, are valid: #
- Target Host
- VM_Host
- UserID
- Password
- Domain
-
Check whether the host name (Target Host or VM_Host) that was specified during setup of the monitoring target can be resolved.
-
Check whether the WMI connection has been set up correctly. For PFM - RM for Virtual Machine, WMI connection settings must be specified for each monitoring target.
Authorization failed
Authorization failed at the target virtual environment.
-
Check whether the host of the virtual environment is active.
-
Check whether the following items that were specified during setup of the monitoring target are correct:#
- UserID
- Password
- Domain
Timeout
Performance data collection did not end within the specified time.
-
Check whether the host of the virtual environment is active.
-
Check whether the following items that were specified during setup of the instance environment are correct:#
- VM_Type
-
Check whether the following items, which were specified during setup of the monitoring target, are valid: #
- Target Host
- VM_Host
- UserID
- Password
- Domain
-
Check whether the host name (Target Host or VM_Host) that was specified during setup of the monitoring target can be resolved.
-
Check whether the WMI connection has been set up correctly. For PFM - RM for Virtual Machine, WMI connection settings must be specified for each monitoring target.
-
If the cause cannot be determined, collect maintenance data and contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
Collection Error
A collection error occurred.
-
Check whether the following items that were specified during setup of the instance environment are correct:#
- HostUserID
- HostPassword
- HostDomain
-
If the cause cannot be determined, collect maintenance data and contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
- #
-
Use the jpcconf command to check what has been set for an item. Alternatively, the Remote Monitor Configuration properties from the Remote Monitor Collector service of PFM - RM for Virtual Machine can be viewed on PFM - Web Console.
-
-
In all other cases, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
-
If the OS of the monitoring-target host is Windows, check the application event log.
(c) For KVM
-
If the Status field of the PD record stores the value ERROR, take action according to the value of the Reason field. The following table lists the values of the Reason field and describes the action to be taken for each value.
Table 7‒4: Action to be taken according to the value of the Reason field Reason field value
Explanation
Action
Connection failed
Connection to the destination virtual environment failed.
Check whether the host in the virtual environment has started.
Make sure that the following settings specified during instance environment setup are correct:#
- VM_Type
- SSH_Type
- SSH_Client
Make sure that the following settings specified during monitoring target setup are correct:#
- Target Host
- VM_Host
- Port
- UserID
- Private_Key_File
Make sure that the host name specified (for Target Host and VM_Host) during monitoring target setup can resolve to an IP address.
Make sure that the SSH connection settings are specified. For PFM - RM for Virtual Machine, these settings must be specified for each monitoring target.
Authorization failed
Authentication failed in the destination virtual environment.
Check whether the host in the virtual environment has started.
Make sure that the following settings specified during instance environment setup are correct:#
- HostUserID
- HostPassword
- HostDomain
Make sure that the following settings specified during monitoring target setup are correct:#
- UserID
Timeout
Performance data collection did not end within the specified length of time.
Check whether the host in the virtual environment has started.
Make sure that the following settings specified during instance environment setup are correct:#
- VM_Type
- HostUserID
- HostPassword
- HostDomain
Make sure that the following settings specified during monitoring target setup are correct:#
- Target Host
- VM_Host
- Port
- UserID
- Private_Key_File
Make sure that the host name specified (for Target Host and VM_Host) during monitoring target setup can resolve to an IP address.
Make sure that the SSH connection settings are specified. For PFM - RM for Virtual Machine, these settings must be specified for each monitoring target.
If the cause of the problem cannot be determined, collect maintenance data, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
Collection Error
A collection error occurred.
Make sure that the following settings specified during instance environment setup are correct:#
- HostUserID
- HostPassword
- HostDomain
If the cause of the problem cannot be determined, collect maintenance data, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
- #
-
To check the settings that have been specified, execute the jpcconf command. You can also check them from PFM - Web Console by referencing the Remote Monitor Configuration properties of the Remote Monitor Collector service of PFM - RM for Virtual Machine.
-
In any conditions other than the above condition, collect maintenance data, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
(d) For Docker environment
-
If the Status field of the PD record stores the value ERROR, take action according to the value of the Reason field. The following table lists the values of the Reason field and describes the action to be taken for each value.
Table 7‒5: Action to be taken according to the value of the Reason field Reason field value
Explanation
Action
Connection failed
Connection to the destination virtual environment failed.
-
Check whether the destination server is running.
-
Check whether the Docker Engine is configured to allow TCP connection on the destination server.
-
Check whether the following items, which were specified during setup of the instance environment, are valid: #
- VM_Type
-
Check whether the following items, which were specified during setup of the monitoring target, are valid:#
- Target Host
- VM_Host
- Security
- Port
-
Check whether the host name (Target Host or VM_Host), which was specified during setup of the monitoring target, can be resolved.
-
Check whether root certificate of the certificate authority and client certificates have been embedded. For details, see 2.5.4 For Docker environment.
-
Whether Port has been set to a correct value other than 0.
Timeout
Performance data collection did not end within the specified length of time.
-
Check whether the destination server is running.
-
Check whether the Docker Engine is configured to allow TCP connection on the destination server.
-
Check whether Docker environment running on the destination server uses API version 1.19 or later.
-
Check whether the following items, which were specified during setup of the instance environment, are valid: #
- VM_Type
- HostUserID
- HostPassword
- HostDomain
-
Check whether the following items, which were specified during setup of the monitoring target, are valid: #
- Target Host
- VM_Host
- Security
- Port
-
Check whether the host name (Target Host or VM_Host) which was specified during setup of the monitoring target, can be resolved.
-
Check whether root certificate of the certificate authority and client certificates have been embedded. For details, see 2.5.4 For Docker environment.
-
Whether Port has been set to a correct value other than 0.
-
If the cause cannot be determined, collect maintenance data and contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
Collection Error
A collection error occurred.
-
Check whether the following items that were specified during setup of the instance environment are correct:#
- HostUserID
- HostPassword
- HostDomain
-
If the cause cannot be determined, collect maintenance data and contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
- #
-
To check an item that has been set, execute the jpcconf command. Alternatively, on PFM - Web Console, view the Remote Monitor Configuration properties from the Remote Monitor Collector service of PFM - RM for Virtual Machine.
-
-
In all other cases, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter that explains troubleshooting in the JP1/Performance Management User's Guide.
(e) For Podman environment
-
If the Status field of the PD record stores the value ERROR, take action according to the value of the Reason field. The following table lists the values of the Reason field and describes the action to be taken for each value.
Table 7‒6: Action to be taken according to the value of the Reason field Reason field value
Explanation
Action
Connection failed
Connection to the destination virtual environment failed.
-
Check whether the host in the virtual environment has started.
-
Make sure that the following settings specified during instance environment setup are correct:#
- VM_Type
- SSH_Type
- SSH_Client
-
Make sure that the following settings specified during monitoring target setup are correct:#
- Target Host
- VM_Host
- Port
- UserID
- Private_Key_File
-
Make sure that the host name specified (for Target Host and VM_Host) during monitoring target setup can resolve to an IP address.
-
Make sure that the SSH connection settings are specified. For PFM - RM for Virtual Machine, these settings must be specified for each monitoring target.
Authorization failed
Authentication failed in the destination virtual environment.
-
Check whether the host in the virtual environment has started.
-
Make sure that the following settings specified during instance environment setup are correct:#
- HostUserID
- HostPassword
- HostDomain
-
Make sure that the following settings specified during monitoring target setup are correct:#
- UserID
Timeout
Performance data collection did not end within the specified length of time.
-
Check whether the host in the virtual environment has started.
-
Make sure that the following settings specified during instance environment setup are correct:#
- VM_Type
- HostUserID
- HostPassword
- HostDomain
- SSH_Type
- SSH_Client
-
Make sure that the following settings specified during monitoring target setup are correct:#
- Target Host
- VM_Host
- Port
- UserID
- Private_Key_File
-
Make sure that the host name specified (for Target Host and VM_Host) during monitoring target setup can resolve to an IP address.
-
Make sure that the SSH connection settings are specified. For PFM - RM for Virtual Machine, these settings must be specified for each monitoring target.
-
If the cause of the problem cannot be determined, collect maintenance data, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
Collection Error
A collection error occurred.
-
Make sure that the following settings specified during instance environment setup are correct:#
- HostUserID
- HostPassword
- HostDomain
-
If the cause of the problem cannot be determined, collect maintenance data, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
- #
-
To check the settings that have been specified, execute the jpcconf command. You can also check them from PFM - Web Console by referencing the Remote Monitor Configuration properties of the Remote Monitor Collector service of PFM - RM for Virtual Machine.
-
-
In any conditions other than the above condition, collect maintenance data, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
(f) For logical partitioning feature
-
If the Status field of the PD record stores the value ERROR, take action according to the value of the Reason field. The following table lists the values of the Reason field and describes the action to be taken for each value.
Table 7‒7: Action to be taken according to the value of the Reason field Reason field value
Explanation
Action
Connection failed
Connection to the destination virtual environment failed.
-
Check whether the HvmSh command (HvmSh.exe) has been copied to installation-folder\agt8\plugin\jpcagt5virtage.d\. For details about how to copy the HvmSh command (HvmSh.exe), see 2.5.6 For logical partitioning feature.
-
Make sure that the following setting specified during monitoring target setup is correct:#
- VM_Host
-
It is possible that after performance data was obtained last time, the state of logical partitions or the configuration of Hitachi Compute Blade logical partitioning feature was changed. This is not an error.
For details, see the Action column for the Message ID KAVL20201-W in Table 7-8.
-
If the cause cannot be determined, collect maintenance data and the LPAR Manager dump, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide. Also, for details about how to capture the LPAR Manager dump, see the topic that describes the LPAR Manager dump capture command in the User's Guide for each Hitachi Compute Blade device.
Response invalid
There was an unintended response from the server.
-
The version of Hitachi Compute Blade logical partitioning feature or HvmSh command (HvmSh.exe) is not supported. For details about the version of Hitachi Compute Blade logical partitioning feature and HvmSh command, see 2.1.1(4) Prerequisite programs. For details about how to upgrade the version of Hitachi Compute Blade logical partitioning feature, see the topic that describes the upgrade procedures of Hitachi Compute Blade logical partitioning feature in the User's Guide for each Hitachi Compute Blade device. For details about the HvmSh command (HvmSh.exe), see 2.5.6 For logical partitioning feature.
Timeout
Performance data collection did not end within the specified length of time.
-
Check whether Hitachi Compute Blade logical partitioning feature is working properly.
-
Make sure that the following setting specified during instance environment setup is correct:#
- VM_Type
-
Make sure that the following settings specified during monitoring target setup are correct:#
- Target Host
- VM_Host
-
Check whether the IP address for PFM - RM for Virtual Machine specified during the environment setup of Hitachi Compute Blade logical partitioning feature is correct. For details about the environment settings of Hitachi Compute Blade logical partitioning feature, see 2.5.6 For logical partitioning feature.
-
Check the state of the network for any problem.
-
If there is a firewall between PFM - RM for Virtual Machine and Hitachi Compute Blade logical partitioning feature, check whether the ports used for communication are properly set in the firewall. For details about the settings of the firewall, see D.2(7) Firewall passage direction during communication between PFM - RM for Virtual Machine and logical partitioning feature.
-
If the cause cannot be determined, collect maintenance data and the LPAR Manager dump, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide. Also, for details about how to capture the LPAR Manager dump, see the topic that describes the LPAR Manager dump capture command in the User's Guide for each Hitachi Compute Blade device.
Collection error
A collection error occurred.
-
Opening of or writing to a work file may have failed. Check to make sure that there is no disk space shortage.
-
Make sure that the work file used by PFM - RM for Virtual Machine is not open. If the work file is not open, check the memory usage with Task Manager. If free memory is not sufficient, exit some applications to free up memory.
-
If the cause cannot be determined, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
- #
-
To check the settings that have been specified, execute the jpcconf command. You can also check them from PFM - Web Console by referencing the Remote Monitor Configuration properties of the Remote Monitor Collector service of PFM - RM for Virtual Machine.
-
-
When one of the following messages is output to the common message log, take the corresponding action for each message ID. The following table shows the actions to take.
Table 7‒8: Message IDs and actions Message ID
Explanation
Action
KAVL20201-W
The system could not connect to the monitored virtual environment.
If the message in the message text is message=<Virtage(instance-name) Connect failed.>:
-
Check whether the HvmSh command (HvmSh.exe) has been copied to installation-folder\agt8\plugin\jpcagt5virtage.d\. For details about how to copy the HvmSh command (HvmSh.exe), see 2.5.6 For logical partitioning feature.
-
Check the memory usage with Task Manager. If free memory is not sufficient, exit some applications to free up memory.
-
If the problem is not solved, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
- If the message in the message text is message=<Virtage(instance-name) No data. record-type>:
-
See 5. Records to check whether the record type of the record is supported by Hitachi Compute Blade logical partitioning feature.
-
If the record is not supported, performance data cannot be collected. Configure the system so that the data is not collected.
-
If the record is supported, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
-
If the message in the message text is message=<Virtage(instance-name) Connect failed. Return:exit-code>:
-
When the exit code is one of 0x01000000, 0x11000000, and 0xFFFFFFFF:
According to the subsection 2.1.1(4) Prerequisite programs, check whether the version of the firmware of your Hitachi Compute Blade logical partitioning feature is consistent with that of the HvmSh command (HvmSh.exe).
For details about how to upgrade the version of Hitachi Compute Blade logical partitioning feature, see the topic that describes the upgrade procedures of Hitachi Compute Blade logical partitioning feature in the User's Guide for each Hitachi Compute Blade device.
-
When the exit code is none of the above:
Based on the exit code, see the action described in the topic on error messages in the Hitachi Compute Blade User's Guide.
If the cause cannot be determined, collect maintenance data and the LPAR Manager dump, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide. Also, for details about how to capture the LPAR Manager dump, see the topic that describes the LPAR Manager dump capture command in the User's Guide for each Hitachi Compute Blade device.
KAVL20203-W
Initialization of the collector process log file failed.
In the message in the message text, check the details shown in message=<Virtage(instance-name) Log failed. detailed-information>.
-
If the detailed information is either (Initialization error=%d) or (Level setting error=%d):
Opening of or writing to a work file may have failed. Check to make sure that there is no disk space shortage. If disk space is not a problem, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
-
If the detailed information is none of the above:
Check the memory usage with Task Manager. If free memory is not sufficient, exit some applications to free up memory. If free memory is sufficient, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
KAVL20204-W
Initialization of the collector process failed.
If the message in the message text is message=<Virtage(instance-name) Initialization failed. Param error=detailed-information>:
-
If the detailed information is VM_Host, check the following items:
- For the setting items of the monitoring target, if an IP address is specified for VM_Host, check whether the IP address is correct.
- For the setting items of the monitoring target, if a host name is specified for VM_Host, check whether the host name can be resolved.
-
If the detailed information is none of the above, check the memory usage with Task Manager. If free memory is not sufficient, exit some applications to free up memory.
-
If the problem is not solved, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
If the message in the message text is message=<Virtage(instance-name) Initialization failed.>:
-
Make sure that the work file used by PFM - RM for Virtual Machine is not open. Also, check to make sure that there is no disk space shortage.
If the problem is not resolved, check the memory usage with Task Manager. If free memory is not sufficient, exit some applications to free up memory.
-
If the cause cannot be determined, collect maintenance data and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
- #
-
To check the settings that have been specified, execute the jpcconf command. You can also check them from PFM - Web Console by referencing the Remote Monitor Configuration properties of the Remote Monitor Collector service of PFM - RM for Virtual Machine.
-
-
In any conditions other than the above condition, collect maintenance data, and then contact the system administrator. For details about how to collect maintenance data, see the chapter on troubleshooting in the JP1/Performance Management User's Guide.
(2) There is a large time difference between Record Time and Sampling Time
Because Sampling Time is the time in the monitored virtual environment, a slight time deviation may occur in normal operations. If the time deviation is large enough to cause an operational problem, check the following items:
-
Is the time set on the RM host different from that set in the monitored virtual environment?
If they are different, they need to be synchronized. Before changing the time settings, carefully read the notes related to the OS of the RM host and the virtual environment software.
-
If SSL/TLS is being used for communication with VMware, has a certificate been embedded?
If a certificate has not been embedded correctly, information collection may be delayed. See 2.5.1 For VMware, and correctly embed a certificate in the RM host. For details about how to embed certificates into VMware, see the VMware manuals.
-
If SSL/TLS is being used for communication with Docker environment, has a certificate been embedded?
If a certificate has not been embedded correctly, information collection may be delayed. See 2.5.4 For Docker environment and correctly embed a certificate in the RM host.