9.5.1 Product plugin

(a) Type of SID of the target configuration and its available functions

The format of the configuration SID that corresponds to IM managed node that JP1/IM - Agent creates depends on the type of add-on program that you use to monitor.

For each type of exporter, the SID type of configuration information and the functions that can be used are shown below.

(b) SID Creation Condition for JP1/IM agent base

Obtain JP1/IM agent base configuration from Intelligent Integrated Management Base and create the following SID:

• Integrated agent host SID

• JP1/IM agent control base SID

If you use user-defined Prometheus or Fluentd to monitor a target and you do not have integrated agent installed, JP1/IM agent control base SID for that host is not created.

If JP1/IM - Manager is in a hierarchical configuration and IM Managed Node Creation Perform function is performed on the Integrated Manager, the configuration managed by lower manager is also retrieved. Therefore, Intelligent Integrated Management Base of lower manager must be accessible. For details about the prerequisites for JP1/IM - Manager in a hierarchical structure, see "3.5.8 Monitoring of multiple sites" in Intelligent Integrated Management Base.

(c) Creation condition of IM management node other than JP1/IM agent base

JP1/IM - Agent creates an IM management node other than JP1/IM agent base from trend data stored in Trend data Management Database of JP1/IM - Manager. Therefore, you can start add-on program service. After waiting for about one minute^#, you can create IM management node.

#

If the value of Prometheus configuration file (jpc_prometheus_server.yml) scrape_interval has been changed, wait for the value and then operate.

When jddcreatetree command is executed, an IM management node is not created for an add-on program that has not been saved in Trend data Management Database, or user-defined Prometheus or Fluentd.

If either of the following is true, metric is not saved in Trend data Management Database.

• With Add-on program, or user-defined Prometheus or Fluentd services are stopped or deleted, Trend data Management Database retention period (defaults to 32 days) has passed.

• With Add-on program, or user-defined Prometheus or Fluentd services are stopped or deleted, metric is removed due to an Trend data Management Database upper limit^#

  #

  For details about deleting metric for Trend data Management Database, see "2.7.2(3) Deleting trend data".

Tagging is required on AWS when using Yet another cloudwatch exporter. For more information, see "9.5.1(1)(k) Creating an IM Management Node for Yet another cloudwatch exporter".

Even if you use user-defined Prometheus or Fluentd, create an IM management node if trend data is stored in Trend data Management Database.

If JP1/IM - Manager is in a hierarchical configuration and the Integration Manager performs IM Managed Node Creation Perform function, IM Managed Node under lower manager is created from trend data stored in Trend data Management Database of lower manager. Therefore, Intelligent Integrated Management Base of lower manager must be accessible. In addition, metric must be stored in Trend data Management Database of lower manager.

• For details about JP1/IM - Manager prerequisites for hierarchical structure, see "(1) Prerequisites" in "3.5.8 Monitoring of multiple sites".

• For details about the various data managed by JP1/IM - Manager, see " 3.6.1 Introduction" in " 3.6 Operation using WebGUI (integrated operation viewer)".

(d) Creating and Operating IM management node for stopped Service

IM management node is created even if integrated agent host, add-on program, or user-defined Prometheus or Fluentd service is temporarily stopped for maintenance or other reasons. However, if the trend database expires or if metric is deleted because of an upper limit of size, IM management node is no longer created. If IM management node was unintentionally deleted, IM management node will be created again by creating and importing IM management node tree while the service is running and metric is saved in Trend data Management Database.

(e) Creating and Operating IM management node for Configuration Changes or Deletions

If you change or remove the configuration for add-on program, or user-defined Prometheus or Fluentd, IM management node is created prior to the configuration change or deletion unless their metric is removed from Trend data Management Database. For details about how to not create an IM management node before configuration change or deletion, see "1.21.2(10) Creating and importing IM management node tree information (for Windows)" and "2.19.2(11) Creating and importing IM management node tree information (for Linux)" in the manual "JP1/Integrated Management 3 - Manager Configuration Guide".

(f) Property display

Displays the properties of the JP1/IM - Agent IM management node in the node details area of the JP1/IM - Manager Integrated Operation Viewer.

The values that the JP1/IM - Agent sets as properties of the IM management node are shown below. Note that properties are not set for SIDs not listed here.

(Legend)

--: Not applicable

#1

If the value of each property exceeds 255 characters, the 256th and subsequent characters are truncated.

#2

Displays only if JP1/IM-Agent is a IM managed node of the included Blackbox exporter.

#3

If the "Any dimension_ string" label does not exist for the Yet another cloudwatch exporter metric, do not register it. If more than one exists, register that number.

Prometheus hostname

For PrometheusSID or AlertmanagerSID, sets the host name where the Prometheus server is set up. Otherwise, set the host name of the Prometheus server scraping the exporter corresponding to the IM management node.

Scrape Job Name

The scrape_configs of the Prometheus configuration file (jpc_prometheus_server.yml) contains the value set to job_name.

jp1_pc_exporter value

In the scrape_configs of the Prometheus configuration file (jpc_prometheus_server.yml), the value set to jp1_pc_exporter is entered. If no value is set, Unknown Exporter is entered.

jp1_pc_module value

scrape_configs of the Prometheus configuration file (jpc_prometheus_server.yml) contains the value set in the params module.

Account String

Contains the value of the AWS account string corresponding to the account ID to be monitored specified in the AWS definition file (aws_settings.conf).

Region name

Contains the value of the region label included in the Yet another cloudwatch exporter metric.

AWS namespace name

Contains the value set in the cloud_srv member of the trend data metric corresponding to SID set in the metric definition file (metrics_ya_cloudwatch_exporter.conf) of Yet another cloudwatch exporter.

jp1_pc_addon_program

The value set in jp1_pc_addon_program in Fluentd monitor definition file is entered. If no value is set, "Fluentd" is entered.

(g) Configuration information SID format

The format of the SID of the configuration information corresponding to the IM management node created by the JP1/IM - Agent is shown below.

If the URL-encoded string exceeds 255, the structured ID of the SID of the target configuration information is split. Set up to character 255 to the name of the structured ID, and set the remaining character string to the name of the lower-level structured ID.

P host name

URL encoded host name of the host where Prometheus server is set up.

E host name

A URL-encoded string of the host name of the host where the exporter is set up.

I host name

This is a URL-encoded string of the host name of the host to be monitored.

When an IP address is specified for targets in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of Blackbox exporter (ICMP monitoring), the I-host name of the remote monitoring host SID and the remote agent SID are IP addresses.

For Fluentd, this is the value specified in "instance" of the monitor definition file.

jp1_pc_nodelabel

This is a URL encoded string of the value set in the jp1_pc_nodelabel in the scrape_configs of the Prometheus configuration file (jpc_prometheus_server.yml).

In the case of EC2 host SID, CloudWatchSID of EC2, CloudWatchSID other than EC2, this is a URL encoded string of the value set in the jp1_pc_nodelabel of the AWS resource.

For Fluentd, this is URL encoded value set in "jp1_pc_nodelabel" of the monitor definition file.

jp1_pc_remote_monitor_instance

This is a URL encoded string of the string after the value ":" (colon) set in the jp1_pc_remote_monitor_instance in the discovery configuration file. If the value of the jp1_pc_remote_monitor_instance is changed in the relabel_configs, it is a URL encoded string of the changed value.

jp1_pc_nodelabel_fluentd

This is a URL encoded value set in "jp1_pc_nodelabel_fluentd" of Fluentd monitor definition file.

AWS name space name

This is the value specified for the cloud_srv member in the Yet another cloudwatch exporter metrics definition file (metrics_ya_cloudwatch_exporter.conf). Change the forward slash (/) to a hyphen (-).

The following table shows the meanings of the character strings of SID of the configuration information. (The leading "JP1PC" is fixed character string in JP1/IM - Agent)

(h) IM management node per integrated agent

IM management node that you create depends on the type of Exporter that integrated agent ships with. Whether or not IM management node can be created for each Exporter type is shown below.

(Legend)

Y: Create an IM Management Node

--: Do not create an IM admin node

#1

This is applicable when connecting directly to JP1/IM - Manager (Intelligent Integrated Management Base).

#2

Depending on the contents specified in the Prometheus configuration file (jpc_prometheus_server.yml), it may or may not be created.

(i) Tree Format

The contents displayed in the JP1/IM - Manager Integrated Operation Viewer tree depend on the type of exporter used for monitoring.

In the JP1/IM - Manager's Integrated Operation Viewer, the IM management nodes created by JP1/IM - Agent are displayed in the tree shown below.

All Systems
 + BizSystem#1
 |   + Integration Manager host name
 |   |  + Management Applications#2
 |   |     + Metric forwarder(Prometheus server)#3
 |   |     + Alert forwarder(Alertmanager)#4
 |   |     + AWS metric collector(Yet another cloudwatch exporter)#6
 |   |     + Synthetic metric collector(Blackbox exporter)#6
 |   + Monitored host name or container name
 |   |  + Management Applications#2
 |   |     + JP1/IM agent control base#7
 |   |  |  + Metric forwarder(Prometheus server)#3
 |   |  |  + Alert forwarder(Alertmanager)#4
 |   |  |  + Linux metric collector(Node exporter)#5
 |   |  |  + Windows metric collector(Windows exporter)#5
 |   |  |  + Synthetic metric collector(Blackbox exporter)#6
 |   |  |  + jp1_pc_nodelabel value#8
 |   |  |  + Log trapper(Fluentd)#9
 |   |  + Platform#2
 |   |  |  + Linux metric collector(Node exporter)#10
 |   |  |  + Windows metric collector(Windows exporter)#10
 |   |  + Categories specified in the jp1_pc_category#11
 |   |  |  + jp1_pc_nodelabel value#12
 |   |  + Categories specified in the jp1_pc_category#2
 |   |     + jp1_pc_nodelabelvalue#13
 |   + Monitored host name or IP address#14
 |   |  + Platform#2
 |   |     + Synthetic metric collector(Blackbox exporter(ICMP))#15
 |   + Host name of the host to monitor with HTTP/HTTPS#16
 |   |  + ServiceResponse#2
 |   |     + jp1_pc_nodelabel value#15
 |   + Host name of the host on AWS#17
 |      + Platform#2
 |         + AWS metric collector(Yet another cloudwatch exporter)#18
 + Amazon Simple Storage Service
     + Storage Name#19

#1

Indicates the system node. Created when the user has defined a system node.

#2

Indicates a category.

#3

Indicates the PrometheusSID.

#4

Indicates the AlertmanagerSID.

#5

Indicates the agent service SID.

#6

Indicates the Remote Monitoring Service SID.

*7

Indicates an JP1/IM agent control base SID.

#8

Indicates the Agent Service SID (your own Exporter).

#9

Indicates the log-monitoring-service SID.

#10

Indicates the agent SID.

#11

Indicates the category (user's own Exporter).

#12

Indicates the agent SID (your own Exporter).

#13

Indicates SID of the logging target.

#14

In the targets of the discovery configuration file (file_sd_config_blackbox_icmp.yml) of Blackbox exporter (ICMP monitoring), the host name is displayed when a host name is specified, and the IP address is displayed when an IP address is specified.

#15

Indicates the remote agent SID.

#16

Indicates the agent host SID.

#17

Indicates the EC2 host SID.

The label displays the string set in the AWS jp1_pc_nodelabel tag.

#18

Indicates the CloudWatchSID for EC2.

#19

Indicates a CloudWatch SID other than EC2.

For IM management nodes other than EC2, create an object root node with "JP1CS-AWS namespace name" as the object root node type.

The label displays the string set in the AWS jp1_pc_nodelabel tag.

For agent SID, remote agent SID, and log-monitored SID, create a tree SID for each monitored target. When monitoring targets by different add-on programs, this item is displayed in a single tree SID. The tree SID is the same ID. For example, if you use Blackbox exporter to monitor Web server's outline, and then use Fluentd to monitor the log of that Web server, the same tree SID will occur.

If all of the following values are the same, it is judged as the same monitoring target.

• Host name of the monitored host (name of _HOST_ in SID of the configuration)

• Label-name of IM management node

• IM management node Category ID

- About IM management node Label-Names and Category ID

Label-name and category ID are set by the user when the monitoring method is as follows: For label-name and category ID for other monitoring methods, see the tree view above.

• For HTTP/HTTPS monitoring of Blackbox exporter

  Set Label-name to targets of the discovery configuration file and set the category ID to the jp1_pc_category.

• For user-defined Exporter

  In Prometheus configuration file (jpc_prometheus_server.yml)'s scrape defnition, set the label name to jp1_pc_nodelabel and category ID to jp1_pc_category of user-specific discovery configuration file.

• For Fluentd logging monitoring

  In the [Metric Settings] section of monitoring definition file of text file or Windows event-log monitoring definition file, set the label name to jp1_pc_nodelabel and set the category ID to jp1_pc_category.

If the same tree SID exists for JP1/IM - Manager functions such as event list and trend information view, the newly created tree SID grant information is handled as shown in the following table.

(j) Viewing IM management node in Containers

When you install integrated agent, or user-defined Prometheus or Fluentd in a container, integrated operation viewer displays the container name as a hostname in a tree.

(k) Creating an IM Management Node for Yet another cloudwatch exporter

Among the metrics set in Yet another cloudwatch exporter's metric definition file (metrics_ya_cloudwatch_exporter.conf), for resources monitored by Yet another cloudwatch exporter and for which the jp1_pc_nodelabel tag is set on AWS, Create an IM management node.

The following AWS namespaces are supported by the JP1/IM - Agent's Yet another cloudwatch exporter for monitoring.

#

The categorization name that AWS CloudWatch categorizes metrics by dimension. You can check it on the CloudWatch web page.

There are two ways to specify the monitoring target in Yet another cloudwatch exporter, Auto-discovery configuration and static configuration, but in JP1/IM - Agent, only for the resources specified in the Auto-discovery configuration, You can create an IM management node. Do not create an IM management node for the entire resource or grouped metrics that can be viewed on CloudWatch.

If you delete the resource to be created for the IM management node or the jp1_pc_nodelabel tag set on the resource on AWS, the IM management node will not be created.

Also, you need to wait about 10 minutes after you finish editing the tag until the tag information is reflected in CloudWatch.

See "Yet another cloudwatch exporter configuration file (jpc_ya_cloudwatch_exporter.yml)" (2. Definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File, and API Reference" manual for information on setting Auto-discovery configuration in Yet another cloudwatch exporter configuration file (jpc_ya_cloudwatch_exporter.yml).

If you want to monitor using Yet another cloudwatch exporter, make sure that CloudWatch displays metrics for the resources you want to monitor, start the Prometheeus server and Yet another cloudwatch exporter services, and then use the Prometheus configuration file (jpc_ After the time set in item "scrape_interval" in prometheus_server.yml) (default: 1 minute) has elapsed, execute the jddcreatetree command and the jddupdatetree command (specify the configuration change mode (-c option)). After running the command, check for excess or deficiency of the IM management node of Yet another cloudwatch exporter displayed in the Integrated Operations Viewer tree.

For details on what to do if there is an excess or deficiency of the IM management node displayed, see "3.4.3 (4) IM management node of Yet another cloudwatch exporter is not created" in "Operation" of the instruction manual.

(l) About Configuring Multiple Scrape Definitions to Monitor the Same Monitored Object

Blackbox exporter and Yet another cloudwatch exporter are Exporter that remotely monitor hosts and resources that are not install destination. It does not support the configuration to monitor the same targets in several scrape definitions of the same Exporter, or to monitor the same targets by setting up the same Exporter to be monitored remotely on different hosts. If set, IM management node tree SID displayed in integrated operation viewer will be the same. Therefore, the trend data displayed in the [Trend] tab is undetermined which Exporter collected.

(m) IM management node and metric Labeling

As shown in "(c) Creation condition of IM management node other than JP1/IM agent base", create a IM management node from the trend data saved in Trend data Management Database. Trend data contains metric information. Each metric is labeled.

Labeling for metric is set in the following definition-file:

• Discovery configuration file

  Set the same labeling for each Exporter. For more information, see each Exporter discovery configuration file in the manual "JP1/Integrated Management 3 - Manager Commandm Definition File, and API Reference" (2. Definition Files).

• Prometheus configuration file

  Configure labels that cannot be set in the discovery configuration file and labels that must be changed for each metric. For more information, see "Prometheus configuration file (jpc_prometheus_server.yml)" (2. Definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File, and API Reference" manual.

(a) Prerequisites

JP1/IM - Agent creates and returns trend data from performance data stored in JP1/IM - Manager's Trend data Management Database. You cannot return trend data if you do not use integrated agent feature to store performance data in Trend data Management Database.

If JP1/IM - Manager is in a hierarchical configuration and the Integration Manager runs trend data display function, IM managed node under lower manager creates and returns trend data from performance data stored in Trend data Management Database of lower manager. Therefore, Intelligent Integrated Management Base of lower manager must be accessible. In addition, performance-data must be stored in Trend data Management Database of lower manager.

• For details about JP1/IM - Manager prerequisites for hierarchical structure, see "(1) Prerequisites" in "3.5.8 Monitoring of multiple sites".

• For details about the various data managed by JP1/IM - Manager, see " 3.6.1 Introduction" in " 3.6 Operation using WebGUI (integrated operation viewer)".

(b) Instance name (a string to display as a legend in the graph)

• If metric is set to a label other than the following name and specified in drop_legend_labels^#1 of metric definition file, convert the object with a member of that label name to JSON format, and return the string with the leading "{" and trailing "}" removed as instance name (e.g., "core":"0,0", "mode":"idle"). If a label other than the following name and specified in drop_legend_labels^#1 of metric definition file is not set, instance name is not returned.

  __name__, instance, job, jp1_pc_nodelabel, jp1_pc_prome_hostname, jp1_pc_exporter, jp1_pc_remote_monitor_instance, jp1_pc_category, jp1_pc_trendname, jp1_pc_module, jp1_pc_rm_agent_create_flag, jp1_pc_agent_create_flag, jp1_pc_multiple_node, jp1_pc_nodelabel_fluentd, jp1_pc_addon_program, account_id^#2, region^#2, name^#2

  #1

  For drop_legend_labels of metric definition file, see "Node exporter metric definition file (metrics_node_exporter.conf)" (2. definition file) in "JP1/Integrated Management 3 - Manager Command, Definition File, and API Reference" manual.

  #2

  This is only the case with Yet another cloudwatch exprorer.

• If the instance name exceeds 249 characters, the 250th and subsequent characters are deleted.

• If there are multiple strings that are the same up to the 249th character of the instance name, the term number "#n" (n=1, 2, 3...) granted.

(c) About Performance Data to Retrieve

Of the performance data stored in Trend data Management Database of JP1/IM - Manager, which performance data is returned as trend data is specified by PromQL expression set in promql of metric definition file.

For details about the default value of promql, see "promql for metric definition files (including $jp1im_TrendData_labels)" in "JP1/Integrated Management 3 - Manager Command, Definition File, and API Reference" (2. Definition Files), which describes metric definition files for each Exporter, and describes the default settings for each metric.

For PromQL expression notes, see "PromQL expression Notes" (2. Definition file) in "JP1/Integrated Management 3 - Manager Command, Definition File, and API Reference" manual.

The string "$jp 1im_TrendData_labels" contained in the promql value of the metric definition file is replaced with a PromQL statement to narrow down the acquisition target when retrieving performance data. For example, by specifying a string containing "$jp 1im_TrendData_labels" in the promql value as follows, only the performance data of the IM management node that returns trend data can be retrieved as trend data.

- String to specify for promql

(windows_memory_available_bytes and $jp1im_TrendData_labels) / (1024*1024)

- Replaced string for "$jp 1im_TrendData_labels"

(windows_memory_available_bytes and {jp1_pc_prome_hostname="Prometheus hostname",job="Scrape Job Name",instance="instance label value"}) / (1024*1024)

Also, the replaced strings of "$jp1im_TrendData_labels" depends on IM management node. For more information, see the description of "$jp1im_TrendData_labels replacement" in "Node exporter metric definition file (metrics_node_exporter.conf) (2. definition file) in "JP1/Integrated Management 3 - Manager Command, Definition File, and API Reference" manual.

If JP1/IM - Manager is in a hierarchical configuration and the Integration Manager runs trend data display function, IM managed node under lower manager must also be set in the Integration Manager metric definition file.

(d) About the maximum number of data per instance

The upper limit of the number of data per instance is passed as an argument from JP1/IM - Manager. The returned data for each instance is returned at intervals of "number of hours of collection period ÷ upper limit of the number of data per instance". The number of data returned per instance does not exceed the argument limit, and the return value "countPerInstance" for this method is always "false".

(e) About performance data time

Trend data is returned by setting a time for each performance data, but this time is not the time when performance data was collected and stored, but the time ticked at equal intervals from the start time to the end time. The time at equal intervals is calculated based on "(Time Series Data Acquisition API End Time--Time Series Data Fetch API Start Time) ÷ Upper limit of the number of data per instance". When trend data is displayed on the Trends tab of the Integrated Operation Viewer, the upper limit of the number of data per instance is 60.

- Example of trend data time displayed in the Trend tab of the integrated operation viewer

The following are the prerequisites:

• Time when Prometheus server collected trend data

  Time of collection of the first data: 2022/02/19 12:15:00

  Time of collection of the second data: 2022/02/19 12:16:00

  Time of collection of the third data: 2022/02/19 12:17:00

  :

• Time range for trend data displayed in the Trends tab of the integrated operation viewer

  Start Time: 2022/02/19 12:15:30

  End Time: 2022/02/19 13:15:30

When trend data is displayed on the Trend tab of the integrated operation viewer, the upper limit of the number of data per instance is 60, so from the assumption, the time from the start time to the end time of the display range (end time--start time) is 60 minutes, and the time at equal intervals ((end time--start time) ÷ upper limit of the number of data per instance) is 1 minute.

As a result, the time of trend data displayed on the Trend tab of the integrated operation viewer (the time set for the trend data to be returned) is the time in 1-minute increments starting from the start time of the display range, as shown below.

Time of display of the first data: 2022/02/19 12:15:30

Time of display of the second data: 2022/02/19 12:16:30

Time of display of the third data: 2022/02/19 12:17:30

:

(f) Relationship between collected performance data and trend data

In the return trend data function, performance data collected in the past 5 minutes at a certain output interval is output as trend data for performance data collected at a certain collection interval during the specified collection period. If the performance data collected in the past 5 minutes is the same as the previous output, it will be supplemented with that data and trend data will be output. If there is no performance data collected in the last 5 minutes, the data will not be imputed.

Also, if performance data cannot be collected, such as when the exporter is stopped, trend data is not output.

(g) Things to note when the trend data output period includes a future period

If the trend data output period includes a future period, the trend data output time output to the future period from the usage time (current time) of the trend data return function will be the future time. If you want to prevent the trend data output time from being in the future, do not include a future period in the trend data output period.

(h) Relationship between alerts and trend data

Since the alert rule is evaluated when Prometheus collects the performance data, the alert notification time is the collection time of the performance data that evaluated the alert. When matching trend data with alert notification time, the output time must match with the most recent trend data in the future direction.

(i) About Resetting Metric Values

There is a metric that restarting the OS resets the performance data. When trending such metrics, if the display period includes OS restarts, instantaneous jumping values (large values, small values, and minus values) may be displayed, or the graph may show extreme changes.

In such a case, check whether the OS has been restarted and ignore the data during the relevant time zone.