Hitachi

JP1 Version 13 JP1/Integrated Management 3 - Manager Configuration Guide


1.21.2 Change settings in JP1/IM - Agent

Organization of this subsection

(1) Common way for Setting

(a) Edit the configuration files (for Windows)

Configuration file is stored in conf directory. There are two ways to modify the content of configuration file:

  • The way to use integrated operation viewer

  • The way to Login and Setup the Hosts

About setting files that can be edited when you are using integrated operation viewer, refer to the notes of the definition File about JP1/IM - Agent (JP1/IM - Agent control base) in "List of Definition File" (2. Definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference". If you Login host and Setup, All configuration files can be edited.

■ How to use integrated operation viewer

  1. Download configuration file from integrated operation viewer.

    Select File you want to edit from integrated operation viewer and download it.

    If you want to Add a defined File that you can optionally create, do the following:

    1. Download user-created definition file list definition file.

    2. Write information about definition File you want to Add in User-created definition file list definition file

    3. Upload user-created definition file list definition file.

    4. Upload definition File that you want to Add.

  2. Edit the downloaded File.

    Note

    Because Format check can be done for Prometheus server defined File with promtool command, it is recommended to be checked at this point.

    Promtool command is included with Prometheus server. Prometheus server can be downloaded from GitHub website. Use the same Version as Prometheus server that came with your JP1/IM - Agent.

    Version of add-on program of JP1/IM - Agent can be checked in the add-on function list in the [integrated agent management] window of integrated operation viewer or in the addon_info.txt File stored in "Agent-path\addon_management\add-on name\".

  3. Upload the edited File with integrated operation viewer.

    Setup is automatically reflected when uploaded.

■ How to Login and Setup the Hosts

  1. Login to integrated agent host.

  2. Stop JP1/IM - Agent servicing.

  3. Edit the configuration files.

    Note

    Because Format check can be done for Prometheus server defined File with promtool command, it is recommended to be checked at this point.

  4. Start JP1/IM - Agent service.

(b) Changing service definition file (for Windows)

Service definition file storage destination and File name are as follows:

  • Storage destination: install destination folder\jp1ima\bin\

  • File name: jpc_Service name _service.xml

    Important
    • If you make changes to service definition file items, you will need to Restart the service or reinstall the service# to Apply the changes. For details about what you need to do to import each items, see "When to Import Definitions" in "service definition file (jpc_program-name_service.xml)"(2. definition File) in the manual JP1/Integrated Management 3 - Manager Command, Definition File and API Reference.

    • If you change any of the items that require service reinstallation #, you must disable registration of the service and Enable again after that. For details about how to Enable or disable registration of service, see 1.21.1(1) Enable or disable add-on program.

    #

    Reinstalling a service means that you delete the service and then create the service again (using the jpc_service command.).

To change service definition file, follow these steps:

  1. Login to integrated agent host.

  2. Stop JP1/IM - Agent service.

  3. Edit service definition file.

  4. Start JP1/IM - Agent service.

(c) Change command-line options (for Windows)

Change the command-line options in service definition file <arguments> tag.

For how to edit, see (b) Changing service definition file (for Windows).

(2) Setup for JP1/IM - Agent control base

(a) Change Integrated manager to connect to (for Windows) (optional)

  1. Stop JP1/IM - Agent service.

  2. Change Integrated manager to connect to.

    Change the destination Integration Manager defined in imagent Common configuration file (jpc_imagentcommon.json) to the new destination.

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  3. Check initial secret.

    Check initial secret (secret for first-time connection) with integrated operation viewer in Integrated manager hosting.

    For details, see 2.2.3 [Displaying initial secret] window in the manual JP1/Integrated Management 3 - Manager GUI Reference.

  4. Obfuscate and register initial secret.

    In the Secret Manager command, obfuscate initial secret and register it.

    Jimasecret -add -key immgr.initial_secret -s " initial secret "
  5. Delete Individual secret.

    In the Secret Management command, Delete Individual secrets.

    Jimasecret -rm -key immgr.client_secret
  6. Modify a certificate

    For details on how to change CA certificate, see 1.21.2(2)(c) Place CA certificate (for Windows) (optional).

    This step is not required if authentication station that issued the server certificate for Integrated manager to which the old connection was made and imbase to which the new connection was made are the same.

  7. Start JP1/IM - Agent.

(b) Change the port (for Windows) (optional)

The listen port that JP1/IM - Agent control base uses is specified in imagent configuration file (jpc_imagent.json) and imagentproxy configuration file (jpc_imagent_proxy.json).

For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

For information about the default Port number, see "Port number used by integrated agent for communications" in "JP1/Integrated Management 3 - Manager Overview and System Design Guide".

If you change the port of imagentproxy process, you must change Remote Write destination of Prometheus, the alert notification destination of Alertmanager, and the alert notification destination of Fluentd. For Details for each change, see below.

(c) Place CA certificate (for Windows) (optional)

This Setup is required to encrypt communication between JP1/IM - Agent management base and JP1/IM - Agent control base. If you do not want to encrypt, this Setup is not required.

For instructions on deploying a CA certificate, see 9.4.5 Settings for JP1/IM - Agent (JP1/IM agent control base).

■ To verify the server certificate of JP1/IM - Agent management base

  1. Place CA certificate.

    Place CA certificate of authentication station that issued the server certificate of imbase you are connecting to in the following directory:

    • In Windows

      Agent-path\conf\user\cert\

    • In Linux

      /opt/jp1ima/conf/user/cert/

  2. Provide CA certificate path in imagent Common configuration file (jpc_imagentcommon.json).

  3. Restart imagent and imagentproxy.

■ Not to verify the server certificate of JP1/IM - Agent management base

  1. Set "true" in the tls_config.insecure_skip_verify of imagent shared configuration file (jpc_imagentcommon.json) tls_config.insecure_skip_verify.

(d) Modify settings related to Action Execution (for Windows) (optional)

Setup for Action Execution is defined in imagent configuration file (jpc_imagent.json).

For details about how to set, see 1.21.2(1)(a) Edit the configuration files (for Windows).

(e) Setup the proxy authentication's authentication ID and Password (for Windows) (optional)

If there is a proxy server between agent host and manager host that requires Basic authentication, authentication ID and Password must be Setup.

Set authentication ID to the immgr.proxy_user of imagent shared configuration file (jpc_imagentcommon.json). For details about Setting of each definition files, see 1.21.2(1)(a) Edit the configuration files (for Windows).

You set Password in the following ways: For details, refer to the explanation for each item.

  • Secret management command

    For more information, see jimasecret (1. command) in the manual JP1/Integrated Management 3 - Manager Command, Definition File and API Reference.

  • [Manage Secrets] dialogue of integrated operation viewer

    For more information, see 2.2.2(4) [Manage Secrets] Dialog in the manual JP1/Integrated Management 3 - Manager GUI Reference.

  • Integrated operation viewer Secret Management REST API

    For more information, see 5.4.3 Issuing initial secret in the manual JP1/Integrated Management 3 - Manager Command, Definition File and API Reference.

(3) Setup of Prometheus server

(a) Changing Ports (For Windows) (Optional)

The listen port used by Prometheus server is specified in --web.listen-address option of prometheus command.

For information about changing prometheus command options, see 1.21.2(1)(c) Change command-line options (for Windows). For more information about--web.listen-address options, see prometheus Command Options in service definition file (jpc_program-name_service.xml) (2. definition file) in the manual JP1/Integrated Management 3 - Manager Command, Definition File and API Reference.

The default port is "20713". If Port number is changed, review Setup of the firewall and prohibit accessing from outside. However, if you want to monitor Prometheus server with external shape monitoring by Blackbox exporter in other host, allow it to be accessed. In such cases, consider security measures such as limiting the source IP address as required.

(b) To Add the alert definition (for Windows) (optional)

Alert definitions are defined in alert configuration file (jpc_alerting_rules.yml).

For details on how to edit alert configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

For information about the items that require Setup in the alert definition, see Alert Rule Definitions for Converting to JP1 Events in (a) Alert Assessment Feature of 9.5.3(4) Performance data monitoring notification function in the manual JP1/Integrated Management 3 - Manager Overview and System Design Guide. For Details of the individual items, refer to alert configuration file (jpc_alerting_rules.yml) (2. Definition File) in the manual JP1/Integrated Management 3 - Manager Command, Definition File and API Reference.

For information about sample of the alert-definition, see settings(the default Status) of model file in following definition Files.

  • Node exporter metric definition file (metrics_node_exporter.conf)

  • Windows exporter metric definition file (metrics_windows_exporter.conf)

  • Blackbox exporter metric definition file (metrics_blackbox_exporter.conf)

  • Yet another cloudwatch exporter metric definition file (metrics_ya_cloudwatch_exporter.conf)

    Important

    The following are the points for when you create an alert definition:

    • Monitoring performance function on JP1/IM - Agent allows you to specify a duration (for). If the alert condition is met continuously during the specified period, it is judged as an error.

    • If you want to detect that metric is present, use absent() function in the alerting criteria.

      Absent (metric {label})

    • If you want the alert to be enabled for a certain duration, use PromQL for the alert criteria to Setup it.

      (Example) When monitoring from 8 o'clock to 12 o'clock in Japan time

      Alert-condition and ON() (23 <= hour() or 0 <= hour() < 3)

      Note that "hour()" returns UTC time, so you need to consider UTC.

    • Monitoring performance function on JP1/IM - Agent notifies you of the abnormal and recovery. If you want to be notified in two stages: Warning and abnormal, create alerts for Warning and alerts for abnormal.

    • Message that is displayed when an alert occurs can include the following:

      - Message at abnormal

      - Message on Recovery

    • For information about the variables that can be embedded in alert Message, see 9.5.3(4)(a) Alert Assessment Feature in the manual JP1/Integrated Management 3 - Manager Overview and System Design Guide.

(c) Add Blackbox exporter scrape job (for Windows) (optional)

Prior to Add a Blackbox exporter scrape job, you must add the module to configuration file on Blackbox exporter. For details, see 1.21.2(6)(b) Add, change, and delete modules (for Windows) (optional).

After you Add the module, perform the following steps to Setup a scrape job that scrape the newly created module:

  1. Create a discovery configuration file for your Blackbox exporter.

    Copy the original model File shown below and rename it to the definition File of Copy destination to create a discovery configuration file for Blackbox exporter.

    - When performing HTTP/HTTPS monitoring

    • For Windows:

      Copy source: Agent-path\conf\jpc_file_sd_config_blackbox_http.yml.model

      Copy to: Agent-path\conf\modules starting with file_sd_config_blackbox_http

    • For Linux:

      Copy source: /opt/jp1ima/conf/jpc_file_sd_config_blackbox_http.yml.model

      Copy to: /opt/jp1ima/conf/file_sd_config_blackbox name begins with http.yml

    - When performing ICMP monitoring

    • For Windows:

      Copy source: Agent-path\conf\jpc_file_sd_config_blackbox_icmp.yml.model

      Copy to: Agent-path\conf\file_sd_config_blackbox_module name begins with icmp.yml

    • For Linux:

      Copy source: /opt/jp1ima/conf/jpc_file_sd_config_blackbox_icmp.yml.model

      Copy to: /opt/jp1ima/conf/file_sd_config_blackbox name begins with icmp.yml

    The module name indicates the module that was added by 1.21.2(6)(b) Add, change, and delete modules (for Windows) (optional).

  2. Edit the discovery configuration files in Blackbox exporter.

    • For monitoring HTTP/HTTPS Discovery configuration file

    For descriptions, see Blackbox exporter (HTTP/HTTPS monitoring) discovery configuration file (jpc_file_sd_config_blackbox_http.yml) (2. Definition File) in the manual JP1/Integrated Management 3 - Manager Command, Definition File and API Reference.

    • For monitoring ICMP Discovery configuration file

    For descriptions, see "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference", "Blackbox exporter (ICMP monitoring) discovery configuration file (jpc_file_sd_config_blackbox_icmp.yml)" (2. Definition File).

  3. Use integrated operation viewer to add definition File.

    For instructions on how to add a definition File, see 1.21.2(1)(a) Edit the configuration files (for Windows).

  4. Add a scrape job to Prometheus configuration file.

    • When performing HTTP/HTTPS monitoring

    In Prometheus configuration file (jpc_prometheus_server.yml), Copy the Definition of Scrape Job with Job Name "jpc_blackbox_http "to add a new scrape job.

    • When performing ICMP monitoring

    In Prometheus configuration file (jpc_prometheus_server.yml), Copy definition of Scrape Job with Job Name "jpc_blackbox_icmp" to Add a new scrape job.

    <Sample Setup>

    scrape_configs:
      - job_name: Any scrape job name
        metrics_path: /probe
        params:
          module: [module-name]
        file_sd_configs:
          - files:
            - 'Discovery configuration file Name'
        relabel_configs:
          (Omitted)
        metric_relabel_configs:
          (Omitted)
    Any scrape job name

    Specify any name that does not overlap with any other scrape job name, in the range of 1 to 255 characters, except for control characters.

    Module name

    Specify the module name that was added in 1.21.2(6)(b) Add, change, and delete modules (for Windows) (optional).

    Discovery configuration file Names

    Specify File that you created in step 1.

    For descriptions of Prometheus configuration file, see "Prometheus configuration file (jpc_prometheus_server.yml)" (2. Definition File) <scrape_config> in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual.

    For details about editing Prometheus configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

(d) Add user-defined Exporter scrape job (for Windows) (optional)

To scrape user-defined Exporter, you need the following Setup:

  • Add for user-specific discovery configuration file

  • Editing Prometheus configuration file (jpc_prometheus_server.yml)

For details about how to Add and edit File each definition files, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  1. Add user-specific discovery configuration file.

    Specify user-defined Exporter that you want to scrape to user-specific discovery configuration file.

    For descriptions, see "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference", "user-specific discovery configuration file (file_sd_config_ Any Name.yml)" (2. Definition File).

  2. Add a scrape job to Prometheus configuration file.

    In Prometheus configuration file (file_sd_config_ any name.yml), add the scrape job to scrape user-defined Exporter.

    Scrape jobs are listed in scrape_configs.

    <Sample Setup>

    scrape_configs:
      - job_name: Scrape job name
     
        file_sd_configs:
          - files:
            - Discovery configuration file Names
     
        relabel_configs:
          - target_label: jp1_pc_nodelabel
            replacement: Linux metric collector(Node exporter)
     
        metric_relabel_configs:
          - source_labels: ['__name__']
            regex: ' metric 1| Metric 2| Metric 3'
            action: 'keep'
    Scrape job name

    Specify an arbitrary string. This Value is Setup on job label of metric.

    Discovery configuration file Names

    Specify File of user-specific discovery configuration file created in step 1 above.

    Metric 1, metric 2, metric 3

    Specify metric that you want to collect. If there is more than one metric to be collected, separate them with |.

    If you want to collect all metrics, you do not need to include "metric_relabel_configs". However, if a large amount of metric is present, the amount of data will be large. Therefore, we recommend that you list "metric_relabel_configs" and limit it to metric to be monitored.

  3. Add metric Definition file.

    Add metric Definition file for user-defined Exporter.

    For descriptions, see "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference", "user-specific metric definition file (metrics_any Prometheus TrendName.conf)"(2. definition File).

(e) Changing Remote Write destination (for Windows) (optional)

Specifies URL and ports of imagentproxy processes running on the same host in the remote_write.url of Prometheus configuration file (jpc_prometheus_server.yml) for Remote Write destination. You need to change it only if you want to change imagentproxy process port.

<Sample Setup>

remote_write:
- url: http://localhost:20727/ima/api/v1/proxy/service/promscale/api/v1/write

For instructions on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

(4) Setup of Alertmanager

(a) Changing Ports (For Windows) (Optional)

The listen port used by Alertmanager is specified in --web.listen-address option of alertmanager command.

For details about how to change alertmanager command options, see "1.21.2(1)(c) Change command-line options (for Windows)". For Details of --web.listen-address option, refer to "alertmanager command options" in "service definition file (jpc_program-name_service.xml)"(2. definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual.

The default port is "20714". If Port number is changed, review Setup of the firewall and prohibit accessing from outside. However, if you want to monitor Alertmanager with external shape monitoring by Blackbox exporter in other host, allow it to be accessed. In such cases, consider security measures such as limiting the source IP address as required.

(b) Changing the alert notification destination (for Windows) (optional)

To specify Alert destinations, write the URL and port of imagentproxy processes running on the same host to recieivers.webhook_config.url in Alertmanager configuration file (jpc_alertmanager.yml). You need to change it only if you want to change imagentproxy process port.

For instructions on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

(c) Setup silence (for Windows) (optional)

Execute the command from JP1/IM - Manager to the host where Alertmanager whose silence you want to Setup is running. Use curl command to call REST API that Setup silence.

For REST API on how to Setup silence, see "5.21.4 Creation silence for Alertmanager" in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual.

Settings in silence to be specified in Message body of the request is passed in curl command argument.

(5) Setup of Windows exporter

(a) Change Port (Optional)

The listen port used by Windows exporter is specified in --telemetry.addr option of the windows_exporter command.

For details about how to change windows_exporter command options, see "1.21.2(1)(c) Change command-line options (for Windows)". For details about --telemetry.addr option, see "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual, "windows_exporter Command Options" in "service definition file (jpc_program-name_service.xml)"(2. definition File).

The default port is "20717". If Port number is changed, review Setup of the firewall and prohibit accessing from outside.

(b) Modify metric to Collect (Optional)

  1. Add metric to Prometheus configuration file.

    In the metric_relabel_configs of Prometheus configuration file (jpc_prometheus_server.yml), metric to be collected are defined separated by "|". Delete metric that you do not need to collect and Add metric that you want to collect.

    For instructions on updating configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

    <Sample Setup>

      - job_name: 'jpc_windows'
        :
        metric_relabel_configs:
          - source_labels: ['__name__']
            regex: 'windows_cache_copy_read_hits_total| Windows_cache_copy_reads_total| Windows_cpu_time_total| Windows_logical_disk_free_bytes| Windows_logical_disk_idle_seconds_total| Windows_logical_disk_read_bytes_total|....|windows_net_packets_sent_total| Windows_net_packets_received_total| Windows_system_context_switches_total| Windows_system_processor_queue_length| Windows_system_system_calls_total [Add metric here]'
  2. If required, define a trend view in metric Definition file.

    In Windows exporter metric definition file, you define a trend view.

    For descriptions, see "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference", "Windows exporter metric definition file (metrics_windows_exporter.conf" (2. Definition File).

(6) Setup of Blackbox exporter

(a) Changing Ports (For Windows) (Optional)

The listen port used by Blackbox exporter is specified in --web.listen-address option of the blackbox_exporter command.

For details about how to change blackbox_exporter command options, see "1.21.2(1)(c) Change command-line options (for Windows)". For details about --web.listen-address option, see "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference", "blackbox_exporter Command Options" in "service definition file (jpc_program-name_service.xml)" (2. definition File).

The default port is "20715". If Port number is changed, review Setup of the firewall and prohibit accessing from outside.

(b) Add, change, and delete modules (for Windows) (optional)

For each target (host or URL), it must be defined monitoring methods, such as protocols and authentication data in Blackbox exporter.

The following modules are defined in the default Setup:

Table 1‒11: Modules Defined in the Initial Setup

Module name

Feature

http

  • Monitor http/https.

  • The method is "GET" and the headers are not Setup.

  • Client authentication, Server authentication, and HTTP authentication (Basic authentication) are not performed.

  • When http/https's URL is accessed and a status code in 200-299 is returned, 1 is Setup to the probe_success (metric).

  • If communication to URL is not possible or if the status code is not in 200-299, 0 is Setup to metric.

  • If the target is redirected, it depends on the status code of the redirected target.

icmp

  • Monitor icmp.

  • Authentication is not performed.

  • If icmp communication can be performed for the host or IP address to be monitored, 1 is Setup to metric. If communication is not possible, 0 is setup.

If monitoring is possible with the module of the default Setup, there is no need to define a new one. If there are requirements that cannot be monitored by the module in the initial Setup, as shown below, the module definition must be added.

  • When authentication is required

  • To change the judgment based on the content of the response

Modules are defined in Blackbox exporter configuration file. For descriptions, see "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference", "Blackbox exporter configuration file (jpc_blackbox_exporter.yml) (2. Definition File), in the manual.

The following shows description rules:

  • Setup the new module-name as follows:

    • When performing HTTP/HTTPS monitoring

      Setup the name starting with http.

    • For ICMP monitoring

      Setup the name starting with icmp.

  • If you are creating a client-side authentication, a Server authentication, or a HTTP authentication (Basic authentication) module, you will need a certificate and a Setup of Password.

    Refer to the list of files/directories in the Appendix A.4 JP1/IM-Agent in the manual JP1/Integrated Management 3 - Manager Overview and System Design Guide for the location of the certificate.

    For details about HTTP authentication (Basic authentication) Password's Setting method, see 1.21.2(6)(e) Setup the proxy authentication ID and Password (for Windows) (Optional) and 1.21.2(6)(f) Setup authentication ID, Password, and Bearer tokens for accessing the monitored Web Server (for Windows) (optional).

    Table 1‒12: Monitoring requirements and required Setup

    Monitoring conditions

    Required File

    Required Setup

    Server authentication

    Place CA certificate of authentication station that issued the server certificate of the target in Agent-path\conf\user/cert.

    Setup the contents below to tls_config of Blackbox exporter configuration file.

    • Setup ca_file for CA certificate path

    • Setup false to the insecure_skip_verify

    No server authentication

    None.

    Setup the contents below to the tls_config of Blackbox exporter configuration file.

    • Setup true to insecure_skip_verify

    Client authentication

    • Place the client certificate in Agent-path\conf\cert.

    • Place the client certificate key File in Agent-path\conf\user\secret.

    Setup the contents below to tls_config of Blackbox exporter configuration file.

    • Setup the client certificate path to cert_file

    • Setup the client certificate key File to key_file

    No client authentication

    None.

    None.

    Basic authentication

    None.

    Setup the contents below to basic_auth of Blackbox exporter configuration file.

    • Setup User name used for Basic authentication in username

    For details about Basic authentication's Password's Setup, see 1.21.2(6)(f) Setup authentication ID, Password, and Bearer tokens for accessing the monitored Web Server (for Windows) (optional).

For instructions on updating Blackbox exporter configuration file and deploying the certificate File, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

If it needs to from Blackbox exporter to access the monitored Web Server through a proxy server, the proxy server's Setup is required. See 1.21.2(6)(d) Monitoring HTTP through proxy (for Windows) (optional)".

If you Add the module definition, you will need to define a scrape job to scrape with the newly created module from Prometheus server. For details about Setup on Prometheus server, see "1.21.2(3)(c) Add Blackbox exporter scrape job (for Windows) (optional)".

(c) Add, change, or Delete the monitoring target (for Windows) (mandatory)

Monitoring targets of Blackbox exporter are listed in definition file in the following tables.

After you Add the targets, you must refresh IM management node tree. For details, see "1.21.2(10) Creation and import of IM management node tree data (for Windows) (mandatory)".

  • Blackbox exporter (HTTP/HTTPS monitoring) discovery configuration file

    Item

    Description

    File Name

    • Jpc_file_sd_config_blackbox_http.yml

    • file_sd_config_blackbox_module name begins http.yml

    Setup target

    Define the monitoring target of HTTP/HTTPS.

    Format

    See "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference", "Blackbox exporter (HTTP/HTTPS monitoring) discovery configuration file (jpc_file_sd_config_blackbox_http.yml)" (2. Definition File).

    Update procedure

    See "1.21.2(1)(a) Edit the configuration files (for Windows)".

  • Blackbox exporter (ICMP monitoring) discovery configuration file

    Item

    Description

    File Name

    • Jpc_file_sd_config_blackbox_icmp.yml

    • file_sd_config_blackbox_module name begins with icmp.yml

    Setup target

    Define the monitoring target of ICMP.

    Format

    See "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference", "Blackbox exporter (ICMP monitoring) discovery configuration file (jpc_file_sd_config_blackbox_icmp.yml)" (2. Definition File).

    Update procedure

    See "1.21.2(1)(a) Edit the configuration files (for Windows)".

(d) Monitoring HTTP through proxy (for Windows) (optional)

Setup "proxy_url" to Blackbox exporter configuration file (jpc_blackbox_exporter.yml).

For information on Blackbox exporter configuration file, see "Blackbox exporter configuration file (jpc_blackbox_exporter.yml)" in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference".

For details on updating Blackbox exporter configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

Note that authentication ID and Password must be Setup when authentication is routed through the required proxies. For Setting method of authentication ID and Password, see "1.21.2(6)(e) Setup the proxy authentication ID and Password (for Windows) (Optional)".

(e) Setup the proxy authentication ID and Password (for Windows) (Optional)

When performing HTTP/HTTPS monitoring, if there is a proxy server that requires a Basic authentication between Blackbox exporter and the monitored Web Server, authentication ID and Password must be Setup.

Authentication ID is specified in "modules. module-name. http.proxy_user" of Blackbox exporter configuration file (jpc_blackbox_exporter.yml). For details about Setting method of Blackbox exporter configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

Set Password in the following ways: For details, refer to the explanation for each item.

  • Secret management command

  • [Manage Secrets] dialogue of integrated operation viewer

  • REST API of Secret Management of Integrated operation viewer

(f) Setup authentication ID, Password, and Bearer tokens for accessing the monitored Web Server (for Windows) (optional)

When you perform HTTP/HTTPS monitoring, you must Setup authentication ID, Password, and Bearer tokens if Basic authentication is required for accessing the monitored Web Server.

Authentication ID is specified in "modules. module-name. http.basic_auth".username" of Blackbox exporter configuration file (jpc_blackbox_exporter.yml). For details about Setting method of Blackbox exporter configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

Password and Bearer tokens are Setup in the following ways: For details, refer to the explanation for each item.

  • Secret management command

  • [Manage Secrets] dialogue of integrated operation viewer

  • REST API of Secret Management of Integrated operation viewer

(7) Setup of Fluentd

(a) Changing Setup of Common Definition file for Log Monitor (For Windows) (Optional)

If you want to change the following Setup, change Setup of log monitoring common definition file:

  • Integrated agent Control Infrastructure Port number

  • Buffer Plug-In Setup

For log monitoring common definition file information, see "log monitoring common definition file" in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference (jpc_fluentd_common.conf)" (2. Definition File).

For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

(b) Monitoring the text-format logging File (Windows) (optional)

If you want to monitor a new text-format logging File, perform the following steps:

  1. Create a text-formatted log file monitoring definition file.

    Create a text-formatted log file monitoring definition file by Copying the original template shown below and renaming it to File that you want to Copy.

    Copy source: Agent-path\conf \fluentd_@@trapname@@_tail.conf.template

    Copy destination: Agent-path\conf\user\fluentd_Log file trap name _tail.conf

    Copy the template (fluentd_@@trapname@@_tail.conf.template) to create text-formatted log file monitoring definition file. Rename the copy destination file to "fluentd_Log file trap name _tail.conf".

    For descriptions of text-formatted log file monitoring definition file, see "text-formatted log file monitoring definition file (fluentd_@@trapname@@_tail.conf.template)" (2. definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference".

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  2. Edit the Log Monitor Target Definition File (jpc_fluentd_common_list.conf).

    If you want to temporarily stop the logging monitoring of some monitoring definitions File, define by enumerating monitoring definition Files in the log monitoring target definition File.

    For details about the log monitoring target definition File, see "the log monitoring target definition File (jpc_fluentd_common_list.conf)" (2. Definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual. If the log monitoring target definition File is not being edited, no editing is required.

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  3. Apply in integrated operation viewer tree.

    For details about Application method, see "1.21.2(10) Creation and import of IM management node tree data (for Windows) (mandatory)".

    Note

    If you change the monitoring Setup of the textual log File when, for example, the Log file trap name of the monitoring definition File is changed, perform steps 2 and 3 above.

(c) Modifying the Monitoring Setup of the Text-Format Logging File (Windows) (Optional)

If you want to change the monitoring Setup for a textual logging File, perform the following steps:

  1. Change text-formatted log file monitoring definition file.

    Modify the created monitor definition File (fluentd_log file trap name _tail.conf).

    For descriptions of text-formatted log file monitoring definition file, see "text-formatted log file monitoring definition file (fluentd_@@trapname@@_tail.conf.template)" (2. definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" documentation.

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  2. Edit the Log Monitor Target Definition File (jpc_fluentd_common_list)

    In the Log Monitor Target Definition File, define by listing Files of the monitor definition File:

    • When the Log file trap name of the monitoring definition File is changed

    • If you are performing operations that temporarily stop logging for some monitoring definition file

    For details about the log monitoring target definition File, see "the log monitoring target definition File (jpc_fluentd_common_list.conf" (2. Definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual. If the log monitoring target definition File is not being edited, no editing is required.

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  3. Reflect in integrated operation viewer tree.

    If Value in the [Metric Settings] section is changed, the changes are reflected in integrated operation viewer tree. For details about reflection method, see "1.21.2(10) Creation and import of IM management node tree data (for Windows) (mandatory)".

(d) Delete Monitoring File in text-format settings (for Windows) (optional)

To Delete monitoring settings in text-format logfile, perform the following steps:

  1. Delete text-formatted log file monitoring definition file.

    Delete the created monitor definition File (fluentd_log_file trap name _tail.conf).

    For details about how to Delete configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  2. Edit the Log Monitor Target Definition File (jpc_fluentd_common_list.conf)

    To temporarily stop log monitoring for some monitor definitions File, Delete file name of monitoring definition file defined in the log monitor target definition file.

  3. Reflect in integrated operation viewer tree.

    For details about reflection method, see "1.21.2(10) Creation and import of IM management node tree data (for Windows) (mandatory)".

(e) Monitor Windows Event Log (Optional)

To monitor a new Windows event log, perform the following steps:

  1. Create a Windows event-log monitoring definition file.

    Create a Windows event log monitoring definition file by copying the following source template and renaming it to the destination definition file:

    Copy source: Agent-path\conf\fluentd_@@trapname@@_wevt.conf.template

    Copy to: Agent-path\conf\user\fluentd_log monitor name_wevt.conf

    Copy the template (fluentd_@@trapname@@_wevt.conf.template) to create Windows event-log monitoring definition file. Rename File of copy destination to "fluentd_log file trap name _wevt.conf".

    For descriptions of Windows event-log monitoring definition file, refer to "Windows event-log monitoring definition file (fluentd_@@trapname@@_wevt.conf.template)"(2. definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual.

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  2. Edit Windows event-log monitoring definition file.

    If you want to temporarily stop the logging monitoring of some monitoring definitions File, you must define by listing filenames of the monitoring definition files in Windows event-log monitoring definition file.

    For details about the log monitoring target definition File, see "log monitoring target definition File (jpc_fluentd_common_list.conf)" (2. Definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual. If the log monitoring target definition File is not being edited, no editing is required.

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  3. Reflect in integrated operation viewer tree.

    For details about reflection method, see "1.21.2(10) Creation and import of IM management node tree data (for Windows) (mandatory)".

    Note

    If you change the monitoring Setup of the textual log File when, for example, the Log file trap name of the monitoring definition File is changed, perform steps 2 and 3 above.

(f) Modify the Monitor Setup for Windows Event Log (Optional)

If you want to change monitoring settings of Windows Event Log, perform the following steps:

  1. Change Windows event-log monitoring definition file.

    Change the monitoring-definition File (fluentd_log file trap name _wevt.conf) that has been created.

    For descriptions of Windows event-log monitoring definition file, refer to "Windows event-log monitoring definition file (fluentd_@@trapname@@_wevt.conf.template)"(2. definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual.

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  2. Edit the Log Monitor Target Definition File (jpc_fluentd_common_list.conf)

    Define by listing filenames of monitoring definition files in log monitor target definition file in the following condition:

    • When the log file trap name of the monitor-definition File is changed

    • If you are performing operations that temporarily stop monitoring logs for some monitor-difinition file

    For details about the log monitoring target definition File, see "log monitoring target definition File (jpc_fluentd_common_list)" (2. Definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual. If the log monitoring target definition File is not being edited, no editing is required.

    For details on how to change configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  3. Apply in integrated operation viewer tree.

    If Value in the [Metric Settings] section is changed, the changes are reflected in integrated operation viewer tree. For details about reflection method, see "1.21.2(10) Creation and import of IM management node tree data (for Windows) (mandatory)".

(g) Delete Monitoring settings of Windows Event Logs (Optional)

To delete Monitoring settings of Windows Event Logs, perform the following steps:

  1. Delete Windows event-log monitoring definition file.

    Delete the monitoring-definition File (fluentd_log file trap name _wevt.conf) that has been created.

    For descriptions of Windows event-log monitoring definition file, refer to "Windows event-log monitoring definition file (fluentd_@@trapname@@_wevt.conf.template)"(2. definition File) in "JP1/Integrated Management 3 - Manager Command, Definition File and API Reference" manual.

    For details about how to Delete configuration file, see "1.21.2(1)(a) Edit the configuration files (for Windows)".

  2. Edit the Log Monitor Target Definition File (jpc_fluentd_common_list)

    To temporarily stop log monitoring for some monitor definitions File, Delete the filenames of monitoring definition files defined in log monitoring target definition file.

  3. Apply in integrated operation viewer tree.

    For details about reflection method, see "1.21.2(10) Creation and import of IM management node tree data (for Windows) (mandatory)".

(8) Firewall's Setup (for Windows) (mandatory)

You must Setup the firewall to restrict external accessibility as follows:

Table 1‒13: Firewall Setup

Port

Firewall Setup

Imagent port

Access from outside is prohibited.

Imagentproxy port

Access from outside is prohibited.

Imagentaction port

Access from outside is prohibited.

Alertmanager port

Access from outside is prohibited.

However, if you want to monitor Alertmanager with external shape monitoring by Blackbox exporter in other host, allow it to be accessed. In this case, consider security measures such as limiting source IP address.

Prometheus_server port

Access from outside is prohibited.

However, if you want to monitor Prometheus server with external shape monitoring by Blackbox exporter in other host, allow it to be accessed. In this case, consider security measures such as limiting source IP address.

Node_exporter port

Access from outside is prohibited.

Windows_exporter port

Access from outside is prohibited.

Blackbox_exporter port

Access from outside is prohibited.

Ya_cloudwatch_exporter port

Access from outside is prohibited.

Fluentd port

Access from outside is prohibited.

Service of add-on program for IM Exporter

See "12.4 Configuring your firewall for IM Exporter" in the manual "JP1/Integrated Management 3 - Manager Overview and System Design Guide".

(9) Setup of integrated agent process alive monitoring (for Windows) (optional)

You monitor integrated agent processes in the following ways:

(a) External shape monitoring by other-host Blackbox exporter

Prometheus server and Alertmanager services monitors from Blackbox exporter of integrated agent running on other hosts. The following tables show URL to be monitored.

For details about how to add HTTP monitor of Blackbox exporter, see "1.21.2(6)(c) Add, change, or Delete the monitoring target (for Windows) (mandatory)". For details about Setting method of the alert definition, see "1.21.2(3)(b) To Add the alert definition (for Windows) (optional)".

Table 1‒14: URL monitored by HTTP monitoring of Blackbox exporter

Service

URL to monitor

Prometheus server

http:// Host name of integrated agent: Port number of Prometheus server /-/healthy

Alertmanager

http:// Host name of integrated agent: Port number of Alertmanager /-/healthy

The following is a sample alert-definition that you want to monitor with HTTP Monitor for Blackbox exporter.

groups:
  - name: service_healthcheck
    rules:
    - alert: jp1_pc_prometheus_healthcheck
      expr: probe_success{instance=~".*:20713/-/healthy"} == 0
      for: 3m
      labels:
        jp1_pc_product_name: "/HITACHI/JP1/JPCCS2"
        jp1_pc_component: "/HITACHI/JP1/JPCCS/CONFINFO"
        jp1_pc_severity: "Error"
        jp1_pc_eventid: "00007600"
        jp1_pc_metricname: "probe_success"
      annotations:
        jp1_pc_firing_description: "Communication to Prometheus server failed. "
        jp1_pc_resolved_description: "Communication to Prometheus server was successful. "
    - alert: jp1_pc_alertmanager_healthcheck
      expr: probe_success{instance=~".*:20714/-/healthy"} == 0
      for: 3m
      labels:
        jp1_pc_product_name: "/HITACHI/JP1/JPCCS2"
        jp1_pc_component: "/HITACHI/JP1/JPCCS/CONFINFO"
        jp1_pc_severity: "Error"
        jp1_pc_eventid: "00007600"
        jp1_pc_metricname: "probe_success"
      annotations:
        jp1_pc_firing_description: "Communication to Alertmanager failed. "
        jp1_pc_resolved_description: "Communication to Alertmanager was successful. "

(b) Alive Monitoring Processes by Windows exporter

Imagentproxy service, imagentaction service, Fluentd service, and Windows servicing program are monitored by Windows exporter's Process Monitor activity information. The processes to be monitored are described in the following table.

For details about Setting method of the alert definition, see "1.21.2(3)(b) To Add the alert definition (for Windows) (optional)".

Table 1‒15: Processes monitored by the Windows exporter

Service

Processes to monitor

Monitored Name

jpc_imagent_service#

Agent-path\bin\jpc_imagent_service.exe

Monitoring target 1:imagent

Monitoring target 2:imagent

jpc_imagentproxy_service#

Agent-path\bin\jpc_imagentproxy_service.exe

Monitoring target 1:imagentproxy

Monitoring target 2:imagentproxy

jpc_imagentaction_service#

Agent-path\bin\jpc_imagentaction_service.exe

Monitoring target 1:imagentaction

Monitoring target 2:imagentaction

jpc_prometheus_server_service#

Agent-path\bin\jpc_prometheus_server_service.exe

Monitoring target 1:prometheus

Monitoring target 2:prometheus

jpc_alertmanager_service#

Agent-path\bin\jpc_alertmanager_service.exe

Monitoring target 1:alertmanager

Monitoring target 2:alertmanager

jpc_windows_exporter_service#

Agent-path\bin\jpc_windows_exporter_service.exe

Monitoring target 1:Windows metric collector(Windows exporter)

Monitoring target 2:windows_exporter

jpc_blackbox_exporter_service#

Agent-path\bin\jpc_blackbox_exporter_service.exe

Monitoring target 1:RM Synthetic metric collector(Blackbox exporter)

Monitoring target 2:blackbox_exporter

jpc_ya_cloudwatch_exporter_service#

Agent-path\bin\jpc_ya_cloudwatch_exporter_service.exe

Monitoring target 1:RM AWS metric collector(Yet another cloudwatch exporter)

Monitoring target 2:ya_cloudwatch_exporter

jpc_fluentd_service#

Agent-path\bin\jpc_fluentd_service.exe

  • When to Use fluentd

    Monitoring target 1:fluentd_win Log trapper (Fluentd)

    Monitoring target 2:fluentd

  • When using only log metrics feature

    Monitoring target 1:fluentd_prome_win Log trapper (Fluentd)

  • Monitoring target 2:fluentd

#

Indicates Windows service program.

Here is a sample alert-definition that Windows exporter monitors:

groups:
 - name: windows_exporter
  rules:
   - alert: jp1_pc_procmon_ Monitor target 1
     expr: expr: absent (windows_process_start_time {instance="imahost:20717", job = "jpc_windows", jp1_pc_exporter= "JPC Windows exporter", jp1_pc_nodelabel= "jpc_monitor target 2_service", process = "jpc_monitor target 2_service"}) = 1.
      for: 3m
      labels:
        jp1_pc_product_name: "/HITACHI/JP1/JPCCS2"
        jp1_pc_component: "/HITACHI/JP1/JPCCS/CONFINFO"
        jp1_pc_severity: "Error"
        jp1_pc_eventid: "00007600"
        jp1_pc_metricname: "windows_process_start_time"
      annotations:
        jp1_pc_firing_description: "The number of processes was less than the threshold Value (1). "
        jp1_pc_resolved_description: "The number of processes exceeded the threshold Value (1). "
  • Specify integrated agent host name in imahost part. Specify Port number of Windows exporter in 20717 part

  • For monitoring target 1 and monitoring target 2, specify the monitoring target name of Table 1-17 Processes monitored by the Windows exporter.

  • If you specify more than one alert definition, repeat Setup multiple times after the line starting with "-alert:".

(c) Monitoring with Prometheus server up metric

Windows exporter service, Blackbox exporter service, and Yet another cloudwatch exporter service are monitored through Prometheus server alert-monitoring. For details about Setting method of the alert definition, see "1.21.2(3)(b) To Add the alert definition (for Windows) (optional)".

Here is a sample alert-definition that monitors up metric:

groups:
  - name: exporter_healthcheck
    rules:
    - alert: jp1_pc_exporter_healthcheck
      expr: up{jp1_pc_remote_monitor_instance=""} == 0 or label_replace(sum by (jp1_pc_remote_monitor_instance,jp1_pc_exporter) (up{jp1_pc_remote_monitor_instance!=""}), "jp1_pc_nodelabel", "${1}", "jp1_pc_remote_monitor_instance", "^[^:]*:([^:]*)$") == 0
      for: 3m
      labels:
        jp1_pc_product_name: "/HITACHI/JP1/JPCCS2"
        jp1_pc_component: "/HITACHI/JP1/JPCCS/CONFINFO"
        jp1_pc_severity: "Error"
        jp1_pc_eventid: "00007600"
        jp1_pc_metricname: "up"
      annotations:
        jp1_pc_firing_description: " Communication to Exporter failed. "
        jp1_pc_resolved_description: " Communication to Exporter was successful. "

(10) Creation and import of IM management node tree data (for Windows) (mandatory)

Follow the steps below to create and import IM management node tree.

  1. If you add a new integrated agent host or change hostname of integrated agent host, start service JP1/IM - Agent control base on that host.

  2. Start add-on program and the integrated agent control base service in the same host, when you add an new add-on program or change program settings leads to configuration changes.

  3. After all integrated agent host have started the corresponding service in steps 1 and 2, wait for one minute# after starting the service.

    #: If Value of the scrape_interval of Prometheus configuration file (jpc_prometheus.yml) has been changed, wait for that Value time.

  4. Perform the steps in Integrated manager host.

    For details about the procedure, see steps 2 to 5 in "1.19.3(1)(c)Creation and import of IM management node tree data (for Windows) (mandatory)".

(11) Security-product exclusion Setup (for Windows) (optional)

If you are deploying antivirus software or security products, Setup the following directories to exclude them:

(12) Notes on updating the difinition file (for Windows)

If you Restart the service of JP1/IM - Agent to reflect the updated content of definition files, monitoring stops during restart of the service is ongoing.