3.14.8 Cooperation with user's own Prometheus

The following describes how to work with user-defined Prometheus.

Verifying the prerequisites

The prerequisite is that JP1/IM - Manager,Prometheus, Alertmanager, Exporter is installed and set up.
Publish client secret

Execute JP1/IM - Manager's client secret publishing API(/im/api/v1/secret/client/generateInfo) to obtain client secret ^#.

#

Client secret can be retrieved only once, so be sure to keep a copy of it.

Here is an example of the message body for a request and response:
- Message body of the request
```
{
  "clientId": ">prometheus"
}
```
- Message body of the response
```
{
    "secret": "kABNY0UFwa4K+W5KhLbkva945Whx+sguH65QinFggaW3CZaB6bA0MkgNgKN2FSSs1kORnQHpWUCHitkeBUzEmYHY09d9BR+tg73vSfbh05xStE7qB1idgfzisqhlqu7R+Q+XNA",
    "lastUpdateTime": "2023-05-18T07:59:23Z"
}
```
For details about client secret issue API, see 5.4.5 IM Client secret issue in the JP1/Integrated Management 3 - Manager Command, Definition File and API Reference.

Editing prometheus.yml

Edits prometheus.yml directly under Prometheus destination (the path where prometheus.exe resides).

An example is shown below. Bold text indicates points added or changed from the default. Specify values in italics to suit your environment.

(Example of description)

# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).
  external_labels:
    jp1_pc_prome_hostname: "Prometheus-hostneme"
# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
           - Prometheus-hostneme:Prometheus-Port-number
 
 
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  - "alerting_rules.yml^#1"
  # - "first_rules.yml"
  # - "second_rules.yml"
 
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
 
- job_name: 'OPTIONAL NAME ^#2'
    file_sd_configs:
      - files:
        - 'file_sd_config_any name.yml^#3'
    relabel_configs:
      - target_label: jp1_pc_nodelabel
        replacement: Windows exporter
 
    metric_relabel_configs:
      - source_labels: ['__name__']
        regex: 'windows_cs_physical_memory_bytes|windows_cache_copy_read_hits_total|windows_cache_copy_reads_total|windows_cpu_time_total|windows_logical_disk_free_bytes|windows_logical_disk_idle_seconds_total|windows_logical_disk_read_bytes_total|windows_logical_disk_read_latency_seconds_total|windows_logical_disk_read_seconds_total|windows_logical_disk_reads_total|windows_logical_disk_requests_queued|windows_logical_disk_size_bytes|windows_logical_disk_write_bytes_total|windows_logical_disk_write_latency_seconds_total|windows_logical_disk_write_seconds_total|windows_logical_disk_writes_total|windows_memory_available_bytes|windows_memory_cache_bytes|windows_memory_cache_faults_total|windows_memory_page_faults_total|windows_memory_pool_nonpaged_allocs_total|windows_memory_pool_paged_allocs_total|windows_memory_swap_page_operations_total|windows_memory_swap_pages_read_total|windows_memory_swap_pages_written_total|windows_memory_system_cache_resident_bytes|windows_memory_transition_faults_total|windows_net_bytes_received_total|windows_net_bytes_sent_total|windows_net_bytes_total|windows_net_packets_sent_total|windows_net_packets_received_total|windows_system_context_switches_total|windows_system_processor_queue_length|windows_system_system_calls_total'
        action: 'keep'
 
remote_write:
  - url: http://JP1/IM - Manager hostname: Intelligent Integrated Management Base portnumber/im/api_system
    remote_timeout: 30s
    send_exemplars: false
    queue_config:
      capacity: 10000
      max_shards: 200
      min_shards: 4
      max_samples_per_send: 3000
      batch_send_deadline: 10s
      min_backoff: 100ms
      max_backoff: 10s
    basic_auth:
      username: 'Client ID specified in the client secret publishing API in JP1/IM - Manager'
      password: 'Client secret issued by the client secret publishing API in JP1/IM - Manager'

#1: Specify the filename of alert configuration file described in step 6.
#2: Write a scrape definition for each Exporter.
#3: Specify the filename of the discovery configuration file as described in step 4.

Creating a discovery configuration file

Create a discovery configuration file (file_sd_config_any name.yml) and place it directly under Prometheus destination.

An example is shown below. In italics, specify a value appropriate for your environment.

(Example of description)
```
- targets:
  - Hostname of Exporter: Port number of Exporter
  labels:
    jp1_pc_exporter: any-name ^#1
    jp1_pc_category: platform
    jp1_pc_trendname: any-name ^#2
```
#1

Specify any name that does not start with "JPC_".

#2

You must specify a name that matches the "Any Prometheus trend name" part of the filename of metric definition file (metrics_any Prometheus trend name.conf).

Editing alertmanager.yml

Edits alertmanager.yml directly under Alertmanager destination (the path where alertmanager.exe resides).

An example is shown below. Bold text indicates points added from the default. Specify values in italics to suit your environment.

(Example of description)

route:
  receiver: 'JP1IMDD'
  group_by: ['...']
  continue: false
  group_wait: 5s
  group_interval: 5s
  repeat_interval: 7d
 
receivers:
- name: 'JP1IMDD'
 
  webhook_configs:
  - send_resolved: true
    url: 'http:// JP1/IM - Manager Host Name: Intelligent Integrated Management Platform port number/im/api_system/v1/events/transform'
    http_config:
      basic_auth:
        username: 'Client ID specified in the client secret publishing API in MJP1/IM - Manager'
        password: 'Client ID specified in the client secret publishing API in JP1/IM - Manager'

Creating a alert configuration file

Create an alert-definition file (with any name) and place it directly under Prometheus destination.

An example is shown below. If you want to create an alert definition file other than the example description, see Definition example in alert configuration file (jpc_alerting_rules.yml) in Chapter 2. Definition Files in the JP1/Integrated Management 3 - Manager Command, Definition File and API Reference.

(Example of definition)

groups:
  - name: windows_exporter
    rules:
    - alert: cpu_used_rate(Windows exporter)
      expr: 0 < 100 - (avg by (instance,job,jp1_pc_nodelabel,jp1_pc_exporter)
      for: 3m
      labels:
        jp1_pc_product_name: "/HITACHI/JP1/JPCCS2"
        jp1_pc_component: "/HITACHI/JP1/JPCCS/CONFINFO"
        jp1_pc_severity: "Error"
        jp1_pc_eventid: "0401"
        jp1_pc_metricname: "windows_cpu_time_total"
      annotations:
        jp1_pc_firing_description: "CPU utilization has exceeded the threshold value (0%). value={{$value}}%"
        jp1_pc_resolved_description: "CPU utilization has fallen below the threshold (0%). "

Creating a metric definition file

Create a metric definition file and place it in the following path:

JP1/IM - Manager install directory

\conf\imdd\plugin\jp1pccs\user\

If the targets you define are JP1/IM - Agent monitorable Exporter and metric, you can use the files provided by JP1/IM - Agent (JP1/IM - Manager install directory \conf\imdd\plugin\jp1pccs\*.conf).
Starting Exporter ,Alertmanager,Prometheus

Start Exporter ,Alertmanager,Prometheus.
Generating and reflecting trees

Start the command prompt in administrator mode and run the following command:
- jddsetacccessuser (not required if already executed)
- jddcreatetree
- jddupdatetree
Checking the Tree

Log in to integrated operation viewer and verify that nodes of Prometheus, Alertmanager Exporter are created.

To Page Top