3.2.2 Viewing Trend Data with the Dashboard

■ Prerequisites for the prediction detection alert definition function

When using the prediction detection function, make sure that the following conditions are satisfied:

• Metric for which you want to set proactive alerting must be monitored by JP1/IM - Agent.

  For details about metric subject to prediction detection, see the description in the predict_linear of 2.7.4(7)(f) Available functions.

• You must specify a performance data retention period that you specify in Prometheus command's arguments that is greater than the alert definition reference period for prediction detection.

  For details about how long to retain performance data, see prometheus command options in Service definition file (jpc_program-name_service.xml) in Chapter 2. Definition Files in the JP1/Integrated Management 3 - Manager Command, Definition File and API Reference. There is no default alert rule definition.

■ Prerequisites for approximate straight line viewing function

• Metric for which you want to view approximate straight line must be able to view it in the Trends pane of the dashboard window. Unlike prediction detection alert definition function, approximate straight line can be displayed if metric can be displayed in the Trends pane even if metric is not collected by JP1/IM - Agent (for example, the number of unhandled events).

■ Temporary measures for Web response time drop

The following shows an operation example of detecting a pre-failure in response time drop and temporarily dealing with it.

• Configuration of Web system

  Web server, AP server, and DB server are each configured with multiple machines, and JP1/IM - Agent is installed on each server for OS monitoring and logging monitoring. In addition, JP1/IM - Manager and JP1/IM - Agent that integrate agent on a single server machine are installed, and Synthetic metric collector (Blackbox exporter) monitors URL response time of this Web system.

• Defining Dashboards for resource comparison

  Dashboard tab of integrated operation viewer defines a dashboard that compares CPU utilization, memory requirements, and free disk-space of servers in Web servers, AP servers, and DB servers.

• Defining related node

  The IM management node link definition file (imdd_nodeLink_def.conf) defines the relationships between the following nodes:

  • The nodes that monitor the URL response time of the Web system and each node of the web server group

  • Each node of Web server group and each node of AP server group

  • Each node of AP server group and each node of DB server group

• Prediction detection alert definition

  In the Alert configuration file (jpc_alerting_rules.yml) of JP1/IM - Agent for which the synthetic monitoring is being performed, the response time after one day is predicted based on the response time of 2 days, and a prediction event is registered when the predicted value is greater than 10 seconds It defines an alert for the prediction detection.

  groups:
   - name: blackbox_exporter
     rules:
     - alert: predict_probe_http_duration_seconds(Blackbox exporter)
       expr: 10 < predict_linear(probe_http_duration_seconds[2d], 24*3600)
       for: 3m
       labels:
         jp1_pc_product_name: "/HITACHI/JP1/JPCCS2"
         jp1_pc_component: "/HITACHI/JP1/JPCCS/CONFINFO"
         jp1_pc_severity: "Warning"
         jp1_pc_eventid: "0504"
         jp1_pc_metricname: "probe_http_duration_seconds"
       annotations:
         jp1_pc_firing_description: "The predicted value one day later exceeded the threshold value (10 seconds)."
         jp1_pc_resolved_description: "The predicted value one day later dropped below the threshold value (10 seconds)"

• Operator action

  Synthetic metric collector (Blackbox exporter) in JP1/IM - Agent monitors URL response time of Web system. If the predicted value exceeds the threshold value, a preliminary detection alert issues a JP1 event. The operator escalates to the system administrator if an email notification via an automatic action or alert information on the dashboard window detects a predictable drop-in response time after one day, for example, as shown in the following figure.

  

• System administrator action

1. On integrated operation viewer, the administrator selects the node on which the alert is occurring and browses to the related node tabbed page to find out which resources are related to the response time. When warnings or errors occur in related resources, check the operational information of each resource and investigate the cause.

   For example, if you find that AP server group's CPU utilization on one server has not reached the warning/error threshold but is on an upward trend compared to other resources, as shown in the following figure, you can assume that CPU utilization of this AP server is the cause of the reduced response time and perform a scale-out of AP server as a temporary measure.

   

2. After the temporary action is complete, view the dashboard with a panel similar to the following to see the validity of the temporary action and future trends.

   • Panel type: Trend

   • Target metric: Metric of predictive triggering alerts

   • Target node: Node of predictive triggering alerts

   • Approximate straight line viewing settings: View

   • Display range: To prediction period from reference range set in prediction alert

   For example, if the Panel type: Trend, Target metric: Response Time, Display Setting of Approximate straight line: Display, Display Range: 2 days ago to 1 day later panel is displayed, temporary countermeasures are enabled, and the response time is degraded as shown in the following figure, and it is confirmed that there is no problem in the predicted value after 1 day as the present trend, proceed to investigation of the root cause and the examination of the main countermeasures.

   

3. Investigate other operational information and refer to the log of the application to investigate the root cause and to investigate the main measures.

■ Temporary measures for running out of free disk space on Web

The following shows an operation example of detecting a prediction of disk space exhaustion and temporarily handling it.

• Configuration of Web system

  Same as Temporary measures for Web response time drop.

• Defining Dashboards for resource comparison

  Same as Temporary measures for Web response time drop.

• Prediction detection alert definition

  In the Alert configuration file (jpc_alerting_rules.yml) of the servers of DB servers group, the free disk space after 3 days is predicted using the free disk space of 1 day as the source data, and a warning event is registered when the predicted value is smaller than 100GB.

  groups:
   - name: node_exporter
     rules:
     - alert: predict_disk_unused(Node exporter)
       expr: 100*1024*1024*1024 > predict_linear(node_filesystem_free_bytes[1d], 3*24*3600)
       for: 3m
       labels:
         jp1_pc_product_name: "/HITACHI/JP1/JPCCS2"
         jp1_pc_component: "/HITACHI/JP1/JPCCS/CONFINFO"
         jp1_pc_severity: "Warning"
         jp1_pc_eventid: "0304"
         jp1_pc_metricname: "disk_unused"
       annotations:
         jp1_pc_firing_description: "Predicted value at 3 days below threshold (100GB)"
         jp1_pc_resolved_description: "Predicted value after 3 days exceeds threshold (100GB)"

• Operator action

  Synthetic metric collector (Blackbox exporter) in JP1/IM - Agent monitors the amount of free disk space on Web system. If the estimated value exceeds the threshold, the prediction detection alert definition issues a JP1 event. The operator escalates to the system administrator if an email notification via automatic action or alert information on the dashboard screen detects a predictable disk space exhaustion after 3 days, for example, as shown in the following figure.

  

• System administrator action

1. As a temporary measure, the administrator expands the disk space of DB servers that detect the possibility of running out of free disk space to 200GB.

2. After the temporary action is complete, view the dashboard with a panel similar to the following to see the validity of the temporary action and future trends.

   • Panel type: Trend

   • Target metric: Metric of predictive triggering alerts

   • Target node: Node of predictive triggering alerts

   • Approximate straight line viewing settings: View

   • Display range: To date and time when approximate straight line reaches the threshold value from reference range set in prediction alert

   For example, if the Panel type: Trend, Target metric: Free Disk Space, and Approximate straight line Display Settings: Display, Display Range: 1 day ago to Approximate straight line reaches the threshold, and the date and time panels are displayed, the temporary measures function is enabled, and it is confirmed that the free disk space is increasing and that it takes more than a month to reach the threshold as shown in the following figure, then the root cause is investigated and the main measures are examined.

   

3. Investigate other operational information and refer to the log of the application to investigate the root cause and to investigate the main measures.

■ Use Dashboard display data for analytics

You can CSV out the performance values collected by JP1/IM - Agent or trend data representing the occurrence of JP1 events and combine them with other data not covered by JP1/IM (such as cost information) to analyze trends in data such as correlations.

■ Utilize dashboard display data as a report

You can export the metric data collected by JP1/IM - Agent in a CSV format that is easy to summarize in a report.

■ Trend Panel

• Header

  The following format shows the header output for the panel type "Trend". The line feed "LF" (0x0a) is entered at the end of the line.

  If there is no single row of data to be output, no data is output (only "Date and Time" is output in the headers).

  "Date and Time","y-axis metric display name (legend of y-axis instance 1)","y-axis metric display name (legend of y-axis instance 2)",...,"y2 axis metric display name(legend of y2 axis instance 1)","y2 axis metric display name (legend of y2 axis instance 2)",...

  Table 3‒5: Output items for headers (when the panel type is "Trend")

  Item	Description
  Date and Time	Fixed character string Prints a fixed "Date and Time" indicating a date and time column.
  y-axis metric display name #1	Metric display name of the y-axis time-series data displayed in the pane.
  Legend #1 for y-axis instance x	Legend for instance x of the y-axis time-series data displayed in the pane This item will be printed even if the legend is hidden in the panel settings.
  y2-axis metric display name #2	Metric display name the y2-axis time-series data displayed in the pane. If y2 axis is not specified in the pane, ", y2 axis metric display name (y2 axis instance x legend)" is not displayed.
  Legend #2 for y2-axis instance x	Legend for instance x of the y2-axis time-series data displayed in the pane This item will be printed even if the legend is hidden in the panel settings. If y2 axis is not specified in the pane, ", y2 axis metric display name (y2 axis instance x legend)" is not displayed.

  #1

  Prints metric display name and instance legend for time-series data on the y-axis, as many as instance.

  #2

  When y2 axis is set, metric display name and instance legend set to time-series data of y2 axis are outputted for the number of instance.

• Data

  Here is the data output format for the panel type "Trend":

  Print one time-series data per line. The line feed "LF" (0x0a) is entered at the end of the line.

  Repeatedly outputting as many times as time-series data.

  If a legend is selected in the panel, it is output even when the graph is hidden on the panel.

  "Date and Time","Performance value of y-axis instance 1","Performance value of y-axis instance 2",...,"Performance value of y2 axis instance 1","Performance value of y2 axis instance 2",...

  Table 3‒6: Data output (when the panel type is "Trend")

  Item	Description
  Date and Time	Date and time of time-series data (YYYY/MM/DD hh:mm:ss format) Each performance value printed on the same line as this date and time represents a value for this datetime. The time zone is the local time of the client.
  Performance #1, #3 of y-axis instance x	Performance of time-series data's instance x on the y-axis An empty string is used if there is no corresponding value for datetime. #3
  Performance #2 #3 of y2-axis instance x	Performance of time-series data's instance x on the y2-axis An empty string is used if there is no corresponding value for datetime. #3

  #1

  Outputs metric of time-series data on the y-axis as many times as instance.

  #2

  When y2 axis is set, metric value set in time-series data of y2 axis is outputted for the number of instance.

  #3

  If there is no performance value for both the y-axis and y2 axis, the data for that row is not output (rows with no data, including the date and time).

  If there is no performance value of instance corresponding to the date and time on either of the y-axis or y2 axis, the output performance value will be empty. (If y2 axis is valid and the plot number setting of the y-axis and y2 axis differs, the date and time will differ on the y-axis and y2 axis, so either of the performance values will be empty.)

■ Ranking panel

• Header

  The following format shows the header output for the panel type "Ranking". The line feed "LF" (0x0a) is entered at the end of the line.

  If there is no single row of data to be output, no data is output.

  "Date and Time","Instance","Metric display name"

  Table 3‒7: Output items for headers (when the panel type is "Ranking")

  Item	Description
  Date and Time	Fixed character string Prints a fixed "Date and Time" indicating a date and time column.
  Instance	Fixed character string Prints a fixed "Instance" indicating that the column identifies instance.
  Metric display name	Metric display name of the data displayed in the pane.

• Data

  Here is the data output format for the panel type "Ranking".

  One instance is output per line. The line feed "LF" (0x0a) is entered at the end of the line.

  The output order is the same as the order specified in Sort on the pane. Repeatedly outputting as many times as instance.

  "Date and time","Instance x legend","Instance x performance data",...

  Table 3‒8: Data output (when the panel type is "Ranking")

  Item	Description
  Date and Time	Date and time of time-series data (YYYY/MM/DD hh:mm:ss format) Each performance value printed on the same line as this date and time represents a value for this datetime. The time zone is the local time of the client.
  Instance x legend	Instance x legend for the ranking data displayed in the pane.
  Instance x performance data	Instance x performance of the ranked data displayed in the pane.

■ Numerical value Panel

• Header

  The following format shows the header output for the panel type "Numerical value". The line feed "LF" (0x0a) is entered at the end of the line.

  If there is no single row of data to be output, no data is output.

  "Date and Time","Metric display name(Instance name of instance 1)","Metric display name(Instance name of instance 2)",...,"Metric display name Total value or Average value"

  Table 3‒9: Output items for headers (when the panel type is "Numerical value")

  Item	Description
  Date and Time	Fixed character string Prints a fixed "Date and Time" indicating a date and time column.
  Metric display name	Metric display name of the data displayed in the pane.
  Instance name of instance x	Instance name of instance x From the second column to the "Number of instance + 1" column, "Metric display name(instance name)" is displayed, and the last column is "Metric display name Total value or Average value".
  Total value or Average value	- When "Total value" is selected in Display value of Display options For the Japanese environment 合計値 For the English environment Total value - When "Average value" is selected in Display value of Display options For the Japanese environment 平均値 For the English environment Average value

• Data

  Here is the data output format for the panel type "Numerical value". The line feed "LF" (0x0a) is entered at the end of the line.

  "Date and Time","Instance 1 performance data","Instance 2 performance data",...,"Total value or Average value of performance values"

  Table 3‒10: Data output (when the panel type is "Numerical value")

  Item	Description
  Date and Time	Date and time of time-series data (YYYY/MM/DD hh:mm:ss format) Each performance value printed on the same line as this date and time represents a value for this datetime. The time zone is the local time of the client.
  Instance x performance data	Instance x performance data An empty string is used if there is no corresponding value.
  Total value or Average value of performance values	Total value or Average value of the performance values (performance values displayed on the panel) for each instance.

■ Gauge Panel

• Header

  Panel type Same as "Ranking".

• Data

  Panel type Same as "Ranking".

■ Alert information Panel

• Header

  The following format shows the header output for the panel type "Alert information". The line feed "LF" (0x0a) is entered at the end of the line.

  For the Japanese environment:

    "Date and Time","Node name","Title","緊急/警戒/致命的","エラー","警告","通知/情報/デバック","その他"

  For non-Japanese environments:

    "Date and Time","Node name","Title","Emergency/Alert/Critical","Error","Warning","Notice/Information/Debug","Others"

  Table 3‒11: Output items for headers (when the panel type is "Alert information")

  Item	Description
  Date and Time	Fixed character string Prints a fixed "Date and Time" indicating a date and time column.
  Node name	Fixed character string Prints a fixed "Node name" indicating a node name.
  Third column	Text displayed in red on the donut chart For the Japanese environment Outputs a fixed string of "Emergency/Warning/Fatal". For non-Japanese environments Fixed text of "Emergency/Alert/Critical" is output.
  Fourth and subsequent columns	String displayed by color in the donut chart For the Japanese environment Outputs the fixed text string of "エラー" displayed in orange, "警告" displayed in yellow, "通知/情報/デバック" displayed in green, and "その他" displayed in gray, in descending order of severity. For non-Japanese environments Fixed text of "Error" displayed in orange, "Warning" displayed in yellow, "Notice/Information/Debug" displayed in green, and "Others" displayed in gray is output in descending order of severity.

• Data

  Here is the data output format for the panel type "Alert Information". The line feed "LF" (0x0a) is entered at the end of the line.

  "Date and Time","Node name","Title","Red alert count","Orange alert count","Yellow alert count","Green alert count","Gray Alert count"

  Table 3‒12: Data output (when the panel type is "Alert Information")

  Item	Description
  Date and Time	Date and time when alert information was retrieved (YYYY/MM/DD hh:mm:ss format) Each performance value printed on the same line as this date and time represents a value for this datetime. The time zone is the local time of the client.
  Node name	Node name of the specified IM manage node
  Title	Represents the title specified in the display options. If no title is specified, the last name of the tree path of the IM management node is output.
  Red alert count#	Number of alerts with severity Emergency, Alert, or Critical
  Orange alert count#	Number of alerts with a severity of Error
  Yellow alert count#	Number of alerts with a severity of Warning
  Green alert count#	Number of alerts with a severity of Notify, Informational, or Debug
  Gray Alert count#	Number of alerts with a severity other than those listed above

  #

  The second and subsequent columns output the number of alerts classified by color in the donut chart, in descending order of severity.

  Items that do not have an alert of that severity will output 0.

■ Node state Panel

• Header

  The following format shows the header output for the panel type "Node state". The line feed "LF" (0x0a) is entered at the end of the line.

  "Date and Time","Node name","Severity"

  Table 3‒13: Output items for headers (when the panel type is "Node state")

  Item	Description
  Date and Time	Fixed character string Prints a fixed "Date and Time" indicating a date and time column.
  Node name	Fixed character string Prints a fixed "Node name" indicating a node name.
  Severity	Fixed character string Prints a fixed "Severity" indicating a severity column.

• Data

  Here is the data output format for the panel type "Node state". The line feed "LF" (0x0a) is entered at the end of the line.

  "Date and Time","Node name","Severity"

  Table 3‒14: Data output (when the panel type is "Node state")

  Item	Description
  Date and Time	Date and time when alert information was retrieved (YYYY/MM/DD hh:mm:ss format) Each performance value printed on the same line as this date and time represents a value for this datetime. The time zone is the local time of the client.
  Node name	Node name of the specified IM manage node
  Severity	IM management node Status Depending on the color of the panel, output the following text: For the Japanese environment Red: 緊急 Orange: エラー Yellow: 警告 Green: 異常なし For non-Japanese environments Red: Emergency Orange: Error Yellow: Warning Green: Normal

■ Node state map Panel

• Header

  The following format shows the header output for the panel type "Node state map". The line feed "LF" (0x0a) is entered at the end of the line.

  "Date and Time","Node name","Severity"

  Table 3‒15: Output items for headers (when the panel type is "Node state map")

  Item	Description
  Date and Time	Fixed character string Prints a fixed "Date and Time" indicating a date and time column.
  Node name	Fixed character string Prints a fixed "Node name" indicating a node name column.
  Severity	Fixed character string Prints a fixed "Severity" indicating a severity column.

• Data

  Here is the data output format for the panel type "Node state map". The line feed "LF" (0x0a) is entered at the end of the line.

  Repeatedly prints as many as there are Node state map nodes.

  "Date and Time","Node name","Severity"

  Table 3‒16: Data output (when the panel type is "Node state map")

  Item	Description
  Date and Time	Date and time when alert information was retrieved (YYYY/MM/DD hh:mm:ss format) Each performance value printed on the same line as this date and time represents a value for this datetime. The time zone is the local time of the client.
  Node name	Node name of the node of Node state map you are viewing
  Severity	IM management node Status Depending on the color of the panel, output the following text: For the Japanese environment Red: 緊急 Orange: エラー Yellow: 警告 Green: 異常なし For non-Japanese environments Red: Emergency Orange: Error Yellow: Warning Green: Normal

■ Text Panel

• Header

  The following format shows the header output for the panel type "Text". The line feed "LF" (0x0a) is entered at the end of the line.

  "Date and Time","Text"

  Table 3‒17: Output items for headers (when the panel type is "Text")

  Item	Description
  Date and Time	Fixed character string Prints a fixed "Date and Time" indicating a date and time column.
  Text	Fixed character string Prints a fixed "Text" indicating that it is text panel data.

• Data

  Here is the data output format for the panel type "Text". The line feed "LF" (0x0a) is entered at the end of the line.

  "Date and Time","Text data"

  Table 3‒18: Data output (when the panel type is "Text")

  Item	Description
  Date and Time	Date and time the data in the text panel was retrieved (YYYY/MM/DD hh:mm:ss format) Each performance value printed on the same line as this date and time represents a value for this datetime. The time zone is the local time of the client.
  Text data	Text data in the text format set by the user in the text panel Even if the data contains newline codes or markdown settings, it is output as is.

■ Trend Panel

• File name

  1_dashboard-1_1_CPU-utilization-and-free-memory.csv

• Output contents

  "ID","1"
  "Title","Dashboard 1"
  "Panel Number","1"
  "Panel Type","Trend"
  "Panel Title","CPU Utilization and Free-Memory Percentage"
  "Tree SID","_ROOT_AllSystems/_HOST_HOSTA/_CATEGORY_platform/_OBJECT_JP1PC-ALinux%20metric%20collector%28Node%20exporter%29"
  "Server Host","immHost"
   
  "Date and Time","CPU Utilization [%](hostA)","Free Memory Ratio [%](hostA)"
  "2024/3/14 12:48:00","4.31342681937852","70.86472444540427"
  "2024/3/14 12:49:00","10.3163827614285","70.86472444540427"
  "2024/3/14 12:50:00","15.378526841972","58.7666666666670303"
  .........

■ Ranking panel

• File name

  1_dashboard-1_2_Free-Disk-Space.csv

• Output contents

  "ID","1"
  "Title","Dashboard 1"
  "Panel Number","2"
  "Panel Type","Ranking"
  "Panel Title","Free Disk Space"
  "Tree SID","_ROOT_AllSystems/_HOST_HOSTA/_CATEGORY_platform/_OBJECT_JP1PC-ALinux%20metric%20collector%28Node%20exporter%29"
  "Server Host","immHost"
   
  "Date and Time","Instance","Free Disk Space [gigabyte]"
  "2024/3/14 12:48:46","hostA [device:/dev/mapper/hostA-home, fstype:xfs, mountpoint:/home]","52.8446666666670303"
  "2024/3/14 12:48:46","hostA [device:/dev/mapper/hostA-root, fstype:xfs, mountpoint:/]","42.626182674125"
  "2024/3/14 12:48:46","hostA [device:tmpfs, fstype:tmpfs, mountpoint:/run]","2.22497352142"
  "2024/3/14 12:48:46","hostA [device:/dev/sda2, fstype:xfs, mountpoint:/boot]","0.669956428"
  "2024/3/14 12:48:46","hostA [device:/dev/sda1, fstype:vfat, mountpoint:/boot/efi]","0.5838742697"

■ Numerical value Panel

• File name

  1_Dashboard-1_3_Free-Disk-Space.csv

• Output contents

  "ID","1"
  "Title","Dashboard 1"
  "Panel Number","3"
  "Panel Type","Numerical value"
  "Panel Title","Free Disk Space"
  "Tree SID","_ROOT_AllSystems/_HOST_HOSTA/_CATEGORY_platform/_OBJECT_JP1PC-ALinux%20metric%20collector%28Node%20exporter%29"
  "Server Host","immHost"
   
  "Date and Time","Free Disk Space [gigabytes](hostA [device:/dev/mapper/ol_hostA-home, fstype:xfs, mountpoint:/home])","Free Disk Space [gigabytes](hostA [device:/dev/mapper/ol_hostA-root, fstype:xfs, mountpoint:/])","Free Disk Space [gigabytes](hostA [device:/dev/sda1, fstype:vfat, mountpoint:/boot/efi])","Free Disk Space [gigabytes](hostA [device:/dev/sda2, fstype:xfs, mountpoint:/boot])","Free Disk Space [gigabytes](hostA [device:sunrpc, fstype:rpc_pipefs, mountpoint:/var/lib/nfs/rpc_pipefs])","Free Disk Space [gigabytes](hostA [device:tmpfs, fstype:tmpfs, mountpoint:/run])","Free Disk Space [gigabytes](hostA [device:tmpfs, fstype:tmpfs, mountpoint:/run/user/0])","Free Disk Space [gigabytes](hostA [device:tmpfs, fstype:tmpfs, mountpoint:/run/user/42])","Free Disk Space [gigabytes] Total value"
  "2024/3/14 12:48:46","52.84421157836914","44.68961715698242","0.5798759460449219","0.669281005859375","0","1.7548828125","0.3542938232421875","0.354278564453125","101.246440887451"

■ Gauge Panel

• File name

  1_Dashboard-1_4_.csv

• Output contents

  "ID","1"
  "Title","Dashboard 1"
  "Panel Number","4"
  "Panel Type","Gauge"
  "Panel Title","Free Disk Space"
  "Tree SID","_ROOT_AllSystems/_HOST_HOSTA/_CATEGORY_platform/_OBJECT_JP1PC-ALinux%20metric%20collector%28Node%20exporter%29"
  "Server Host","immHost"
   
  "Date and Time","Free Disk Space [gigabytes](hostA [device:/dev/mapper/ol_hostA-home, fstype:xfs, mountpoint:/home])","Free Disk Space [gigabytes](hostA [device:/dev/mapper/ol_hostA-root, fstype:xfs, mountpoint:/])","Free Disk Space [gigabytes](hostA [device:/dev/sda1, fstype:vfat, mountpoint:/boot/efi])","Free Disk Space [gigabytes](hostA [device:/dev/sda2, fstype:xfs, mountpoint:/boot])","Free Disk Space [gigabytes](hostA [device:sunrpc, fstype:rpc_pipefs, mountpoint:/var/lib/nfs/rpc_pipefs])","Free Disk Space [gigabytes](hostA [device:tmpfs, fstype:tmpfs, mountpoint:/run])","Free Disk Space [gigabytes](hostA [device:tmpfs, fstype:tmpfs, mountpoint:/run/user/0])","Free Disk Space [gigabytes](hostA [device:tmpfs, fstype:tmpfs, mountpoint:/run/user/42])","Free Disk Space [gigabytes] Total value"
  "2024/3/14 12:48:46","52.84421157836914","44.68961715698242","0.5798759460449219","0.669281005859375","0","1.7548828125","0.3542938232421875","0.354278564453125","101.246440887451"

■ Alert information Panel

• File name

  1_Dashboard-1_5_Alert-Occurrence-Status.csv

• Output contents

  "ID","1"
  "Title","Dashboard 1"
  "Panel Number","5"
  "Panel Type","Alert information"
  "Panel Title,"Alert Occurrence Status"
  "Tree SID","_ROOT_AllSystems/_HOST_HOSTA/_CATEGORY_platform/_OBJECT_JP1PC-ALinux%20metric%20collector%28Node%20exporter%29"
  "Server Host","immHost"
   
  "Date and Time","Node name","Title","Emergency/Alert/Critical","Error","Warning","Notice/Information/Debug","Others"
  "2024/3/14 12:48:46","hostA > Platform > Linux metric collector(Node exporter)"," Linux metric collector(Node exporter) ","0","1","3","5","0"

■ Node state Panel

• File name

  1_Dashboard-1_6_Node-Status.csv

• Output contents

  "ID","1"
  "Title","Dashboard 1"
  "Panel Number","6"
  "Panel Type","Node state"
  "Panel Title","Node Status"
  "Tree SID","_ROOT_AllSystems/_HOST_HOSTA/_CATEGORY_platform/_OBJECT_JP1PC-ALinux%20metric%20collector%28Node%20exporter%29"
  "Server Host","immHost"
   
  "Date and Time","Node name","Severity"
  "2024/3/14 12:48:46","hostA > Platform > Linux metric collector(Node exporter)","Error"

■ Node state map Panel

• File name

  1_Dashboard-1_7_Related-Node-Status".csv

• Output contents

  "ID","1"
  "Title","Dashboard 1"
  "Panel Number","7"
  "Panel Type","Node state map"
  "Panel Title","Related Node Status"
  "Tree SID","_ROOT_AllSystems/_HOST_HOSTA/_CATEGORY_platform/_OBJECT_JP1PC-ALinux%20metric%20collector%28Node%20exporter%29"
  "Server Host","immHost"
   
  "Date and Time","Node name","Severity"
  "2024/3/14 12:48:46","hostA > Platform > Linux metric collector(Node exporter)","Emergency"
  "2024/3/14 12:48:46","hostA > Platform > processA","Error"
  "2024/3/14 12:48:46","hostA > Platform > processB","Warning"

■ Text Panel

• File name

  1_Dashboard-1_8_Note.csv

• Output contents

  "ID","1"
  "Title","Dashboard 1"
  "Panel Number","8"
  "Panel Type","Text"
  "Panel Title","Note"
  "Tree SID","_ROOT_AllSystems/_HOST_HOSTA/_CATEGORY_platform/_OBJECT_JP1PC-ALinux%20metric%20collector%28Node%20exporter%29"
  "Server Host","immHost"
   
  "Date and Time","Text"
  "2024/3/14 12:48:46","Note: ** performed **"