12.5.1 How to isolate faults

msg="Error on ingesting samples that are too old or are too far into the future"

You have changed the system time in the past.

Wait until the time before the change.

msg="Append failed" err="out of bounds"

msg="Appending scrape report failed" err="out of bounds"

component=remote

msg="Failed to send batch, retrying"

The JP1/IM - Manager host was unable to send performance information from Prometheus due to high load.

Check the load status of the JP1/IM - Manager host.

The connection to the JP1/IM - Manager host was not established or was in an unstable situation.

Check the connection status with the JP1/IM - Manager host.

The JP1/IM - Manager service was not started.

Check the startup status of the JP1/IM - Manager service.

The JP1/IM - Manager service returned an HTTP response that indicates an error.

Check to see if there are any errors on the JP1/IM - Manager service side.

The remote light destination specified in the Prometheus configuration file (jpc_prometheus_server.yml) was incorrect.

Review the contents of the Jpc_prometheus_server.yml configuration file (jpc_prometheus_server.yml).

msg="Skipping resharding, last successful send was beyond threshold"

Because the threshold was reached when prometheus failed to send performance information due to the load situation of the JP1/IM - Manager host, the connection status, etc., the load balancing was omitted again.

Check the load status and connection status of the JP1/IM - Manager host.

msg="Error sending alert"

Alertanager had stopped.

Start Alertonanager.

Alertonanager returned an HTTP response that indicates an error.

Check to see if there are any errors on the Alertanager side.

The alert notification destination specified in the Prometheus configuration file (jpc_prometheus_server.yml) was incorrect.

Review the specifications in the Prometheus configuration file (jpc_prometheus_server.yml).

msg="Scrape failed"

The scrape failed because the exporter is hesitation.

Launch Exporter.

Scrape failed because you specified a host name that does not exist in the discovery configuration file (file_sd_config_*.yml).

Review the specified contents of the discovery configuration file (file_sd_config_*.yml).

An HTTP response indicating an error was returned from the scrape destination.

Check if there are any errors at the scrape destination.

msg="Unable to start web listener"

An invalid host name was specified on the command line Optional--web.listen-address.

Specify the correct host name.

On the command line --web.listen-address Optional you specified a port that is already in use.

Specify a port that is not in use.

msg="Error loading config (--config.file=jpc_prometheus_server.yml)"

msg="Error reading file"

An invalid value was specified in the item that specifies the period of rometheus configuration file (jpc_prometheus_server.yml).

Check the invalid line number or item name displayed in err and review the specified contents of the Prometheus configuration file (jpc_prometheus_server.yml).

The format of the Prometheus configuration file (jpc_prometheus_server.yml) was not followed.

The format of the Prometheus configuration file (jpc_prometheus_server.yml) was not followed.

• msg="Notify for alerts failed"

• msg="Notify attempt failed, will retry later"

Alerts could not be sent from AlertManager due to high on load of the JP1/IM - Manager host.

Check the on-load status of the JP1/IM - Manager host.

The Connection with the JP1/IM - Manager host was not established or was unstable.

Check the status of the Connection with the JP1/IM - Manager host.

JP1/IM - Manager service was not running.

Check the startup status of the JP1/IM - Manager's service.

JP1/IM - Manager service returned an HTTP response indicating an error.

Check if there are any errors on the service side of the JP1/IM - Manager.

The alert destination specified in the Alertmanager configuration file (alertmanager.yml) was incorrect.

Review the specifications in the Alertmanager configuration file (alertmanager.yml).

msg="Loading configuration file failed"

The specified contents of the Alertmanager configuration file (alertmanager.yml) were invalid.

Check the contents described in err and review the specified contents of the Alertmanager configuration file (alertmanager.yml).

msg="Listen error"

An invalid host name was specified on the command line Optional--web.listen-address.

Specify the correct host name.

On the command line --web.listen-address Optional you specified a port that is already in use.

Specify a port that is not in use.

msg=" Unable to do unprivileged listen on socket, will attempt privileged " err=" socket: permission denied"

Occurs when the kernel parameter 'net.ipv4.ping_group_range' creates a socket without authority to create a ping socket. This also happens for privileged users (root).

This message can be safely ignored because ICMP sockets are created and continue with the authority of the privileged user (root) after the message is Output.

It is also not recommended to Assistant to authority by setting the kernel parameter "net.ipv4.ping_group_range" only to suppress this message.

module=icmp

msg="Timeout reading from socket"

The ICMP probe failed because the host to be monitored specified in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of the Blackbox exporter (ICMP monitoring) is hesitation.

Review the specifications in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of Blackbox exporter (ICMP monitoring).

module=icmp

msg="Resolution with IP protocol failed"

The ICMP probe failed because a nonexistent host name was specified in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of the Blackbox exporter (ICMP monitoring).

Review the specifications in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of Blackbox exporter (ICMP monitoring).

module=http

msg="Error for HTTP request"

The monitored host specified in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring) was hesitation.

Review the specifications in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring).

The monitoring target service specified in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring) was hesitation.

A Calipers that does not exist was specified in modules.module name .http.basic_auth.password_file) of the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring).

module=http

msg="Error resolving address"

A nonexistent host name was specified in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring).

Review the specifications in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring).

module=http

msg="Failed to get decompressor for HTTP response body"

The HTTP probe failed because the user name specified in modules.module name .http.basic_auth.username in the Blackbox exporter configuration file (blackbox_exporter.yml) is invalid.

Review the specifications in the Blackbox exporter configuration file (blackbox_exporter.yml).

The HTTP probe failed because the Calipers word specified in modules.module name .http.basic_auth.password in the Blackbox exporter configuration file (blackbox_exporter.yml) is invalid.

The HTTP probe failed because the modules.module name .http.basic_auth.bearer_token in the Blackbox exporter configuration file (blackbox_exporter.yml) is invalid.

module=icmp

msg="Error listening to socket"

An invalid IP address was specified in the modules.module name .icmp.source_ip_address of the Blackbox exporter configuration file (blackbox_exporter.yml).

Review the specifications in the Blackbox exporter configuration file (blackbox_exporter.yml).

module=http

msg="Error generating HTTP client"

The HTTP probe failed because the Calipers specified in the modules.module name .http.tls_config.ca_file of the Blackbox exporter configuration file (blackbox_exporter.yml) does not exist.

Review the specifications in the Blackbox exporter configuration file (blackbox_exporter.yml).

The HTTP probe failed because the Calipers specified in the modules.module name .http.tls_config.cert_file of the Blackbox exporter configuration file (blackbox_exporter.yml) does not exist.

The HTTP probe failed because the Calipers specified in the modules.module name .http.tls_config.key_file of the Blackbox exporter configuration file (blackbox_exporter.yml) does not exist.

msg="Error loading config"

An invalid value was specified in the item that specifies the numerical value in the Blackbox exporter configuration file (blackbox_exporter.yml).

Check the incorrect line number or field name displayed in err and review the specifications in the Blackbox exporter configuration file (blackbox_exporter.yml).

An invalid value was specified for the item that specifies the regular expression in the Blackbox exporter configuration file (blackbox_exporter.yml).

An invalid character string was specified in the item that specifies the value of the boolean type in the Blackbox exporter configuration file (blackbox_exporter.yml).

module=icmp

msg="Failed to set Control Message for retrieving TTL" err=" not implemented on windows/amd64"

(omission)

module=icmp

msg=" Cannot get TTL from the received packet. 'probe_icmp_reply_hop_limit' will be missing."

In a Windows environment, ICMP monitoring is performed by setting the discovery configuration file (file_sd_config_blackbox_icmp.yml) of Blackbox exporter (ICMP monitoring).

No action is required.

err="listen tcp: lookup hostname on DNS: no such host"

An invalid host name was specified on the command line Optional--web.listen-address.

Specify the correct host name.

err="listen tcp :port: bind: address already in use"

On the command line --web.listen-address Optional you specified a port that is already in use.

Specify a port that is not in use.

msg="Parsed flag --collector.filesystem.option name" flag=[aaa

panic: regexp: Compile(`[aaa`): error parsing regexp: missing closing ]: `[aaa`

An illegal regular expression was specified in the command line Optional.

Review the command line Optional.

msg="cannot start windows_exporter: listen tcp: lookup hostname: no such host"

An invalid host name was specified in --telemetry.addr Optional the command line.

Review the command line Optional.

msg="cannot start windows_exporter: listen tcp :port: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted."

On the command line, --telemetry.addr Optional a port that is already in use.

Review the command line Optional.

msg="Loading configuration file: windows_exporter.yml"

An invalid regular expression was specified in the item for specifying a regular expression in the Windows exporter configuration file (windows_exporter.yml).

Review the specifications in the Windows exporter configuration file (windows_exporter.yml).

An invalid value was specified for the item that specifies the numerical value of the Windows exporter configuration file (windows_exporter.yml).

"msg": "Couldn't get account Id for role : NoCredentialProviders:..."

Data retrieval from CloudWatch failed because ~/.aws/credentials do not exist.

Place the credentials file.

"msg":"Couldn't get account Id for role : InvalidClientTokenId:..."

Data retrieval from CloudWatch failed because the Access Key ID described in ~/.aws/credentials is invalid.

Please review the contents of the credentials file.

"msg":"Couldn't get account Id for role : SignatureDoesNotMatch:..."

Data retrieval from CloudWatch failed because the Secret Access Key listed in ~/.aws/credentials is invalid.

Please review the contents of the credentials file.

"msg":"Couldn't describe resources for region region name:~"

You specified an AWS Region that does not exist in discovery.jobs.regions in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml).

Review the specifications in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml).

"msg":"Couldn't read config.yml: Discovery job [0]: Service is not in known list!: AWS service name"

In the discovery.jobs.type of Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml), you specified an AWS service that does not exist.

Review the specifications in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml).

"msg":"Couldn't read config.yml: yaml: unmarshal errors:..."

An invalid value was specified in the item that specifies the numerical value in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml).

Check the incorrect line number or field name displayed in msg and review the specifications in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml).

Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml) does not follow the format.

An invalid character string was specified in the item specifying the boolean value in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml).

"msg":"Couldn't read config.yml: Metric [/0] in Discovery job [job name]: Name should not be empty"

You did not specify a value for the discovery.jobs.metrics.name in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml).

Review the specifications in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml).

error="tail: 'path' parameter is required on tail input"

In text-formatted log file monitoring definition file, path of the [Input Settings] section was not specified.

Correct text-formatted log file monitoring definition file.

error="Unsupported timezone~"

In text-formatted log file monitoring definition file, an invalid timezone was specified in the [Input Settings] section..

Correct text-formatted log file monitoring definition file.

error="specifying timezone requires time format"

In text-formatted log file monitoring definition file, timezone is specified without specifying the time_format of the [Input Settings] section..

Correct text-formatted log file monitoring definition file.

error="valid options are string,unixtime,float,mixed but got ~"

In text-formatted log file monitoring definition file, an invalid time_type was specified in the [Input Settings] section.

Correct text-formatted log file monitoring definition file.

error="unknown encoding name~"

In text-formatted log file monitoring definition file, an invalid from_encoding was specified in the [Input Settings] section.

Correct text-formatted log file monitoring definition file.

error="No named captures in 'expression' parameter. The regexp must have at least one named capture"

In text-formatted log file monitoring definition file, you specified a regular expression that does not include a named capture (?<NAME> PATTERN) in expression of the [Input Settings] section..

Correct text-formatted log file monitoring definition file.

error="format should be Regexp, need //,~"

In text-formatted log file monitoring definition file, the regular expression is specified without a delimiter (/) in the regular expression for parsing the log in the [Input Settings] section.

Correct text-formatted log file monitoring definition file.

error="valid options are rfc3164,rfc5424,auto but got ~"

In text-formatted log file monitoring definition file, an invalid message_format was specified in the [Input Settings] section.

Correct text-formatted log file monitoring definition file.

error="'with_priority' parameter is required but ~ is specified"

In text-formatted log file monitoring definition file, an invalid with_priority was specified in the [Input Settings] section.

Correct text-formatted log file monitoring definition file.

"valid options are regexp,string but got~"

In text-formatted log file monitoring definition file, an invalid parser_type was specified in the [Input Settings] section.

Correct text-formatted log file monitoring definition file.

error="'support_colonless_ident' parameter is required but ~ is specified"

In text-formatted log file monitoring definition file, an incorrect support_colonless_ident was specified in the [Input Settings] section.

Correct text-formatted log file monitoring definition file.

got incomplete JSON array configuration at ~(Fluent::ConfigParseError)

In text-formatted log file monitoring definition file, a regular expression is specified pattern the [Inclusion Settings] or [Exclusion Settings] section without a delimiter (/).

Correct text-formatted log file monitoring definition file.

error="Plugin 'tail' does not support multi workers configuration (Fluent::Plugin::TailInput)"

In text-formatted log file monitoring definition file, when workers parameter is specified to 2 or more, id of worker is specified more than once. Or, if you specify a workers parameter that is greater than or equal to 2, you did not specify the <worker> directive.

Correct text-formatted log file monitoring definition file.

error="Unknown parser plugin ~. Run 'gem search -rd fluent-plugin' to find plugins"

In text-formatted log file monitoring definition file, an invalid @type was specified for the <parse> directive in the [Input Settings] section.

Correct text-formatted log file monitoring definition file.

error="Invalid Bookmark XML is loaded.~"

In Windows event-log monitoring definition file, an invalid channels was specified in the [Input Settings] section.

Correct Windows event-log monitoring definition file.

error="Plugin 'windows_eventlog2' does not support multi workers configuration (Fluent::Plugin::WindowsEventLog2Input)"

In Windows event-log monitoring definition file, when workers parameter is specified to 2 or more, id of worker is specified more than once. Or, if you specify a workers parameter that is greater than or equal to 2, you did not specify the <worker> directive.

Correct Windows event-log monitoring definition file.

error="valid options are throw_exception,block,drop_oldest_chunk but got~"

In log monitoring common definition file, an invalid overflow_action was specified in the [Output Settings] section.

Correct log monitoring common definition file.

valid options are trace,debug,info,warn,error,fatal but got~(Fluent::ConfigParseError)

In log monitoring common definition file, an invalid value was specified for log_level in the [System Settings] section.

Correct log monitoring common definition file.

invalid number of workers (must be > 0):0 (Fluent::ConfigError)

In log monitoring common definition file, an invalid workers parameter was specified in the [System Settings] section.

Correct log monitoring common definition file.

error="greater first_worker_id<1> than last_worker_id<0> specified by <worker> directive is not allowed. Available multi worker assign syntax is <smaller_worker_id>-<greater_worker_id>"

In log monitoring common definition file, the <worker N-M> directive specifies the number N>M.

Correct log monitoring common definition file.

error="worker id ~ specified by <worker> directive is not allowed. Available worker id is between ~"

You specified <worker> directive arguments in either log monitoring common definition file, text-formatted log file monitoring definition file, or Windows event-log monitoring definition file that are beyond workers parameter. Or, you specify a nonzero id as an argument to the <worker N-M> or <worker> directive without specifying workers parameter.

Review log monitoring common definition file, text-formatted log file monitoring definition file, and Windows event-log monitoring definition file.

error="Missing worker id on <worker> directive"

You did not specify id of worker as an argument to the <worker> directive in either log monitoring common definition file, text-formatted log file monitoring definition file, or Windows event-log monitoring definition file.

Review log monitoring common definition file, text-formatted log file monitoring definition file, and Windows event-log monitoring definition file.

[error]: failed to read data from plugin storage file path=~

Storage file is corrupt.

Remove storage file. Then start it.

Not a directory @ dir_s_mkdir - ~ (Errno::ENOTDIR)

An invalid monitor name was set.

Review the monitor name.