12.5.1 How to isolate faults
The following table lists the reasons and corrective actions for messages that are output to add-on program logs. For details about the log storage locations, see 12.2.2 JP1/IM - Agent log information. Node exporter for AIX log is output to the system log.
- Organization of this subsection
(1) Prometheus server log
|
Message |
Cause |
Actions to be taken |
|---|---|---|
|
msg="Error on ingesting samples that are too old or are too far into the future" |
You have changed the system time in the past. |
Wait until the time before the change. |
|
msg="Append failed" err="out of bounds" |
||
|
msg="Appending scrape report failed" err="out of bounds" |
||
|
component=remote msg="Failed to send batch, retrying" |
The JP1/IM - Manager host was unable to send performance information from Prometheus due to high load. |
Check the load status of the JP1/IM - Manager host. |
|
The connection to the JP1/IM - Manager host was not established or was in an unstable situation. |
Check the connection status with the JP1/IM - Manager host. |
|
|
The JP1/IM - Manager service was not started. |
Check the startup status of the JP1/IM - Manager service. |
|
|
The JP1/IM - Manager service returned an HTTP response that indicates an error. |
Check to see if there are any errors on the JP1/IM - Manager service side. |
|
|
The remote light destination specified in the Prometheus configuration file (jpc_prometheus_server.yml) was incorrect. |
Review the contents of the Jpc_prometheus_server.yml configuration file (jpc_prometheus_server.yml). |
|
|
msg="Skipping resharding, last successful send was beyond threshold" |
Because the threshold was reached when prometheus failed to send performance information due to the load situation of the JP1/IM - Manager host, the connection status, etc., the load balancing was omitted again. |
Check the load status and connection status of the JP1/IM - Manager host. |
|
msg="Error sending alert" |
Alertanager had stopped. |
Start Alertonanager. |
|
Alertonanager returned an HTTP response that indicates an error. |
Check to see if there are any errors on the Alertanager side. |
|
|
The alert notification destination specified in the Prometheus configuration file (jpc_prometheus_server.yml) was incorrect. |
Review the specifications in the Prometheus configuration file (jpc_prometheus_server.yml). |
|
|
msg="Scrape failed" |
The scrape failed because the exporter is hesitation. |
Launch Exporter. |
|
Scrape failed because you specified a host name that does not exist in the discovery configuration file (file_sd_config_*.yml). |
Review the specified contents of the discovery configuration file (file_sd_config_*.yml). |
|
|
An HTTP response indicating an error was returned from the scrape destination. |
Check if there are any errors at the scrape destination. |
|
|
msg="Unable to start web listener" |
An invalid host name was specified on the command line Optional--web.listen-address. |
Specify the correct host name. |
|
On the command line --web.listen-address Optional you specified a port that is already in use. |
Specify a port that is not in use. |
|
|
msg="Error loading config (--config.file=jpc_prometheus_server.yml)" msg="Error reading file" |
An invalid value was specified in the item that specifies the period of rometheus configuration file (jpc_prometheus_server.yml). |
Check the invalid line number or item name displayed in err and review the specified contents of the Prometheus configuration file (jpc_prometheus_server.yml). |
|
The format of the Prometheus configuration file (jpc_prometheus_server.yml) was not followed. |
||
|
The format of the Prometheus configuration file (jpc_prometheus_server.yml) was not followed. |
(2) Alertmanager log
|
Message |
Cause |
Action to be taken |
|---|---|---|
|
Alerts could not be sent from AlertManager due to high on load of the JP1/IM - Manager host. |
Check the on-load status of the JP1/IM - Manager host. |
|
The Connection with the JP1/IM - Manager host was not established or was unstable. |
Check the status of the Connection with the JP1/IM - Manager host. |
|
|
JP1/IM - Manager service was not running. |
Check the startup status of the JP1/IM - Manager's service. |
|
|
JP1/IM - Manager service returned an HTTP response indicating an error. |
Check if there are any errors on the service side of the JP1/IM - Manager. |
|
|
The alert destination specified in the Alertmanager configuration file (alertmanager.yml) was incorrect. |
Review the specifications in the Alertmanager configuration file (alertmanager.yml). |
|
|
msg="Loading configuration file failed" |
The specified contents of the Alertmanager configuration file (alertmanager.yml) were invalid. |
Check the contents described in err and review the specified contents of the Alertmanager configuration file (alertmanager.yml). |
|
msg="Listen error" |
An invalid host name was specified on the command line Optional--web.listen-address. |
Specify the correct host name. |
|
On the command line --web.listen-address Optional you specified a port that is already in use. |
Specify a port that is not in use. |
(3) blackbox_exporter log
In order for blackbox_exporter to collect the following logs, the log level must be set to "debug":
|
Message |
Cause |
Action to be taken |
|---|---|---|
|
msg=" Unable to do unprivileged listen on socket, will attempt privileged " err=" socket: permission denied" |
Occurs when the kernel parameter 'net.ipv4.ping_group_range' creates a socket without authority to create a ping socket. This also happens for privileged users (root). |
This message can be safely ignored because ICMP sockets are created and continue with the authority of the privileged user (root) after the message is Output. It is also not recommended to Assistant to authority by setting the kernel parameter "net.ipv4.ping_group_range" only to suppress this message. |
|
module=icmp msg="Timeout reading from socket" |
The ICMP probe failed because the host to be monitored specified in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of the Blackbox exporter (ICMP monitoring) is hesitation. |
Review the specifications in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of Blackbox exporter (ICMP monitoring). |
|
module=icmp msg="Resolution with IP protocol failed" |
The ICMP probe failed because a nonexistent host name was specified in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of the Blackbox exporter (ICMP monitoring). |
Review the specifications in the discovery configuration file (file_sd_config_blackbox_icmp.yml) of Blackbox exporter (ICMP monitoring). |
|
module=http msg="Error for HTTP request" |
The monitored host specified in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring) was hesitation. |
Review the specifications in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring). |
|
The monitoring target service specified in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring) was hesitation. |
||
|
A Calipers that does not exist was specified in modules.module name .http.basic_auth.password_file) of the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring). |
||
|
module=http msg="Error resolving address" |
A nonexistent host name was specified in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring). |
Review the specifications in the discovery configuration file (file_sd_config_blackbox_http.yml) of Blackbox exporter (HTTP/HTTPS monitoring). |
|
module=http msg="Failed to get decompressor for HTTP response body" |
The HTTP probe failed because the user name specified in modules.module name .http.basic_auth.username in the Blackbox exporter configuration file (blackbox_exporter.yml) is invalid. |
Review the specifications in the Blackbox exporter configuration file (blackbox_exporter.yml). |
|
The HTTP probe failed because the Calipers word specified in modules.module name .http.basic_auth.password in the Blackbox exporter configuration file (blackbox_exporter.yml) is invalid. |
||
|
The HTTP probe failed because the modules.module name .http.basic_auth.bearer_token in the Blackbox exporter configuration file (blackbox_exporter.yml) is invalid. |
||
|
module=icmp msg="Error listening to socket" |
An invalid IP address was specified in the modules.module name .icmp.source_ip_address of the Blackbox exporter configuration file (blackbox_exporter.yml). |
Review the specifications in the Blackbox exporter configuration file (blackbox_exporter.yml). |
|
module=http msg="Error generating HTTP client" |
The HTTP probe failed because the Calipers specified in the modules.module name .http.tls_config.ca_file of the Blackbox exporter configuration file (blackbox_exporter.yml) does not exist. |
Review the specifications in the Blackbox exporter configuration file (blackbox_exporter.yml). |
|
The HTTP probe failed because the Calipers specified in the modules.module name .http.tls_config.cert_file of the Blackbox exporter configuration file (blackbox_exporter.yml) does not exist. |
||
|
The HTTP probe failed because the Calipers specified in the modules.module name .http.tls_config.key_file of the Blackbox exporter configuration file (blackbox_exporter.yml) does not exist. |
||
|
msg="Error loading config" |
An invalid value was specified in the item that specifies the numerical value in the Blackbox exporter configuration file (blackbox_exporter.yml). |
Check the incorrect line number or field name displayed in err and review the specifications in the Blackbox exporter configuration file (blackbox_exporter.yml). |
|
An invalid value was specified for the item that specifies the regular expression in the Blackbox exporter configuration file (blackbox_exporter.yml). |
||
|
An invalid character string was specified in the item that specifies the value of the boolean type in the Blackbox exporter configuration file (blackbox_exporter.yml). |
||
|
module=icmp msg="Failed to set Control Message for retrieving TTL" err=" not implemented on windows/amd64" (omission) module=icmp msg=" Cannot get TTL from the received packet. 'probe_icmp_reply_hop_limit' will be missing." |
In a Windows environment, ICMP monitoring is performed by setting the discovery configuration file (file_sd_config_blackbox_icmp.yml) of Blackbox exporter (ICMP monitoring). |
No action is required. |
(4) node_exporter log
|
Message |
Cause |
Action to be taken |
|---|---|---|
|
err="listen tcp: lookup hostname on DNS: no such host" |
An invalid host name was specified on the command line Optional--web.listen-address. |
Specify the correct host name. |
|
err="listen tcp :port: bind: address already in use" |
On the command line --web.listen-address Optional you specified a port that is already in use. |
Specify a port that is not in use. |
|
msg="Parsed flag --collector.filesystem.option name" flag=[aaa panic: regexp: Compile(`[aaa`): error parsing regexp: missing closing ]: `[aaa` |
An illegal regular expression was specified in the command line Optional. |
Review the command line Optional. |
|
msg="Parsed flag --collector.systemd.unit-include" flag=(panic: regexp: Compile(`^(?:()$`): error parsing regexp: missing closing ): `^(?:()$` |
An invalid regular expression was specified in --collector.systemd.unit-include. |
Review the specified regular expression. |
(5) process_exporter logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
flag needs an argument: command-line-option |
A value has not been specified in the command line option. |
Review the command line option specified. |
|
Failed to start the server: listen tcp: address port: missing port in address |
An invalid port was specified to the command line option --web.listen-address. |
Review the command line option --web.listen-address specified. |
|
Failed to start the server: listen tcp: lookup host-name on DNS: error-details |
An address that does not exist was specified to the command line option --web.listen-address. |
Review the command line option --web.listen-address specified. |
|
Failed to start the server: listen tcp :port: bind: address already in use |
A port already in use was specified to the command line option --web.listen-address. |
Specify a port that is not in use. |
|
-config.path cannot be used with -namemapping or -procnames |
Both the command line options --procnames and --config.path are specified. |
Review the command line option specified. |
|
Both the command line options --namemapping and --config.path are specified. |
||
|
Error parsing -namemapping argument 'specified-character-string': bad namemapper: odd number of tokens |
An odd number of values have been specified for the command line option --namemapping. |
Review the command line option --namemapping specified. |
|
Error parsing -namemapping argument 'specified-character-string': error compling regexp 'invalid-regular-expression': error parsing regexp: error-details |
An invalid regular expression has been specified for the command line option --namemapping. |
Review the command line option --namemapping specified. |
|
error reading config file "specified-path": error reading config file "specified-path": open specified-path: no such file or directory |
A path that does not exist has been specified for the command line option --config.path. |
Review the command line option --config.path specified. |
|
error reading config file "specified-path": error-details |
The format of the file specified to the command line option --config.path is invalid. |
|
|
error reading config file "specified-path": error-details |
The configuration file contains invalid formatting. |
Review the format of the Process exporter configuration file (jpc_process_exporter.yml). |
|
Error initializing: open /proc/stat: permission denied |
Process exporter executed the command without access permissions for the /proc directory, or access permissions for the /proc directory were lost while Process exporter is running. |
Grant read permissions to users running Process exporter for the /proc directory and its subdirectories. If Exporter is stopped, restart the integrated agent. |
|
error reading procs: Error reading procs: Error reading procs: open /proc: permission denied |
(6) windows_exporter log
|
Message |
Cause |
Action to be taken |
|---|---|---|
|
msg="cannot start windows_exporter: listen tcp: lookup hostname: no such host" |
An invalid host name was specified in --telemetry.addr Optional the command line. |
Review the command line Optional. |
|
msg="cannot start windows_exporter: listen tcp :port: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted." |
On the command line, --telemetry.addr Optional a port that is already in use. |
Review the command line Optional. |
|
msg="Loading configuration file: windows_exporter.yml" |
An invalid regular expression was specified in the item for specifying a regular expression in the Windows exporter configuration file (windows_exporter.yml). |
Review the specifications in the Windows exporter configuration file (windows_exporter.yml). |
|
An invalid value was specified for the item that specifies the numerical value of the Windows exporter configuration file (windows_exporter.yml). |
||
|
level=error msg="failed collecting service metrics:<nil> An exception occurred. (Invalid query) "source="service.go:77" |
An invalid WQL of where phrase was specified in services-where. |
Re-specify services-where of the windows_exporter.yml. |
(7) Logging node_exporter_aix
|
Message |
Cause |
Action |
|---|---|---|
|
Error starting HTTP server: bind: Address already in use |
You started the node_exporter_aix registered to the service by specifying the port number that was used for the command line option when it was started. |
Review the command-line options at startup of the node_exporter_aix registered to the service. |
|
~/node_exporter_aix: illegal option -- ~ |
You invoked the node_exporter_aix registered to the service with an option that is not available for the command-line options at startup. |
Review the command-line options at startup of the node_exporter_aix registered to the service. |
|
Error calling perfstat_fcstat: Invalid argument |
Scrape was executed when one of the following conditions was met while FC data could not be acquired:
|
Review the command-line options at startup of the node_exporter_aix registered to the service. |
(8) ya_cloudwatch_exporter log
|
Message |
Cause |
Action to be taken |
|---|---|---|
|
"msg": "Couldn't get account Id for role : NoCredentialProviders:..." |
Data retrieval from CloudWatch failed because ~/.aws/credentials do not exist. |
Place the credentials file. |
|
"msg":"Couldn't get account Id for role : InvalidClientTokenId:..." |
Data retrieval from CloudWatch failed because the Access Key ID described in ~/.aws/credentials is invalid. |
Please review the contents of the credentials file. |
|
"msg":"Couldn't get account Id for role : SignatureDoesNotMatch:..." |
Data retrieval from CloudWatch failed because the Secret Access Key listed in ~/.aws/credentials is invalid. |
Please review the contents of the credentials file. |
|
"msg":"Couldn't describe resources for region region name:~" |
You specified an AWS Region that does not exist in discovery.jobs.regions in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml). |
Review the specifications in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml). |
|
"msg":"Couldn't read config.yml: Discovery job [0]: Service is not in known list!: AWS service name" |
In the discovery.jobs.type of Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml), you specified an AWS service that does not exist. |
Review the specifications in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml). |
|
"msg":"Couldn't read config.yml: yaml: unmarshal errors:..." |
An invalid value was specified in the item that specifies the numerical value in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml). |
Check the incorrect line number or field name displayed in msg and review the specifications in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml). |
|
Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml) does not follow the format. |
||
|
An invalid character string was specified in the item specifying the boolean value in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml). |
||
|
"msg":"Couldn't read config.yml: Metric [/0] in Discovery job [job name]: Name should not be empty" |
You did not specify a value for the discovery.jobs.metrics.name in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml). |
Review the specifications in the Yet another cloudwatch exporter configuration file (ya_cloudwatch_exporter.yml). |
(9) Promitor logs
(a) Scraper logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
Failed to scrape resource collection discovery-group-name: Connection refused (promitor.agents.resourcediscovery:port) |
The Scraper service was started with the Resource Discovery service stopped. |
Start the Resource Discovery Agent. |
|
The incorrect port number was specified for the resourceDiscovery.port in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the resourceDiscovery.port in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
|
AuthorizationFailed: The client 'object-ID' with object id 'object-ID' does not have authorization to perform action 'microsoft.insights/metricDefinitions/read' over scope '/subscriptions/subscription-ID/resourceGroups/resource-group-name /providers/resource-type/resource-name/providers/microsoft.insights' or the scope is invalid. If access was recently granted, please refresh your credentials. |
Specified an identity with insufficient permissions for the Monitoring Reader role. |
Grant permissions that meet or exceed the requirements of the Monitoring Reader role. |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. System.Security.Authentication.AuthenticationException: No identity secret was configured for service principle authentication |
When the authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml) is ServicePrincipal, the Scraper was started with the incorrect value registered as the secret for connecting to Azure Monitor. |
Review the content specified for the AUTH_APPKEY environment variable. |
|
The following problems were found with the metric configuration: <error-details> Validation failed: Errors were found while deserializing the metric configuration. Promitor is not configured correctly. Please fix validation issues and re-run. |
The format of the Promitor Scraper configuration file (metrics-declaration.yaml) is not followed. |
Check error-details for the invalid line number and item name, and review the content specified for the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an out-of-range value was specified for azureMetadata.cloud in the same file. |
||
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value in an invalid format was specified for metricDefault.aggregation.interval. |
||
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metricDefault.scraping.schedule. |
||
|
An out-of-range value was specified for metrics.resourceType in the Promitor Scraper configuration file (metrics-declaration.yaml). |
||
|
A value in an invalid format was specified for metrics.scraping.schedule in the Promitor Scraper configuration file (metrics-declaration.yaml). |
||
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an out-of-range value was specified for metrics.azureMetricConfiguration.aggregation.type. |
||
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value in an invalid format was specified for metrics.azureMetricConfiguration.aggregation.interval. |
||
|
Validation failed: Errors were found while deserializing the metric configuration. Promitor is not configured correctly. Please fix validation issues and re-run. |
The format of the Promitor Scraper configuration file (metrics-declaration.yaml) is not followed. |
Review the contents of the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metricDefault.aggregation.interval. |
||
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metricDefault.scraping.schedule. |
||
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metrics.azureMetricConfiguration.aggregation.type. |
||
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metrics.azureMetricConfiguration.aggregation.interval. |
||
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Unable to deserialize the configured metrics declaration. Please verify that it is a well-formed YAML specification. |
The format of the Promitor Scraper configuration file (metrics-declaration.yaml) is not followed. |
Review the specified contents of the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource for metric 'metric-name' AADSTS900023: Specified tenant identifier 'specified-character-string' is neither a valid DNS name, nor a valid external domain. |
When the authentication.mode is ServicePrincipal in the Promitor Scraper runtime configuration file (runtime.yaml), a value in an invalid format was specified for azureMetadata.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: No tenant id is configured. Promitor is not configured correctly. Please fix validation issues and re-run. |
When resources is not specified by any metric in metrics in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for authentication.tenantId. |
Review the content specified for authentication.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
When resources is specified by a metric in metrics in the Promitor Scraper runtime configuration file (runtime.yaml), a value is not specified for authentication.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for authentication.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
Failed to scrape resource for metric 'metric-name' AADSTS90002: Tenant 'tenant-ID' not found. Check to make sure you have the correct tenant ID and are signing into the correct cloud. Check with your subscription administrator, this may happen if there are no active subscriptions for the tenant. |
A tenant ID that does not exist was specified for azureMetadata.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
InvalidSubscriptionId: The provided subscription identifier 'specified-character-string' is malformed or invalid. |
A value in an invalid format was specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
A value in an invalid format was specified for metrics.resources.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
Validation failed: - No subscription id is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for azureMetadata.subscriptionId. |
Review the content specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for azureMetadata.subscriptionId. |
Review the content specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
SubscriptionNotFound: The subscription 'subscription-ID' could not be found. |
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a subscription ID that does not exist was specified for azureMetadata.subscriptionId. |
Review the content specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
A subscription ID that does not exist was specified for metrics.resources.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
AuthorizationFailed: The client 'object-ID' with object id 'object-ID' does not have authorization to perform action microsoft.insights/metricDefinitions/read' over scope '/subscriptions/subscription-ID/resourceGroups/specified-character-string/providers/resource-type/resource-name/providers/microsoft.insights' or the scope is invalid. If access was recently granted, please refresh your credentials. |
When the authentication.mode is SystemAssignedManagedIdentity in the Promitor Scraper runtime configuration file (runtime.yaml), and metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for azureMetadata.resourceGroupName. |
Review the content specified for azureMetadata.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
An invalid value was specified for metrics.resources.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
Validation failed: - No resource group name is not configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for azureMetadata.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Microsoft.IdentityModel.Clients.ActiveDirectory.AdalServiceException: AADSTS90038: Tenant ' tenant-ID ' request is being redirected to the National Cloud 'MicrosoftOnline.COM'. |
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an out-of-range value was specified for azureMetadata.cloud. |
Review the content specified for azureMetadata.cloud in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape metric 'metric-name' for resource 'resource-name' .System.NotSupportedException: Environment 'cloud-name' is not supported for scraping Azure Log Analytics resource(s) |
When the authentication.mode is ServicePrincipal in the Promitor Scraper runtime configuration file (runtime.yaml), an incorrect cloud name was specified for azureMetadata.cloud in the metrics-declaration.yaml. |
Review the content specified for azureMetadata.cloud in the metrics-declaration.yaml. |
|
Failed to scrape resource for metric 'metric-name' AADSTS500011: The resource principal named https://management.core.cloudapi.de/ was not found in the tenant named tenant-ID. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant. |
When the authentication.mode is SystemAssignedManagedIdentity or UserAssignedManagedIdentity in the Promitor Scraper runtime configuration file (runtime.yaml), an incorrect cloud name was specified for azureMetadata.cloud in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.cloud in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape: Value cannot be null. (Parameter 'key') |
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), interval and type were not specified in metricDefault.aggregation and metrics.azureMetricConfiguration.aggregation. |
Review the content specified for interval and type in metricDefault.aggregation or metrics.azureMetricConfiguration.aggregation in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource collection resource-name: Value cannot be null. (Parameter 'key') |
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), interval and type were not specified in metricDefault.aggregation and metrics.azureMetricConfiguration.aggregation. |
Review the content specified for interval and type in metricDefault.aggregation or metrics.azureMetricConfiguration.aggregation in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - Limit has to be at least 1 Promitor is not configured correctly. Please fix validation issues and re-run. |
A value less than 1 was specified for metricDefault.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metricDefault.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
A value less than 1 was specified for metrics.azureMetricConfiguration.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
||
|
Validation failed: - No default metric scraping schedule is defined. - No metrics scraping schedule is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for metricDefault.scraping.schedule in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metricDefault.scraping.schedule in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - No metric name is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for metrics.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - Metric name 'metric-name' is declared multiple times Promitor is not configured correctly. Please fix validation issues and re-run. |
The same value as another metric was specified for metrics.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - No metric name for Azure is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for metrics.azureMetricConfiguration.Name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.Name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource for metric 'metric-name' Promitor.Integrations.AzureMonitor.Exceptions.MetricNotFoundException: The metric 'specified-character-string' was not found |
An invalid value was specified for metrics.azureMetricConfiguration.Name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.Name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - Limit cannot be higher than metricDefault.limit-value |
A value greater than 10000 was specified for metrics.azureMetricConfiguration.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
BadRequest: Metric: Available Memory Bytes does not support requested dimension combination: specified-character-string, supported ones are: , TraceId: { trace-ID } |
An invalid value was specified for metrics.azureMetricConfiguration.dimension.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.dimension.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. [resourceDiscoveryGroupName] cannot be Null, empty or white-space. (Parameter 'resourceDiscoveryGroupName') |
A value was not specified for metrics.resourceDiscoveryGroups.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resourceDiscoveryGroups.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource collection specified-character-string: Response status code does not indicate success: 404 (Not Found). |
An invalid value was specified for metrics.resourceDiscoveryGroups.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resourceDiscoveryGroups.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
An invalid value was specified for resourceDiscoveryGroups.name in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for resourceDiscoveryGroups.name in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
|
An invalid value was specified for resourceDiscoveryGroups.type in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for resourceDiscoveryGroups.type in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
|
Validation failed: - No property-name is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for metrics.resources.property in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.property in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
ResourceNotFound: The Resource 'resource-type/character-string' under resource group 'resource-group-name' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix |
An invalid value was specified for metrics.resources.property in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.property in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - Queue & topic name are both configured while we only support one or the other. Promitor is not configured correctly. Please fix validation issues and re-run. |
Both the queueName and topicName were specified in metrics.resources when the metrics.resourceType is ServiceBusNamespace in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
BadRequest: Metric: Azure-metric-name does not accept zero dimension case |
A value was not specified for name in metrics.azureMetricConfiguration.dimension when the metrics.resourceType is SqlServer in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.dimension.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
BadRequest: Metric: Azure-metric-name does not support requested dimension combination: specified-value, supported ones are: DatabaseResourceId, TraceId: { trace-ID} |
An invalid value was specified for metrics.azureMetricConfiguration.dimension.name when the metrics.resourceType is SqlServer in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.dimension.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource collection discovery-group-name: Response status code does not indicate success: 501 (Not Implemented). |
A value was not specified for resourceDiscoveryGroups.type in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for resourceDiscoveryGroups.type in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to convert configuration value at 'authentication:Mode' to type 'Promitor.Integrations.Azure.Authentication.AuthenticationMode'. |
An invalid value was specified for the authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
Validation failed: Azure authentication is not configured correctly - No identity was configured for service principle authentication Promitor is not configured correctly. Please fix validation issues and re-run. |
When a secret has not been registered for connecting to Azure Monitor, and metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
When Serviceprincipal is specified for authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml), and metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for authentication.identityId in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.identityId in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
|
Failed to scrape resource for metric 'metric-name' AADSTS700016: Application with identifier 'specified-character-string' was not found in the directory 'Azure-directory-name '. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You may have sent your authentication request to the wrong tenant. Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to bind to address http://[::]:port: address already in use. |
When the authentication.mode is ServicePrincipal in the Promitor Scraper runtime configuration file (runtime.yaml), an invalid value was specified for the authentication.identityId. |
Review the specified content for the authentication.identityId in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
A port that is already in use was specified for the server.httpPort in the Promitor Scraper runtime configuration file (runtime.yaml). |
Specify a port that is not in use for the server.httpPort in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to convert configuration value at 'server:HttpPort' to type 'System.Int32'. |
An invalid value was specified for the server.httpPort in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the server.httpPort in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Could not find file 'path'. |
An invalid value was specified for the metricsConfiguration.absolutePath in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the metricsConfiguration.absolutePath in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Could not find a part of the path '/config/metrics-declaration.yaml'. |
A value was not specified for metricsConfiguration.absolutePath in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the metricsConfiguration.absolutePath in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to convert configuration value at 'resourceDiscovery:Port' to type 'System.Int32'. |
An invalid value was specified for the resourceDiscovery.port in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the resourceDiscovery.port in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
AuthorizationFailed: The client 'client-ID' with object id 'object-ID' does not have authorization to perform action 'microsoft.insights/metricDefinitions/read' over scope 'resource-Uri' or the scope is invalid. If access was recently granted, please refresh your credentials |
When the authentication.mode is ServicePrincipal in the Promitor Scraper runtime configuration file (runtime.yaml), and metrics.resources is specified in the Promitor configuration file (metrics-declaration.yaml), an invalid value was specified for azureMetadata.resourceGroupName in the Promitor configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.resourceGroupName in the Promitor configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource for metric 'metric-name' System.Net.Http.HttpRequestException: Resource temporarily unavailable (host-name:port-number) |
The Scraper is disconnected from the network. |
Connect the network. |
|
Failed to discover resources for group resource-group-name. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. |
The Scraper attempt to retrieve information from the Resource Discovery who is disconnected from the network. |
Connect the network. |
|
Failed to scrape resource for metric 'metric-name' System.Net.Http.HttpRequestException: Name or service not known (host-name:port-number) |
The Scraper was started with the incorrect proxy. |
Review the proxy settings. |
|
Failed to discover resources for group kubernetes-service-group. System.Net.Http.HttpRequestException: Response status code does not indicate success: 500 (Internal Server Error). |
The Scraper attempt to retrieve information from the Resource Discovery with an incorrect proxy configured. |
Review the proxy settings. |
(b) Resource Discovery logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
Unhandled exception in job Azure Subscription Discovery One or more errors occurred. ([subscriptions] cannot be Null. (Parameter 'subscriptions')) |
Specified an identity with insufficient permissions for the Reader role. |
Grant permissions that meet or exceed the requirements of the Reader role. |
|
ClientSecretCredential authentication failed: AADSTS700016: Application with identifier 'specified-character-string' was not found in the directory 'Azure-directory-name'. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You may have sent your authentication request to the wrong tenant. |
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), Resource Discovery was started with an incorrect value registered as the secret for connecting to Azure Resource Graph. |
Review the secret for connecting to Azure Resource Graph. |
|
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), an invalid value was specified for the authentication.identityId. |
Review the specified content for the authentication.identityId in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
|
Unhandled exception in job Azure Subscription Discovery One or more errors occurred. (ClientSecretCredential authentication failed: AADSTS900023: Specified tenant identifier 'specified-character-string' is neither a valid DNS name, nor a valid external domain. |
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), a value in an invalid format was specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Unhandled exception in job Azure Subscription Discovery One or more errors occurred. (ClientSecretCredential authentication failed: AADSTS90002: Tenant 'specified-character-string' not found. Check to make sure you have the correct tenant ID and are signing into the correct cloud. Check with your subscription administrator, this may happen if there are no active subscriptions for the tenant. |
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), a tenant ID that does not exist was specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Validation failed: - No tenant id was configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Operation returned an invalid status code 'BadRequest' |
A value in an invalid format was specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
[queriedSubscriptions] cannot be Null. (Parameter 'queriedSubscriptions') |
A subscription ID that does not exist was specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Validation failed: - No subscription id(s) were configured to query Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Promitor Discovery Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Value cannot be null. (Parameter 'text') |
A value was not specified for resourceDiscoveryGroups.name in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for resourceDiscoveryGroups.name in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Promitor Discovery Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to convert configuration value at 'authentication:Mode' to type 'Promitor.Integrations.Azure.Authentication.AuthenticationMode'. |
An invalid value was specified for the authentication.mode in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.mode in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
Validation failed: Azure authentication is not configured correctly - No identity was configured for service principle authentication Promitor is not configured correctly. Please fix validation issues and re-run. |
A secret has not been registered for connecting to Azure Resource Graph, and a value is not specified for authentication.mode in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.mode in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), a value was not specified for the authentication.identityId. |
Review the specified content for the authentication.identityId in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
|
Promitor Discovery Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to bind to address http://[::]:port: address already in use. |
A port that is already in use was specified for the server.httpPort in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
Review the specified content for the server.httpPort in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
ClientSecretCredential authentication failed: Retry failed after 2 tries. Retry settings can be adjusted in ClientOptions.Retry. (Resource temporarily unavailable (host-name:port-number)) (The operation was canceled.) |
The Scraper and the Resource Discovery started while disconnected from the network. |
Connect the network. |
|
Unhandled exception in job Azure Subscription Discovery System.AggregateException: One or more errors occurred. (ClientSecretCredential authentication failed: Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry. (Resource temporarily unavailable (host-name:port-number)) (Resource temporarily unavailable (host-name:port-number)) (Resource temporarily unavailable (host-name:port-number)) (Resource temporarily unavailable (host-name:port-number))) |
The Resource Discovery started while disconnected from the network. |
Connect the network. |
|
Unhandled exception in job Azure Subscription Discovery System.AggregateException: One or more errors occurred. (Resource temporarily unavailable (host-name:port-number)) |
It was disconnected from the network while the Resource Discovery was running. |
Connect the network. |
|
ClientSecretCredential authentication failed: Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry. (Name or service not known (host-name:port-number)) (Name or service not known (host-name:port-number)) (Name or service not known (host-name:port-number)) (Name or service not known (host-name:port-number)) |
The Resource Discovery was started with the incorrect proxy. |
Review the proxy settings. |
(10) script_exporter logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
flag needs an argument: command-line-option |
A value has not been specified in the command line option. |
Review the command line option. |
|
open path: no such file or directory |
A path that does not exist has been specified for the command line option --config.file. |
Review the command line option --config.file specified. |
|
listen tcp: address specified-character-string: missing port in address |
An invalid value was specified to the command line option --web.listen-address. |
Review the command line option --web.listen-address specified. |
listen tcp: lookup host-name : no such host
listen tcp: lookup host-name on 10.0.0.1:53: read udp 10.1.0.1:46736->10.0.0.1:53: i/o timeout |
An address that does not exist was specified to the command line option --web.listen-address. |
Specify the correct host name. |
listen tcp :port: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.
listen tcp :port: bind: address already in use |
A port already in use was specified to the command line option --web.listen-address. |
Specify a port that is not in use. |
|
invalid value "specified-character-string" for flag -timeout-offset: parse error |
An invalid value was specified to the command line option --timeout-offset. |
Review the command line option --timeout-offset specified. |
|
invalid value "specified-value" for flag -timeout-offset: value out of range |
A value out of float64 range was specified to the command line option --timeout-offset. |
Review the command line option --timeout-offset specified. |
open "specified-path": The system cannot find the file specified.
open "specified-path": no such file or directory |
The incorrect path has been specified for the command line option --config.file. |
Review the command line option --config.file specified. |
|
mapping values are not allowed in this context |
Invalid value type for the item found in the file specified for the command line option --config.file. |
Check configuration file specifications for invalid line numbers that appear. |
|
Script parameter is missing |
The incorrect value has been specified for the name setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Review the content specified for the script.name in the Script exporter configuration file (jpc_script_exporter.yml). |
|
Script not found |
The incorrect value has been specified for the script setting in params in the Prometheus configuration file (jpc_prometheus_server.yml). |
Review the content specified for the params.script in the Prometheus configuration file (jpc_prometheus_server.yml). |
|
Script config " script-name" has neither 'script' nor 'command' |
A value has not been specified the command setting in scripts in the Prometheus configuration file (jpc_script_exporter.yml). |
Review the content specified for the scripts.command in the Prometheus configuration file (jpc_script_exporter.yml). |
|
Script failed: exit status 1 |
Failed to run the script specified for the command setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Review the script specified. |
|
Script 'script-name' execution failed
exec: specified-character-string: file does not exist
fork/exec specified-character-string: exec format error |
A file that cannot be executed has been specified for the command setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Review the content specified in the Script exporter configuration file (jpc_script_exporter.yml). |
|
Script 'script-name' execution failed
exec: specified-character-string: Access is denied
fork/exec specified-character-string: permission denied |
A script without execute permissions has been specified for the command setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Change permissions to allow the script specified to be run. |
|
Script 'script-name' execution failed
exec: "specified-character-string": file does not exist
fork/exec: "specified-character-string": no such file or directory |
A script that does not exist has been specified for the command setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Review the content specified for the scripts.command in the Script exporter configuration file (jpc_script_exporter.yml). |
|
Script 'script-name' execution failed
exit status 1
signal: killed |
Execution of the script timed out. |
Review the content specified for the scripts.timeout Script exporter configuration file (jpc_script_exporter.yml). |
(11) Logging Fluentd
|
Message |
Cause |
Action to be taken |
|---|---|---|
|
error="tail: 'path' parameter is required on tail input" |
In text-formatted log file monitoring definition file, path of the [Input Settings] section was not specified. |
Correct text-formatted log file monitoring definition file. |
|
error="Unsupported timezone~" |
In text-formatted log file monitoring definition file, an invalid timezone was specified in the [Input Settings] section.. |
Correct text-formatted log file monitoring definition file. |
|
error="specifying timezone requires time format" |
In text-formatted log file monitoring definition file, timezone is specified without specifying the time_format of the [Input Settings] section.. |
Correct text-formatted log file monitoring definition file. |
|
error="valid options are string,unixtime,float,mixed but got ~" |
In text-formatted log file monitoring definition file, an invalid time_type was specified in the [Input Settings] section. |
Correct text-formatted log file monitoring definition file. |
|
error="unknown encoding name~" |
In text-formatted log file monitoring definition file, an invalid from_encoding was specified in the [Input Settings] section. |
Correct text-formatted log file monitoring definition file. |
|
error="No named captures in 'expression' parameter. The regexp must have at least one named capture" |
In text-formatted log file monitoring definition file, you specified a regular expression that does not include a named capture (?<NAME> PATTERN) in expression of the [Input Settings] section.. |
Correct text-formatted log file monitoring definition file. |
|
error="format should be Regexp, need //,~" |
In text-formatted log file monitoring definition file, the regular expression is specified without a delimiter (/) in the regular expression for parsing the log in the [Input Settings] section. |
Correct text-formatted log file monitoring definition file. |
|
error="valid options are rfc3164,rfc5424,auto but got ~" |
In text-formatted log file monitoring definition file, an invalid message_format was specified in the [Input Settings] section. |
Correct text-formatted log file monitoring definition file. |
|
error="'with_priority' parameter is required but ~ is specified" |
In text-formatted log file monitoring definition file, an invalid with_priority was specified in the [Input Settings] section. |
Correct text-formatted log file monitoring definition file. |
|
"valid options are regexp,string but got~" |
In text-formatted log file monitoring definition file, an invalid parser_type was specified in the [Input Settings] section. |
Correct text-formatted log file monitoring definition file. |
|
error="'support_colonless_ident' parameter is required but ~ is specified" |
In text-formatted log file monitoring definition file, an incorrect support_colonless_ident was specified in the [Input Settings] section. |
Correct text-formatted log file monitoring definition file. |
|
got incomplete JSON array configuration at ~(Fluent::ConfigParseError) |
In text-formatted log file monitoring definition file, a regular expression is specified pattern the [Inclusion Settings] or [Exclusion Settings] section without a delimiter (/). |
Correct text-formatted log file monitoring definition file. |
|
error="Plugin 'tail' does not support multi workers configuration (Fluent::Plugin::TailInput)" |
In text-formatted log file monitoring definition file, when workers parameter is specified to 2 or more, id of worker is specified more than once. Or, if you specify a workers parameter that is greater than or equal to 2, you did not specify the <worker> directive. |
Correct text-formatted log file monitoring definition file. |
|
error="Unknown parser plugin ~. Run 'gem search -rd fluent-plugin' to find plugins" |
In text-formatted log file monitoring definition file, an invalid @type was specified for the <parse> directive in the [Input Settings] section. |
Correct text-formatted log file monitoring definition file. |
|
error="Invalid Bookmark XML is loaded.~" |
In Windows event-log monitoring definition file, an invalid channels was specified in the [Input Settings] section. |
Correct Windows event-log monitoring definition file. |
|
error="Plugin 'windows_eventlog2' does not support multi workers configuration (Fluent::Plugin::WindowsEventLog2Input)" |
In Windows event-log monitoring definition file, when workers parameter is specified to 2 or more, id of worker is specified more than once. Or, if you specify a workers parameter that is greater than or equal to 2, you did not specify the <worker> directive. |
Correct Windows event-log monitoring definition file. |
|
error="valid options are throw_exception,block,drop_oldest_chunk but got~" |
In log monitoring common definition file, an invalid overflow_action was specified in the [Output Settings] section. |
Correct log monitoring common definition file. |
|
valid options are trace,debug,info,warn,error,fatal but got~(Fluent::ConfigParseError) |
In log monitoring common definition file, an invalid value was specified for log_level in the [System Settings] section. |
Correct log monitoring common definition file. |
|
invalid number of workers (must be > 0):0 (Fluent::ConfigError) |
In log monitoring common definition file, an invalid workers parameter was specified in the [System Settings] section. |
Correct log monitoring common definition file. |
|
error="greater first_worker_id<1> than last_worker_id<0> specified by <worker> directive is not allowed. Available multi worker assign syntax is <smaller_worker_id>-<greater_worker_id>" |
In log monitoring common definition file or log metrics definition file, the <worker N-M> directive specifies the number N>M. |
Correct log monitoring common definition file and log metrics definition file. |
|
error="worker id ~ specified by <worker> directive is not allowed. Available worker id is between ~" |
You specified <worker> directive arguments in either log monitoring common definition file, text-formatted log file monitoring definition file, Windows event-log monitoring definition file, or log metrics definition file that are beyond workers parameter. Or, you specify a nonzero id as an argument to the <worker N-M> or <worker> directive without specifying workers parameter. |
Review log monitoring common definition file, text-formatted log file monitoring definition file, Windows event-log monitoring definition file, and log metrics definition file. |
|
error="Missing worker id on <worker> directive" |
You did not specify id of worker as an argument to the <worker> directive in either log monitoring common definition file, text-formatted log file monitoring definition file, Windows event-log monitoring definition file, or log metrics definition file. |
Review log monitoring common definition file, text-formatted log file monitoring definition file, Windows event-log monitoring definition file, and log metrics definition file. |
|
[error]: failed to read data from plugin storage file path=~ |
Storage file is corrupt. |
Remove storage file. Then start it. |
|
Not a directory @ dir_s_mkdir - ~ (Errno::ENOTDIR) |
An invalid monitor name was set. |
Review the monitor name. |
(a) prometheus-client logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
metric name must match /[a-zA-Z_:][a-zA-Z0-9_:]*/ (ArgumentError) |
The name field in <metric> in the log metrics definition file is empty. |
Specify the name field in <metric> in the log metrics definition file. |
|
docstring must be given (ArgumentError) |
The desc field in <metric> in the log metrics definition file is empty. |
Specify the desc field in <metric> in the log metrics definition file. |
|
label label-key must not start with __ (Prometheus::Client::LabelSetValidator::ReservedLabelError) |
In <labels> in the log metrics definition file, the label-key begins with a __ (two half-width underscore characters). |
In <labels> in the log metrics definition file, specify a label-key that does not begin with a __ (two half-width underscore characters). |
|
label name must match /(?-mix:\\A[a-zA-Z_][a-zA-Z0-9_]*\\Z)/ (Prometheus::Client::LabelSetValidator::InvalidLabelError) |
In <labels> in the log metrics definition file, the label key does not match the regular expression /(?-mix:\\A[a-zA-Z_][a-zA-Z0-9_]*\\Z)/. |
In <labels> in the log metrics definition file, specify a label key that matches the regular expression /(?-mix:\\A[a-zA-Z_][a-zA-Z0-9_]*\\Z)/. |
(b) fluent-plugin-prometheus plug-in logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="Missing '@type' parameter on <source> directive" |
In <source> in the log metrics definition file, the @type prometheus setting does not exist. |
In <source> in the log metrics definition file, specify @type prometheus. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::NotFoundPluginError error="Unknown input plugin 'value-specified-for-@type'. Run 'gem search -rd fluent-plugin' to find plugins" |
In <source> in the log metrics definition file, @type is empty, or the value-specified-for-@type is not prometheus. |
When using the log metrics feature, in <source> in the log metrics definition file, set @type to prometheus. |
|
#0 unexpected error error_class=SocketError error="getaddrinfo: Name or service not known" |
In <source> in the log metrics definition file, bind is an invalid value. |
In <source> in the log metrics definition file, specify a value in the correct hostname for bind. |
|
#0 unexpected error error_class=Errno::EADDRNOTAVAIL error="Cannot assign requested address - bind(2) for user-specified-bind: user-specified-port" |
In <source> in the log metrics definition file, bind is an invalid format. |
In <source> in the log metrics definition file, specify the value for bind in the correct IP address format. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="Missing '@type' parameter on <match> directive" |
In <match> in the log metrics definition file, the @type prometheus setting does not exist. |
In <match> in the log metrics definition file, specify @type prometheus. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::NotFoundPluginError error="Unknown output plugin 'value-specified-for-@type'. Run 'gem search -rd fluent-plugin' to find plugins" |
In <match> in the log metrics definition file, @type is empty, or the value-specified-for-@type is not prometheus. |
When using the log metrics feature, in <match> in the log metrics definition file, set @type to prometheus. |
|
section <section-name> is not used in <match> of prometheus plugin |
In <match> in the log metrics definition file, the section-name specified is neither of the following:
|
In <match> in the log metrics definition file, specify either of the following for the section-name:
|
|
unmatched end tag at fluentd_any-name_logmetrics.conf contents (Fluent::ConfigparseError) |
In the log metrics definition file, either the start or end of the section has not been specified. |
Check whether the start and end of the following sections have been defined in the log metrics definition file:
|
|
parameter 'parameter-name' in section |
In the section in the log metrics definition file, a parameter that cannot be used (parameter-name) has been specified. |
In the section in the log metrics definition file, review the parameter (parameter-name) specified. |
|
#0 unknown placeholder `placeholder` found |
The placeholder set in the label section of the log metrics definition file is not a valid placeholder. |
Review the contents of the placeholder set in the label section of the log metrics definition file. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="labels section must have at most 1" |
Multiple <labels> exist in parallel within a single section in the log metrics definition file. |
Edit the log metrics definition file so that there is single <labels> entry within a section. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="type option must be 'counter', 'gauge', 'summary' or 'histogram'" |
In the type field in <metric> in the log metrics definition file, characters strings other than those shown below have been specified, or the field has been left blank.
|
Specify either of the following character strings in the type field in <metric> in the log metrics definition file:
|
|
The type field in <metric> in the log metrics definition file does not exist. |
||
|
#0 prometheus: failed to instrument a metric. error_class=ArgumentError error=#<ArgumentError: comparison of String with 0 failed> tag="user-specified-tag" name="user-specified-log-metric-name" |
The counter-type log metric value defined in the log metrics definition file is neither a Ruby Integer nor a Float class value. |
In <parse> in the log metrics definition file, verify whether expected log message values are extracted, or whether the expected item is specified in the key field in <metric>. |
|
#0 prometheus: failed to instrument a metric. error_class=ArgumentError error=#<ArgumentError: value must be a number> tag="user-specified-tag" name="user-specified-log-metric-name " |
The gauge-type log metric value defined in the log metrics definition file is neither a Ruby Integer nor a Float class value. |
In <parse> in the log metrics definition file, verify whether expected log message values are extracted, or whether the expected item is specified in the key field in <metric>. |
|
#0 prometheus: failed to instrument a metric. error_class=Prometheus::Client::LabelSetValidator::InvalidLabelSetError error=#<Prometheus::Client::LabelSetValidator::InvalidLabelSetError: labels must have the same signature (keys given: [:label-key] vs. keys expected: [:label-key]> tag="user-specified-tag" name="user-specified-log-metric-name" |
In the log metrics definition file, a log metric (user-specified-log-metric-name) with the same name, type, and desc is defined, and these log metrics have different <labels> definitions. |
In the log metrics definition file, review the content of all log metric (user-specified-log-metric-name) <labels> definitions. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="metric requires 'name' option" |
The name setting in <metric> in the log metrics definition file does not exist. |
Specify the name setting in <metric> in the log metrics definition file. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="metric requires 'desc' option" |
The desc setting in <metric> in the log metrics definition file does not exist. |
Specify the desc setting in <metric> in the log metrics definition file. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="gauge metric requires 'key' option" |
Although gauge is specified as the type setting in <metric> in the log metrics definition file, the key setting does not exist. |
Specify the key setting in <metric> in the log metrics definition file. |
|
user-specified-log-metric-name has already been registered as log-metric-type type (Fluent::Plugin::Prometheus::AlreadyRegisteredError) |
In the log metrics definition file, a log metric (user-specified-log-metric-name) with the same name and desc is defined, with different type values specified. |
In the log metrics definition file, review the content of all log metric (user-specified-log-metric-name) type definitions. |
|
user-specified-log-metric-name has already been registered with different docstring |
In the log metrics definition file, a log metric (user-specified-log-metric-name) with the same name and type is defined, with different desc values specified. |
In the log metrics definition file, review the content of all log metric (user-specified-log-metric-name) desc definitions. |