13.3.1 Isolating failures
The following shows the cause and countermeasures for messages output to Process exporter, Fluentd, Script exporter, and Promitor logs. For details on the storage location of each log, see 12.2.2 JP1/IM - Agent log information.
- Organization of this subsection
(1) process_exporter logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
flag needs an argument: command-line-option |
A value has not been specified in the command line option. |
Review the command line option specified. |
|
Failed to start the server: listen tcp: address port: missing port in address |
An invalid port was specified to the command line option --web.listen-address. |
Review the command line option --web.listen-address specified. |
|
Failed to start the server: listen tcp: lookup host-name on DNS: error-details |
An address that does not exist was specified to the command line option --web.listen-address. |
Review the command line option --web.listen-address specified. |
|
Failed to start the server: listen tcp :port: bind: address already in use |
A port already in use was specified to the command line option --web.listen-address. |
Specify a port that is not in use. |
|
-config.path cannot be used with -namemapping or -procnames |
Both the command line options --procnames and --config.path are specified. |
Review the command line option specified. |
|
Both the command line options --namemapping and --config.path are specified. |
||
|
Error parsing -namemapping argument 'specified-character-string': bad namemapper: odd number of tokens |
An odd number of values have been specified for the command line option --namemapping. |
Review the command line option --namemapping specified. |
|
Error parsing -namemapping argument 'specified-character-string': error compling regexp 'invalid-regular-expression': error parsing regexp: error-details |
An invalid regular expression has been specified for the command line option --namemapping. |
Review the command line option --namemapping specified. |
|
error reading config file "specified-path": error reading config file "specified-path": open specified-path: no such file or directory |
A path that does not exist has been specified for the command line option --config.path. |
Review the command line option --config.path specified. |
|
error reading config file "specified-path": error-details |
The format of the file specified to the command line option --config.path is invalid. |
|
|
error reading config file "specified-path": error-details |
The configuration file contains invalid formatting. |
Review the format of the Process exporter configuration file (jpc_process_exporter.yml). |
|
Error initializing: open /proc/stat: permission denied |
Process exporter executed the command without access permissions for the /proc directory, or access permissions for the /proc directory were lost while Process exporter is running. |
Grant read permissions to users running Process exporter for the /proc directory and its subdirectories. If Exporter is stopped, restart the integrated agent. |
|
error reading procs: Error reading procs: Error reading procs: open /proc: permission denied |
(2) Fluentd logs
The following shows logs related to the Fluentd prometheus-client and fluent-plugin-prometheus plug-in, which provide IM Exporter log metrics functionality. For details on other Fluentd logs, see 12.2.2(5) Fluentd log for JP1/IM - Agent.
(a) prometheus-client logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
metric name must match /[a-zA-Z_:][a-zA-Z0-9_:]*/ (ArgumentError) |
The name field in <metric> in the log metrics definition file is empty. |
Specify the name field in <metric> in the log metrics definition file. |
|
docstring must be given (ArgumentError) |
The desc field in <metric> in the log metrics definition file is empty. |
Specify the desc field in <metric> in the log metrics definition file. |
|
label label-key must not start with __ (Prometheus::Client::LabelSetValidator::ReservedLabelError) |
In <labels> in the log metrics definition file, the label-key begins with a __ (two half-width underscore characters). |
In <labels> in the log metrics definition file, specify a label-key that does not begin with a __ (two half-width underscore characters). |
|
label name must match /(?-mix:\\A[a-zA-Z_][a-zA-Z0-9_]*\\Z)/ (Prometheus::Client::LabelSetValidator::InvalidLabelError) |
In <labels> in the log metrics definition file, the label key does not match the regular expression /(?-mix:\\A[a-zA-Z_][a-zA-Z0-9_]*\\Z)/. |
In <labels> in the log metrics definition file, specify a label key that matches the regular expression /(?-mix:\\A[a-zA-Z_][a-zA-Z0-9_]*\\Z)/. |
(b) fluent-plugin-prometheus plug-in logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="Missing '@type' parameter on <source> directive" |
In <source> in the log metrics definition file, the @type prometheus setting does not exist. |
In <source> in the log metrics definition file, specify @type prometheus. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::NotFoundPluginError error="Unknown input plugin 'value-specified-for-@type'. Run 'gem search -rd fluent-plugin' to find plugins" |
In <source> in the log metrics definition file, @type is empty, or the value-specified-for-@type is not prometheus. |
When using the log metrics feature, in <source> in the log metrics definition file, set @type to prometheus. |
|
#0 unexpected error error_class=SocketError error="getaddrinfo: Name or service not known" |
In <source> in the log metrics definition file, bind is an invalid value. |
In <source> in the log metrics definition file, specify a value in the correct hostname for bind. |
|
#0 unexpected error error_class=Errno::EADDRNOTAVAIL error="Cannot assign requested address - bind(2) for user-specified-bind: user-specified-port" |
In <source> in the log metrics definition file, bind is an invalid format. |
In <source> in the log metrics definition file, specify the value for bind in the correct IP address format. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="Missing '@type' parameter on <match> directive" |
In <match> in the log metrics definition file, the @type prometheus setting does not exist. |
In <match> in the log metrics definition file, specify @type prometheus. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::NotFoundPluginError error="Unknown output plugin 'value-specified-for-@type'. Run 'gem search -rd fluent-plugin' to find plugins" |
In <match> in the log metrics definition file, @type is empty, or the value-specified-for-@type is not prometheus. |
When using the log metrics feature, in <match> in the log metrics definition file, set @type to prometheus. |
|
section <section-name> is not used in <match> of prometheus plugin |
In <match> in the log metrics definition file, the section-name specified is neither of the following:
|
In <match> in the log metrics definition file, specify either of the following for the section-name:
|
|
unmatched end tag at fluentd_any-name_logmetrics.conf contents (Fluent::ConfigparseError) |
In the log metrics definition file, either the start or end of the section has not been specified. |
Check whether the start and end of the following sections have been defined in the log metrics definition file:
|
|
parameter 'parameter-name' in section |
In the section in the log metrics definition file, a parameter that cannot be used (parameter-name) has been specified. |
In the section in the log metrics definition file, review the parameter (parameter-name) specified. |
|
#0 unknown placeholder `placeholder` found |
The placeholder set in the label section of the log metrics definition file is not a valid placeholder. |
Review the contents of the placeholder set in the label section of the log metrics definition file. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="labels section must have at most 1" |
Multiple <labels> exist in parallel within a single section in the log metrics definition file. |
Edit the log metrics definition file so that there is single <labels> entry within a section. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="type option must be 'counter', 'gauge', 'summary' or 'histogram'" |
In the type field in <metric> in the log metrics definition file, characters strings other than those shown below have been specified, or the field has been left blank.
|
Specify either of the following character strings in the type field in <metric> in the log metrics definition file:
|
|
The type field in <metric> in the log metrics definition file does not exist. |
||
|
#0 prometheus: failed to instrument a metric. error_class=ArgumentError error=#<ArgumentError: comparison of String with 0 failed> tag="user-specified-tag" name="user-specified-log-metric-name" |
The counter-type log metric value defined in the log metrics definition file is neither a Ruby Integer nor a Float class value. |
In <parse> in the log metrics definition file, verify whether expected log message values are extracted, or whether the expected item is specified in the key field in <metric>. |
|
#0 prometheus: failed to instrument a metric. error_class=ArgumentError error=#<ArgumentError: value must be a number> tag="user-specified-tag" name="user-specified-log-metric-name " |
The gauge-type log metric value defined in the log metrics definition file is neither a Ruby Integer nor a Float class value. |
In <parse> in the log metrics definition file, verify whether expected log message values are extracted, or whether the expected item is specified in the key field in <metric>. |
|
#0 prometheus: failed to instrument a metric. error_class=Prometheus::Client::LabelSetValidator::InvalidLabelSetError error=#<Prometheus::Client::LabelSetValidator::InvalidLabelSetError: labels must have the same signature (keys given: [:label-key] vs. keys expected: [:label-key]> tag="user-specified-tag" name="user-specified-log-metric-name" |
In the log metrics definition file, a log metric (user-specified-log-metric-name) with the same name, type, and desc is defined, and these log metrics have different <labels> definitions. |
In the log metrics definition file, review the content of all log metric (user-specified-log-metric-name) <labels> definitions. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="metric requires 'name' option" |
The name setting in <metric> in the log metrics definition file does not exist. |
Specify the name setting in <metric> in the log metrics definition file. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="metric requires 'desc' option" |
The desc setting in <metric> in the log metrics definition file does not exist. |
Specify the desc setting in <metric> in the log metrics definition file. |
|
config error file="physical-host-installation-directory/ jp1ima/conf/ jpc_fluentd_common.conf" error_class=Fluent::ConfigError error="gauge metric requires 'key' option" |
Although gauge is specified as the type setting in <metric> in the log metrics definition file, the key setting does not exist. |
Specify the key setting in <metric> in the log metrics definition file. |
|
user-specified-log-metric-name has already been registered as log-metric-type type (Fluent::Plugin::Prometheus::AlreadyRegisteredError) |
In the log metrics definition file, a log metric (user-specified-log-metric-name) with the same name and desc is defined, with different type values specified. |
In the log metrics definition file, review the content of all log metric (user-specified-log-metric-name) type definitions. |
|
user-specified-log-metric-name has already been registered with different docstring |
In the log metrics definition file, a log metric (user-specified-log-metric-name) with the same name and type is defined, with different desc values specified. |
In the log metrics definition file, review the content of all log metric (user-specified-log-metric-name) desc definitions. |
(3) script_exporter logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
flag needs an argument: command-line-option |
A value has not been specified in the command line option. |
Review the command line option. |
|
open path: no such file or directory |
A path that does not exist has been specified for the command line option --config.file. |
Review the command line option --config.file specified. |
|
listen tcp: address specified-character-string: missing port in address |
An invalid value was specified to the command line option --web.listen-address. |
Review the command line option --web.listen-address specified. |
listen tcp: lookup host-name : no such host
listen tcp: lookup host-name on 10.0.0.1:53: read udp 10.1.0.1:46736->10.0.0.1:53: i/o timeout |
An address that does not exist was specified to the command line option --web.listen-address. |
Specify the correct host name. |
listen tcp :port: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.
listen tcp :port: bind: address already in use |
A port already in use was specified to the command line option --web.listen-address. |
Specify a port that is not in use. |
|
invalid value "specified-character-string" for flag -timeout-offset: parse error |
An invalid value was specified to the command line option --timeout-offset. |
Review the command line option --timeout-offset specified. |
|
invalid value "specified-value" for flag -timeout-offset: value out of range |
A value out of float64 range was specified to the command line option --timeout-offset. |
Review the command line option --timeout-offset specified. |
open "specified-path": The system cannot find the file specified.
open "specified-path": no such file or directory |
The incorrect path has been specified for the command line option --config.file. |
Review the command line option --config.file specified. |
|
mapping values are not allowed in this context |
Invalid value type for the item found in the file specified for the command line option --config.file. |
Check configuration file specifications for invalid line numbers that appear. |
|
Script parameter is missing |
The incorrect value has been specified for the name setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Review the content specified for the script.name in the Script exporter configuration file (jpc_script_exporter.yml). |
|
Script not found |
The incorrect value has been specified for the script setting in params in the Prometheus configuration file (jpc_prometheus_server.yml). |
Review the content specified for the params.script in the Prometheus configuration file (jpc_prometheus_server.yml). |
|
Script config " script-name" has neither 'script' nor 'command' |
A value has not been specified the command setting in scripts in the Prometheus configuration file (jpc_script_exporter.yml). |
Review the content specified for the scripts.command in the Prometheus configuration file (jpc_script_exporter.yml). |
|
Script failed: exit status 1 |
Failed to run the script specified for the command setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Review the script specified. |
|
Script 'script-name' execution failed
exec: specified-character-string: file does not exist
fork/exec specified-character-string: exec format error |
A file that cannot be executed has been specified for the command setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Review the content specified in the Script exporter configuration file (jpc_script_exporter.yml). |
|
Script 'script-name' execution failed
exec: specified-character-string: Access is denied
fork/exec specified-character-string: permission denied |
A script without execute permissions has been specified for the command setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Change permissions to allow the script specified to be run. |
|
Script 'script-name' execution failed
exec: "specified-character-string": file does not exist
fork/exec: "specified-character-string": no such file or directory |
A script that does not exist has been specified for the command setting in scripts in the Script exporter configuration file (jpc_script_exporter.yml). |
Review the content specified for the scripts.command in the Script exporter configuration file (jpc_script_exporter.yml). |
|
Script 'script-name' execution failed
exit status 1
signal: killed |
Execution of the script timed out. |
Review the content specified for the scripts.timeout Script exporter configuration file (jpc_script_exporter.yml). |
(4) Promitor logs
(a) Scraper logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
Failed to scrape resource collection discovery-group-name: Connection refused (promitor.agents.resourcediscovery:port) |
The Scraper service was started with the Resource Discovery service stopped. |
Start the Resource Discovery Agent. |
|
The incorrect port number was specified for the resourceDiscovery.port in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the resourceDiscovery.port in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
|
AuthorizationFailed: The client 'object-ID' with object id 'object-ID' does not have authorization to perform action 'microsoft.insights/metricDefinitions/read' over scope '/subscriptions/subscription-ID/resourceGroups/resource-group-name /providers/resource-type/resource-name/providers/microsoft.insights' or the scope is invalid. If access was recently granted, please refresh your credentials. |
Specified an identity with insufficient permissions for the Monitoring Reader role. |
Grant permissions that meet or exceed the requirements of the Monitoring Reader role. |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. System.Security.Authentication.AuthenticationException: No identity secret was configured for service principle authentication |
When the authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml) is ServicePrincipal, the Scraper was started with the incorrect value registered as the secret for connecting to Azure Monitor. |
Review the content specified for the AUTH_APPKEY environment variable. |
|
The following problems were found with the metric configuration: <error-details> Validation failed: Errors were found while deserializing the metric configuration. Promitor is not configured correctly. Please fix validation issues and re-run. |
The format of the Promitor Scraper configuration file (metrics-declaration.yaml) is not followed. |
Check error-details for the invalid line number and item name, and review the content specified for the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an out-of-range value was specified for azureMetadata.cloud in the same file. |
||
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value in an invalid format was specified for metricDefault.aggregation.interval. |
||
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metricDefault.scraping.schedule. |
||
|
An out-of-range value was specified for metrics.resourceType in the Promitor Scraper configuration file (metrics-declaration.yaml). |
||
|
A value in an invalid format was specified for metrics.scraping.schedule in the Promitor Scraper configuration file (metrics-declaration.yaml). |
||
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an out-of-range value was specified for metrics.azureMetricConfiguration.aggregation.type. |
||
|
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value in an invalid format was specified for metrics.azureMetricConfiguration.aggregation.interval. |
||
|
Validation failed: Errors were found while deserializing the metric configuration. Promitor is not configured correctly. Please fix validation issues and re-run. |
The format of the Promitor Scraper configuration file (metrics-declaration.yaml) is not followed. |
Review the contents of the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metricDefault.aggregation.interval. |
||
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metricDefault.scraping.schedule. |
||
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metrics.azureMetricConfiguration.aggregation.type. |
||
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for metrics.azureMetricConfiguration.aggregation.interval. |
||
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Unable to deserialize the configured metrics declaration. Please verify that it is a well-formed YAML specification. |
The format of the Promitor Scraper configuration file (metrics-declaration.yaml) is not followed. |
Review the specified contents of the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource for metric 'metric-name' AADSTS900023: Specified tenant identifier 'specified-character-string' is neither a valid DNS name, nor a valid external domain. |
When the authentication.mode is ServicePrincipal in the Promitor Scraper runtime configuration file (runtime.yaml), a value in an invalid format was specified for azureMetadata.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: No tenant id is configured. Promitor is not configured correctly. Please fix validation issues and re-run. |
When resources is not specified by any metric in metrics in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for authentication.tenantId. |
Review the content specified for authentication.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
When resources is specified by a metric in metrics in the Promitor Scraper runtime configuration file (runtime.yaml), a value is not specified for authentication.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for authentication.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
Failed to scrape resource for metric 'metric-name' AADSTS90002: Tenant 'tenant-ID' not found. Check to make sure you have the correct tenant ID and are signing into the correct cloud. Check with your subscription administrator, this may happen if there are no active subscriptions for the tenant. |
A tenant ID that does not exist was specified for azureMetadata.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.tenantId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
InvalidSubscriptionId: The provided subscription identifier 'specified-character-string' is malformed or invalid. |
A value in an invalid format was specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
A value in an invalid format was specified for metrics.resources.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
Validation failed: - No subscription id is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for azureMetadata.subscriptionId. |
Review the content specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for azureMetadata.subscriptionId. |
Review the content specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
SubscriptionNotFound: The subscription 'subscription-ID' could not be found. |
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a subscription ID that does not exist was specified for azureMetadata.subscriptionId. |
Review the content specified for azureMetadata.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
A subscription ID that does not exist was specified for metrics.resources.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.subscriptionId in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
AuthorizationFailed: The client 'object-ID' with object id 'object-ID' does not have authorization to perform action microsoft.insights/metricDefinitions/read' over scope '/subscriptions/subscription-ID/resourceGroups/specified-character-string/providers/resource-type/resource-name/providers/microsoft.insights' or the scope is invalid. If access was recently granted, please refresh your credentials. |
When the authentication.mode is SystemAssignedManagedIdentity in the Promitor Scraper runtime configuration file (runtime.yaml), and metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an invalid value was specified for azureMetadata.resourceGroupName. |
Review the content specified for azureMetadata.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
An invalid value was specified for metrics.resources.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
|
Validation failed: - No resource group name is not configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for azureMetadata.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.resourceGroupName in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Microsoft.IdentityModel.Clients.ActiveDirectory.AdalServiceException: AADSTS90038: Tenant ' tenant-ID ' request is being redirected to the National Cloud 'MicrosoftOnline.COM'. |
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), an out-of-range value was specified for azureMetadata.cloud. |
Review the content specified for azureMetadata.cloud in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape metric 'metric-name' for resource 'resource-name' .System.NotSupportedException: Environment 'cloud-name' is not supported for scraping Azure Log Analytics resource(s) |
When the authentication.mode is ServicePrincipal in the Promitor Scraper runtime configuration file (runtime.yaml), an incorrect cloud name was specified for azureMetadata.cloud in the metrics-declaration.yaml. |
Review the content specified for azureMetadata.cloud in the metrics-declaration.yaml. |
|
Failed to scrape resource for metric 'metric-name' AADSTS500011: The resource principal named https://management.core.cloudapi.de/ was not found in the tenant named tenant-ID. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant. |
When the authentication.mode is SystemAssignedManagedIdentity or UserAssignedManagedIdentity in the Promitor Scraper runtime configuration file (runtime.yaml), an incorrect cloud name was specified for azureMetadata.cloud in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.cloud in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape: Value cannot be null. (Parameter 'key') |
When metrics.resources is specified by a metric in the Promitor Scraper configuration file (metrics-declaration.yaml), interval and type were not specified in metricDefault.aggregation and metrics.azureMetricConfiguration.aggregation. |
Review the content specified for interval and type in metricDefault.aggregation or metrics.azureMetricConfiguration.aggregation in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource collection resource-name: Value cannot be null. (Parameter 'key') |
When metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), interval and type were not specified in metricDefault.aggregation and metrics.azureMetricConfiguration.aggregation. |
Review the content specified for interval and type in metricDefault.aggregation or metrics.azureMetricConfiguration.aggregation in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - Limit has to be at least 1 Promitor is not configured correctly. Please fix validation issues and re-run. |
A value less than 1 was specified for metricDefault.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metricDefault.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
A value less than 1 was specified for metrics.azureMetricConfiguration.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
||
|
Validation failed: - No default metric scraping schedule is defined. - No metrics scraping schedule is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for metricDefault.scraping.schedule in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metricDefault.scraping.schedule in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - No metric name is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for metrics.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - Metric name 'metric-name' is declared multiple times Promitor is not configured correctly. Please fix validation issues and re-run. |
The same value as another metric was specified for metrics.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - No metric name for Azure is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for metrics.azureMetricConfiguration.Name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.Name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource for metric 'metric-name' Promitor.Integrations.AzureMonitor.Exceptions.MetricNotFoundException: The metric 'specified-character-string' was not found |
An invalid value was specified for metrics.azureMetricConfiguration.Name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.Name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - Limit cannot be higher than metricDefault.limit-value |
A value greater than 10000 was specified for metrics.azureMetricConfiguration.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.limit in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
BadRequest: Metric: Available Memory Bytes does not support requested dimension combination: specified-character-string, supported ones are: , TraceId: { trace-ID } |
An invalid value was specified for metrics.azureMetricConfiguration.dimension.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.dimension.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. [resourceDiscoveryGroupName] cannot be Null, empty or white-space. (Parameter 'resourceDiscoveryGroupName') |
A value was not specified for metrics.resourceDiscoveryGroups.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resourceDiscoveryGroups.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource collection specified-character-string: Response status code does not indicate success: 404 (Not Found). |
An invalid value was specified for metrics.resourceDiscoveryGroups.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resourceDiscoveryGroups.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
An invalid value was specified for resourceDiscoveryGroups.name in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for resourceDiscoveryGroups.name in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
|
An invalid value was specified for resourceDiscoveryGroups.type in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for resourceDiscoveryGroups.type in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
|
Validation failed: - No property-name is configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for metrics.resources.property in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.property in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
ResourceNotFound: The Resource 'resource-type/character-string' under resource group 'resource-group-name' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix |
An invalid value was specified for metrics.resources.property in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources.property in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Validation failed: - Queue & topic name are both configured while we only support one or the other. Promitor is not configured correctly. Please fix validation issues and re-run. |
Both the queueName and topicName were specified in metrics.resources when the metrics.resourceType is ServiceBusNamespace in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.resources in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
BadRequest: Metric: Azure-metric-name does not accept zero dimension case |
A value was not specified for name in metrics.azureMetricConfiguration.dimension when the metrics.resourceType is SqlServer in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.dimension.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
BadRequest: Metric: Azure-metric-name does not support requested dimension combination: specified-value, supported ones are: DatabaseResourceId, TraceId: { trace-ID} |
An invalid value was specified for metrics.azureMetricConfiguration.dimension.name when the metrics.resourceType is SqlServer in the Promitor Scraper configuration file (metrics-declaration.yaml). |
Review the content specified for metrics.azureMetricConfiguration.dimension.name in the Promitor Scraper configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource collection discovery-group-name: Response status code does not indicate success: 501 (Not Implemented). |
A value was not specified for resourceDiscoveryGroups.type in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for resourceDiscoveryGroups.type in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to convert configuration value at 'authentication:Mode' to type 'Promitor.Integrations.Azure.Authentication.AuthenticationMode'. |
An invalid value was specified for the authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
Validation failed: Azure authentication is not configured correctly - No identity was configured for service principle authentication Promitor is not configured correctly. Please fix validation issues and re-run. |
When a secret has not been registered for connecting to Azure Monitor, and metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
When Serviceprincipal is specified for authentication.mode in the Promitor Scraper runtime configuration file (runtime.yaml), and metrics.resources is not specified by any metric in the Promitor Scraper configuration file (metrics-declaration.yaml), a value is not specified for authentication.identityId in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.identityId in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
|
Failed to scrape resource for metric 'metric-name' AADSTS700016: Application with identifier 'specified-character-string' was not found in the directory 'Azure-directory-name '. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You may have sent your authentication request to the wrong tenant. Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to bind to address http://[::]:port: address already in use. |
When the authentication.mode is ServicePrincipal in the Promitor Scraper runtime configuration file (runtime.yaml), an invalid value was specified for the authentication.identityId. |
Review the specified content for the authentication.identityId in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
A port that is already in use was specified for the server.httpPort in the Promitor Scraper runtime configuration file (runtime.yaml). |
Specify a port that is not in use for the server.httpPort in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to convert configuration value at 'server:HttpPort' to type 'System.Int32'. |
An invalid value was specified for the server.httpPort in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the server.httpPort in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Could not find file 'path'. |
An invalid value was specified for the metricsConfiguration.absolutePath in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the metricsConfiguration.absolutePath in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Could not find a part of the path '/config/metrics-declaration.yaml'. |
A value was not specified for metricsConfiguration.absolutePath in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the metricsConfiguration.absolutePath in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
Promitor Scraper Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to convert configuration value at 'resourceDiscovery:Port' to type 'System.Int32'. |
An invalid value was specified for the resourceDiscovery.port in the Promitor Scraper runtime configuration file (runtime.yaml). |
Review the specified content for the resourceDiscovery.port in the Promitor Scraper runtime configuration file (runtime.yaml). |
|
AuthorizationFailed: The client 'client-ID' with object id 'object-ID' does not have authorization to perform action 'microsoft.insights/metricDefinitions/read' over scope 'resource-Uri' or the scope is invalid. If access was recently granted, please refresh your credentials |
When the authentication.mode is ServicePrincipal in the Promitor Scraper runtime configuration file (runtime.yaml), and metrics.resources is specified in the Promitor configuration file (metrics-declaration.yaml), an invalid value was specified for azureMetadata.resourceGroupName in the Promitor configuration file (metrics-declaration.yaml). |
Review the content specified for azureMetadata.resourceGroupName in the Promitor configuration file (metrics-declaration.yaml). |
|
Failed to scrape resource for metric 'metric-name' System.Net.Http.HttpRequestException: Resource temporarily unavailable (host-name:port-number) |
The Scraper is disconnected from the network. |
Connect the network. |
|
Failed to discover resources for group resource-group-name. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. |
The Scraper attempt to retrieve information from the Resource Discovery who is disconnected from the network. |
Connect the network. |
|
Failed to scrape resource for metric 'metric-name' System.Net.Http.HttpRequestException: Name or service not known (host-name:port-number) |
The Scraper was started with the incorrect proxy. |
Review the proxy settings. |
|
Failed to discover resources for group kubernetes-service-group. System.Net.Http.HttpRequestException: Response status code does not indicate success: 500 (Internal Server Error). |
The Scraper attempt to retrieve information from the Resource Discovery with an incorrect proxy configured. |
Review the proxy settings. |
(b) Resource Discovery logs
|
Message |
Cause |
Countermeasures |
|---|---|---|
|
Unhandled exception in job Azure Subscription Discovery One or more errors occurred. ([subscriptions] cannot be Null. (Parameter 'subscriptions')) |
Specified an identity with insufficient permissions for the Reader role. |
Grant permissions that meet or exceed the requirements of the Reader role. |
|
ClientSecretCredential authentication failed: AADSTS700016: Application with identifier 'specified-character-string' was not found in the directory 'Azure-directory-name'. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You may have sent your authentication request to the wrong tenant. |
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), Resource Discovery was started with an incorrect value registered as the secret for connecting to Azure Resource Graph. |
Review the secret for connecting to Azure Resource Graph. |
|
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), an invalid value was specified for the authentication.identityId. |
Review the specified content for the authentication.identityId in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
|
Unhandled exception in job Azure Subscription Discovery One or more errors occurred. (ClientSecretCredential authentication failed: AADSTS900023: Specified tenant identifier 'specified-character-string' is neither a valid DNS name, nor a valid external domain. |
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), a value in an invalid format was specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Unhandled exception in job Azure Subscription Discovery One or more errors occurred. (ClientSecretCredential authentication failed: AADSTS90002: Tenant 'specified-character-string' not found. Check to make sure you have the correct tenant ID and are signing into the correct cloud. Check with your subscription administrator, this may happen if there are no active subscriptions for the tenant. |
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), a tenant ID that does not exist was specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Validation failed: - No tenant id was configured Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.tenantId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Operation returned an invalid status code 'BadRequest' |
A value in an invalid format was specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
[queriedSubscriptions] cannot be Null. (Parameter 'queriedSubscriptions') |
A subscription ID that does not exist was specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Validation failed: - No subscription id(s) were configured to query Promitor is not configured correctly. Please fix validation issues and re-run. |
A value was not specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for azureLandscape.subscriptions.subscriptionId in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Promitor Discovery Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Value cannot be null. (Parameter 'text') |
A value was not specified for resourceDiscoveryGroups.name in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
Review the content specified for resourceDiscoveryGroups.name in the Promitor Resource Discovery configuration file (resource-discovery-declaration.yaml). |
|
Promitor Discovery Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to convert configuration value at 'authentication:Mode' to type 'Promitor.Integrations.Azure.Authentication.AuthenticationMode'. |
An invalid value was specified for the authentication.mode in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.mode in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
Validation failed: Azure authentication is not configured correctly - No identity was configured for service principle authentication Promitor is not configured correctly. Please fix validation issues and re-run. |
A secret has not been registered for connecting to Azure Resource Graph, and a value is not specified for authentication.mode in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
Review the specified content for the authentication.mode in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
When the authentication.mode is ServicePrincipal in the Promitor Resource Discovery runtime configuration file (runtime.yaml), a value was not specified for the authentication.identityId. |
Review the specified content for the authentication.identityId in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
|
Promitor Discovery Agent has encountered an unexpected error. Please open an issue at https://github.com/tomkerkhove/promitor/issues to let us know about it. Failed to bind to address http://[::]:port: address already in use. |
A port that is already in use was specified for the server.httpPort in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
Review the specified content for the server.httpPort in the Promitor Resource Discovery runtime configuration file (runtime.yaml). |
|
ClientSecretCredential authentication failed: Retry failed after 2 tries. Retry settings can be adjusted in ClientOptions.Retry. (Resource temporarily unavailable (host-name:port-number)) (The operation was canceled.) |
The Scraper and the Resource Discovery started while disconnected from the network. |
Connect the network. |
|
Unhandled exception in job Azure Subscription Discovery System.AggregateException: One or more errors occurred. (ClientSecretCredential authentication failed: Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry. (Resource temporarily unavailable (host-name:port-number)) (Resource temporarily unavailable (host-name:port-number)) (Resource temporarily unavailable (host-name:port-number)) (Resource temporarily unavailable (host-name:port-number))) |
The Resource Discovery started while disconnected from the network. |
Connect the network. |
|
Unhandled exception in job Azure Subscription Discovery System.AggregateException: One or more errors occurred. (Resource temporarily unavailable (host-name:port-number)) |
It was disconnected from the network while the Resource Discovery was running. |
Connect the network. |
|
ClientSecretCredential authentication failed: Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry. (Name or service not known (host-name:port-number)) (Name or service not known (host-name:port-number)) (Name or service not known (host-name:port-number)) (Name or service not known (host-name:port-number)) |
The Resource Discovery was started with the incorrect proxy. |
Review the proxy settings. |