Supported telemetry types
The Elasticsearch integration supports these telemetry types:Prerequisites
The Elasticsearch integration has the following prerequisites:- If Elasticsearch security is enabled, create a monitoring user with the
monitorormanagecluster privilege. Themonitorprivilege provides the required read-only access. For more information, see Elasticsearch security privileges. - Make the Elasticsearch HTTP API reachable from the CXDOT Collector.
Configure
The Elasticsearch integration is enabled by default. You can configure targets either by annotating pods in your Kubernetes cluster or by listing static endpoints in your Helm values. To configure the integration, follow these steps:-
To collect from Elasticsearch pods in Kubernetes, add a CXDOT integration annotation
to the pod template. The annotation key must end with the container name, and its
value can reference credentials and a certificate authority stored in a Kubernetes
Secret in the same namespace. For example, add the following annotation to your
Elasticsearch workload:
The Collector replaces
${target.host}with the pod IP address,${target.port.http}with the container port namedhttp, and${k8s_secret:elasticsearch-monitoring/password}and${k8s_secret:elasticsearch-monitoring/ca.crt}with thepasswordandca.crtkeys of theelasticsearch-monitoringSecret. Elasticsearch enables security and HTTPS by default, so the endpoint useshttps://andca.crtholds the certificate authority that signed the Elasticsearch HTTP certificate. To allow the Collector to read the Secret, add the pod’s namespace toapiServer.metadataServer.secretAllowlistin the Helm values for your CXDOT Collector. The Collector also discovers Elasticsearch pods that carry theapp.kubernetes.io/name: elasticsearchlabel and expose an unsecured HTTP API on port9200. A CXDOT integration annotation takes precedence over the label for its container. To stop collecting from a container, setelasticsearch: {enabled: false}in the annotation. Metrics from each discovered pod include that pod’s Kubernetes metadata. For more information, see autodiscovery. -
Optional: To collect from Elasticsearch servers outside your Kubernetes cluster,
configure static targets. Supply the password
through your deployment’s secret management. For example, add the following to
the
values.yamlfor your CXDOT Collector Helm chart:When you configure static targets, this integration instance collects from exactly those targets instead of discovered targets. In Kubernetes cluster mode, the elected cluster Collector scrapes static targets. For more information, see architecture. -
Optional: Configure a second integration instance to collect from both annotated
pods and static targets. For example, add the following to the
values.yamlfor your CXDOT Collector Helm chart:Theelasticsearch/edgekey defines a named instance for the static target. The bareelasticsearchkey continues to discover pods. -
Optional: Disable the integration. For example, add the following to the
values.yamlfor your CXDOT Collector Helm chart:
Validate
To validate the Elasticsearch integration, follow these steps:-
In
Live Telemetry Analyzer,
filter for
__name__=cxdot.integration.target.healthandcxdot.integration.name=elasticsearch. Confirm that each reachable Elasticsearch target reports1. A value of0means the Collector hit an error collecting from that target. Thecxdot.integration.target,server.address, andserver.portattributes identify the target. -
In
Metrics Explorer,
run the following query after Elasticsearch has indexed documents:
With the default
nodes: [_local]setting, confirm that the query returns one time series per target and that each value is greater than0once the cluster has indexed documents. Each value counts active documents on the Elasticsearch node that answered that target’s scrape, not the whole cluster.
Configuration reference
Configure one Elasticsearch integration instance with the following settings. In Helm values, place these settings undercollection.cxdot.integrations.elasticsearch. In a Collector
configuration file, place them under cxdot.integrations.elasticsearch.
Optional settings
-
enabledType:boolean. Optional. Default:true. Whether to enable this Elasticsearch integration instance. If true, the Collector collects Elasticsearch metrics. If false, the Collector doesn’t run this integration instance. -
endpointsType:array of object. Optional. Default:[]. Elasticsearch targets. Eachendpointis eitherhost:portfor an HTTP connection or a full HTTP or HTTPS URL. When this list contains targets, this integration instance collects from exactly those targets and doesn’t use discovery. When the list is empty or omitted, the integration discovers Elasticsearch targets from Kubernetes pod labels and discovery annotations. Configure a second named integration instance to use both listed targets and discovery. -
endpoints[].endpointType:string. Required. Elasticsearch target as eitherhost:portfor an HTTP connection or a full HTTP or HTTPS URL. -
usernameType:string. Optional. Username for HTTP basic authentication to the targets inendpoints. Setusernameandpasswordtogether. Targets discovered through discovery annotations use the credentials in their annotations. -
passwordType:string. Optional. Password for HTTP basic authentication to the targets inendpoints. Setusernameandpasswordtogether. The Collector masks this value in diagnostic output, logs, and errors. Targets discovered through discovery annotations use the credentials in their annotations. -
collection_intervalType:duration. Optional. Default:15s. How often the Collector collects metrics from each Elasticsearch target. -
timeoutType:duration. Optional. Default:10s. Maximum time allowed to collect metrics from one Elasticsearch target. The value must not exceedcollection_interval. -
nodesType:array of string. Optional. Default:["_local"]. Elasticsearch node selectors to collect. Use_localto collect the node that receives the request,_allto collect every node in the cluster, or Elasticsearch node names and selector expressions. An empty list disables node metrics. -
skip_cluster_metricsType:boolean. Optional. Default:false. If true, don’t collect cluster metrics, including pending task and index template metrics. If false, collect cluster metrics. -
indicesType:array of string. Optional. Default:[]. Elasticsearch index names and index selector expressions to collect. Use_allto collect every index. An empty list disables index metrics. -
pending_tasksType:boolean. Optional. Default:true. If true, collect the number, priority, and average queue time of pending cluster tasks. If false, don’t request pending task details or emit pending task metrics. -
tlsType:object. Optional. Transport Layer Security (TLS) settings for the HTTPS targets inendpoints. The Collector verifies server certificates by default. Targets discovered through annotations use the TLS settings in their annotations. -
tls.ca_fileType:string. Optional. Path to a PEM-encoded certificate authority (CA) certificate file used to verify the Elasticsearch server certificate. If omitted, the Collector uses the system certificate authority pool. -
tls.ca_pemType:string. Optional. PEM-encoded certificate authority certificate content used to verify the Elasticsearch server certificate. -
tls.cert_fileType:string. Optional. Path to the PEM-encoded client certificate file to present when the Elasticsearch endpoint requires mutual TLS authentication. -
tls.cert_pemType:string. Optional. PEM-encoded client certificate content to present when the Elasticsearch endpoint requires mutual TLS authentication. -
tls.cipher_suitesType:array of string. Optional. TLS cipher suites that the Collector can use, in preference order. If omitted, the Collector uses the secure defaults from the Go TLS library. For supported names, see https://go.dev/src/crypto/tls/cipher_suites.go. -
tls.curve_preferencesType:array of string. Optional. Elliptic curves that the Collector can use for an elliptic curve Diffie-Hellman ephemeral handshake, in preference order. If omitted, the Collector uses the Go TLS library defaults. -
tls.include_system_ca_certs_poolType:boolean. Optional. Whether to add certificates from the system certificate authority pool to the configured certificate authorities. If true, the Collector trusts both sources. If false, the Collector trusts only the configured certificate authorities. -
tls.insecureType:boolean. Optional. Whether to disable transport security. If true, the Collector uses an unencrypted connection. If false, the Collector uses TLS for HTTPS endpoints. -
tls.insecure_skip_verifyType:boolean. Optional. Whether to skip verification of the Elasticsearch server certificate. If true, the connection is encrypted but isn’t authenticated and is vulnerable to man-in-the-middle attacks. If false, the Collector verifies the certificate. -
tls.key_fileType:string. Optional. Path to the PEM-encoded client private key file used withcert_filefor mutual TLS authentication. -
tls.key_pemType:string. Optional. PEM-encoded client private key content used withcert_pemfor mutual TLS authentication. -
tls.max_versionType:string. Optional. Maximum TLS protocol version the Collector can use. If omitted, the Collector uses the maximum version supported by the Go TLS library. -
tls.min_versionType:string. Optional. Minimum TLS protocol version the Collector can use. If omitted, the minimum version is TLS 1.2. -
tls.reload_intervalType:duration. Optional. How often the Collector reloads certificate and key files. If omitted, the Collector doesn’t reload the files. -
tls.server_name_overrideType:string. Optional. Server name to verify in the Elasticsearch certificate instead of the endpoint hostname. Use this setting when the certificate name differs from the endpoint hostname. -
tls.tpmType:object. Optional. Trusted platform module settings for the TLS private key. -
tls.tpm.authType:string. Optional. Authorization value for the trusted platform module key. -
tls.tpm.enabledType:boolean. Optional. Whether to use a trusted platform module for the TLS private key. If true, the Collector loads the key from the configured device or socket. If false, the Collector uses the configured key file or in-memory key. -
tls.tpm.owner_authType:string. Optional. Owner authorization value for the trusted platform module. -
tls.tpm.pathType:string. Optional. Path to the trusted platform module device or Unix domain socket, such as/dev/tpm0or/dev/tpmrm0.