Skip to main content
The Elasticsearch integration requires CXDOT Collector 1.7.0 or greater. Elasticsearch is a distributed search and analytics engine. Use the Elasticsearch integration with the CXDOT Collector to collect cluster, node, operating system, process, and Java Virtual Machine metrics from Elasticsearch servers running in your environment. The Elasticsearch integration supports Elasticsearch 7.9 or later.

Supported telemetry types

The Elasticsearch integration supports these telemetry types:

Prerequisites

The Elasticsearch integration has the following prerequisites:
  • If Elasticsearch security is enabled, create a monitoring user with the monitor or manage cluster privilege. The monitor privilege provides the required read-only access. For more information, see Elasticsearch security privileges.
  • Make the Elasticsearch HTTP API reachable from the CXDOT Collector.

Configure

The Elasticsearch integration is enabled by default. You can configure targets either by annotating pods in your Kubernetes cluster or by listing static endpoints in your Helm values. To configure the integration, follow these steps:
  1. To collect from Elasticsearch pods in Kubernetes, add a CXDOT integration annotation to the pod template. The annotation key must end with the container name, and its value can reference credentials and a certificate authority stored in a Kubernetes Secret in the same namespace. For example, add the following annotation to your Elasticsearch workload:
    The Collector replaces ${target.host} with the pod IP address, ${target.port.http} with the container port named http, and ${k8s_secret:elasticsearch-monitoring/password} and ${k8s_secret:elasticsearch-monitoring/ca.crt} with the password and ca.crt keys of the elasticsearch-monitoring Secret. Elasticsearch enables security and HTTPS by default, so the endpoint uses https:// and ca.crt holds the certificate authority that signed the Elasticsearch HTTP certificate. To allow the Collector to read the Secret, add the pod’s namespace to apiServer.metadataServer.secretAllowlist in the Helm values for your CXDOT Collector. The Collector also discovers Elasticsearch pods that carry the app.kubernetes.io/name: elasticsearch label and expose an unsecured HTTP API on port 9200. A CXDOT integration annotation takes precedence over the label for its container. To stop collecting from a container, set elasticsearch: {enabled: false} in the annotation. Metrics from each discovered pod include that pod’s Kubernetes metadata. For more information, see autodiscovery.
  2. Optional: To collect from Elasticsearch servers outside your Kubernetes cluster, configure static targets. Supply the password through your deployment’s secret management. For example, add the following to the values.yaml for your CXDOT Collector Helm chart:
    When you configure static targets, this integration instance collects from exactly those targets instead of discovered targets. In Kubernetes cluster mode, the elected cluster Collector scrapes static targets. For more information, see architecture.
  3. Optional: Configure a second integration instance to collect from both annotated pods and static targets. For example, add the following to the values.yaml for your CXDOT Collector Helm chart:
    The elasticsearch/edge key defines a named instance for the static target. The bare elasticsearch key continues to discover pods.
  4. Optional: Disable the integration. For example, add the following to the values.yaml for your CXDOT Collector Helm chart:

Validate

To validate the Elasticsearch integration, follow these steps:
  1. In Live Telemetry Analyzer, filter for __name__=cxdot.integration.target.health and cxdot.integration.name=elasticsearch. Confirm that each reachable Elasticsearch target reports 1. A value of 0 means the Collector hit an error collecting from that target. The cxdot.integration.target, server.address, and server.port attributes identify the target.
  2. In Metrics Explorer, run the following query after Elasticsearch has indexed documents:
    With the default nodes: [_local] setting, confirm that the query returns one time series per target and that each value is greater than 0 once the cluster has indexed documents. Each value counts active documents on the Elasticsearch node that answered that target’s scrape, not the whole cluster.

Configuration reference

Configure one Elasticsearch integration instance with the following settings. In Helm values, place these settings under collection.cxdot.integrations.elasticsearch. In a Collector configuration file, place them under cxdot.integrations.elasticsearch.

Optional settings

  • enabled Type: boolean. Optional. Default: true. Whether to enable this Elasticsearch integration instance. If true, the Collector collects Elasticsearch metrics. If false, the Collector doesn’t run this integration instance.
  • endpoints Type: array of object. Optional. Default: []. Elasticsearch targets. Each endpoint is either host:port for an HTTP connection or a full HTTP or HTTPS URL. When this list contains targets, this integration instance collects from exactly those targets and doesn’t use discovery. When the list is empty or omitted, the integration discovers Elasticsearch targets from Kubernetes pod labels and discovery annotations. Configure a second named integration instance to use both listed targets and discovery.
  • endpoints[].endpoint Type: string. Required. Elasticsearch target as either host:port for an HTTP connection or a full HTTP or HTTPS URL.
  • username Type: string. Optional. Username for HTTP basic authentication to the targets in endpoints. Set username and password together. Targets discovered through discovery annotations use the credentials in their annotations.
  • password Type: string. Optional. Password for HTTP basic authentication to the targets in endpoints. Set username and password together. The Collector masks this value in diagnostic output, logs, and errors. Targets discovered through discovery annotations use the credentials in their annotations.
  • collection_interval Type: duration. Optional. Default: 15s. How often the Collector collects metrics from each Elasticsearch target.
  • timeout Type: duration. Optional. Default: 10s. Maximum time allowed to collect metrics from one Elasticsearch target. The value must not exceed collection_interval.
  • nodes Type: array of string. Optional. Default: ["_local"]. Elasticsearch node selectors to collect. Use _local to collect the node that receives the request, _all to collect every node in the cluster, or Elasticsearch node names and selector expressions. An empty list disables node metrics.
  • skip_cluster_metrics Type: boolean. Optional. Default: false. If true, don’t collect cluster metrics, including pending task and index template metrics. If false, collect cluster metrics.
  • indices Type: array of string. Optional. Default: []. Elasticsearch index names and index selector expressions to collect. Use _all to collect every index. An empty list disables index metrics.
  • pending_tasks Type: boolean. Optional. Default: true. If true, collect the number, priority, and average queue time of pending cluster tasks. If false, don’t request pending task details or emit pending task metrics.
  • tls Type: object. Optional. Transport Layer Security (TLS) settings for the HTTPS targets in endpoints. The Collector verifies server certificates by default. Targets discovered through annotations use the TLS settings in their annotations.
  • tls.ca_file Type: string. Optional. Path to a PEM-encoded certificate authority (CA) certificate file used to verify the Elasticsearch server certificate. If omitted, the Collector uses the system certificate authority pool.
  • tls.ca_pem Type: string. Optional. PEM-encoded certificate authority certificate content used to verify the Elasticsearch server certificate.
  • tls.cert_file Type: string. Optional. Path to the PEM-encoded client certificate file to present when the Elasticsearch endpoint requires mutual TLS authentication.
  • tls.cert_pem Type: string. Optional. PEM-encoded client certificate content to present when the Elasticsearch endpoint requires mutual TLS authentication.
  • tls.cipher_suites Type: array of string. Optional. TLS cipher suites that the Collector can use, in preference order. If omitted, the Collector uses the secure defaults from the Go TLS library. For supported names, see https://go.dev/src/crypto/tls/cipher_suites.go.
  • tls.curve_preferences Type: array of string. Optional. Elliptic curves that the Collector can use for an elliptic curve Diffie-Hellman ephemeral handshake, in preference order. If omitted, the Collector uses the Go TLS library defaults.
  • tls.include_system_ca_certs_pool Type: boolean. Optional. Whether to add certificates from the system certificate authority pool to the configured certificate authorities. If true, the Collector trusts both sources. If false, the Collector trusts only the configured certificate authorities.
  • tls.insecure Type: boolean. Optional. Whether to disable transport security. If true, the Collector uses an unencrypted connection. If false, the Collector uses TLS for HTTPS endpoints.
  • tls.insecure_skip_verify Type: boolean. Optional. Whether to skip verification of the Elasticsearch server certificate. If true, the connection is encrypted but isn’t authenticated and is vulnerable to man-in-the-middle attacks. If false, the Collector verifies the certificate.
  • tls.key_file Type: string. Optional. Path to the PEM-encoded client private key file used with cert_file for mutual TLS authentication.
  • tls.key_pem Type: string. Optional. PEM-encoded client private key content used with cert_pem for mutual TLS authentication.
  • tls.max_version Type: string. Optional. Maximum TLS protocol version the Collector can use. If omitted, the Collector uses the maximum version supported by the Go TLS library.
  • tls.min_version Type: string. Optional. Minimum TLS protocol version the Collector can use. If omitted, the minimum version is TLS 1.2.
  • tls.reload_interval Type: duration. Optional. How often the Collector reloads certificate and key files. If omitted, the Collector doesn’t reload the files.
  • tls.server_name_override Type: string. Optional. Server name to verify in the Elasticsearch certificate instead of the endpoint hostname. Use this setting when the certificate name differs from the endpoint hostname.
  • tls.tpm Type: object. Optional. Trusted platform module settings for the TLS private key.
  • tls.tpm.auth Type: string. Optional. Authorization value for the trusted platform module key.
  • tls.tpm.enabled Type: boolean. Optional. Whether to use a trusted platform module for the TLS private key. If true, the Collector loads the key from the configured device or socket. If false, the Collector uses the configured key file or in-memory key.
  • tls.tpm.owner_auth Type: string. Optional. Owner authorization value for the trusted platform module.
  • tls.tpm.path Type: string. Optional. Path to the trusted platform module device or Unix domain socket, such as /dev/tpm0 or /dev/tpmrm0.