Supported telemetry types
The Concourse CI integration supports these telemetry types:Choose a collection method
The CXDOT Collector collects Concourse metrics in one of two ways:- Recommended: Turn on the Prometheus emitter on each web node and scrape its endpoint with the Concourse CI integration. You get the Concourse CI integration’s metrics, its overview dashboard, and a health metric for each web node. The Prerequisites and Configure sections cover this method.
- Alternative: Keep the Datadog emitter and send its DogStatsD metrics to the Datadog integration. Choose this method if you can’t switch a web node to the Prometheus emitter yet. You get the metric names that the emitter sends, and none of the Concourse CI integration’s metrics, dashboard, or health metric. The Send DogStatsD metrics to the Datadog integration section covers this method.
Prerequisites
The Concourse CI integration has the following prerequisites:-
Turn on the Prometheus metrics emitter on each Concourse web node by setting
CONCOURSE_PROMETHEUS_BIND_IPandCONCOURSE_PROMETHEUS_BIND_PORT, or the--prometheus-bind-ipand--prometheus-bind-portflags ofconcourse web. With the Concourse Helm chart, setconcourse.web.prometheus.enabledtotrue, which binds port9391and creates a Service for it. For more information, see Metrics in the Concourse documentation. A web node runs one metrics emitter, and a web node with more than one emitter configured fails to start with the errormultiple emitters configured. To turn on the Prometheus emitter on a web node that sends its metrics through another emitter, remove that emitter’s settings. The Prometheus emitter serves an endpoint that any number of readers can request, so move whatever reads the node’s metrics today to the Prometheus endpoint when you switch. To confirm that a web node serves metrics, request/metricson the Prometheus port from a machine that can reach it and confirm that the request returns a successful status code: - Make the Prometheus port reachable on each web node from the CXDOT Collector.
Configure
The Concourse CI integration is enabled by default, but it collects nothing until you configure targets. Add a CXDOT integration annotation to your Concourse web pods or to a Service in front of them, or list the web nodes to monitor. To configure the Concourse CI integration, follow these steps:-
To collect from Concourse web pods in Kubernetes, add a CXDOT integration annotation
to the pod template. The annotation key must end with the container name. For
example, add the following annotation to your Concourse web workload:
The Collector replaces
${target.host}with the pod IP address and${target.port.prometheus}with the container port namedprometheus. Metrics from each annotated pod include that pod’s Kubernetes metadata. With the Concourse Helm chart, add the annotation underweb.annotations. The chart names the web containerRELEASE_NAME-weband its Prometheus portprometheus, so end the annotation key with the container name that your release produces. -
Alternatively, to collect from web nodes outside your Kubernetes cluster or at
addresses that you choose, list each web node in
endpoints. For example, add the following to the Helm values file for your CXDOT Collector:When you list endpoints, the integration collects from exactly those endpoints instead of from annotated pods and Services. In Kubernetes, the elected cluster Collector collects from each endpoint that you list, one time rather than once per node. For more information, see architecture. List an address that reaches one web node. Each request to an address that balances across replicas, such as a Kubernetes Service in front of several web pods, reaches a different replica, so the metrics mix the replicas’ values. With one web node, the Service that the Concourse Helm chart creates for the Prometheus port reaches that node. For a Concourse deployment with several web nodes, list each node’s own address. Metrics from listed endpoints don’t carry Kubernetes metadata from the Concourse pods. Use theserver.addressandserver.portattributes to identify each web node. -
Optional: To collect from both annotated pods and listed endpoints, list the
endpoints in a second integration instance. For example, add the following to the
Helm values file for your CXDOT Collector:
The
concourse_ci/edgekey defines a named instance for the listed endpoints. The bareconcourse_cikey lists no endpoints, so the integration continues to collect from annotated pods and Services. -
Optional: If you can’t change the pod template, configure the Service in front of
the Prometheus port with a CXDOT integration annotation instead. The
integrations.cxdot.paloaltonetworks.com/endpointskey makes each ready web pod behind the Service a separate target, so the metrics don’t mix the replicas’ values. For example, add the following annotation to the Prometheus Service of your Concourse deployment:The Collector replaces${target.host}with the IP address of each web pod and${target.port.prometheus}with the pod port that the Service port namedprometheusforwards to. The Collector reads these annotations only whenendpointsis empty. -
Optional: If the OpenMetrics integration also scrapes your Concourse web pods or
Service, disable OpenMetrics on that workload to avoid duplicate metrics. For
example, set
openmetrics: {enabled: false}in a CXDOT integration annotation on the pod or Service. For more information, see autodiscovery.
Validate
To validate the Concourse CI integration, follow these steps:-
In the Live Telemetry Analyzer,
filter for
__name__=cxdot.integration.target.healthandcxdot.integration.name=concourse_ci. Confirm that each web node reports1, identified by itsserver.addressandserver.portattributes. -
In Metrics Explorer,
run the following query after Concourse finishes builds:
Confirm that the query returns a time series for each web node and build status.
Known limitations
- Concourse emits
concourse.api.concurrent_request.countandconcourse.api.concurrent_request.limit_hitsonly for API actions with a configured concurrent request limit. - Concourse emits
concourse.log.errorsonly when error metric capture is enabled.
Send DogStatsD metrics to the Datadog integration
SetCONCOURSE_DATADOG_AGENT_HOST and CONCOURSE_DATADOG_AGENT_PORT, or the
--datadog-agent-host and --datadog-agent-port flags of concourse web, to the
DogStatsD listener of the
Datadog
integration. The listener accepts UDP on port 9125 by default.
The Concourse CI integration doesn’t collect these DogStatsD metrics. The emitter sends
each metric as a gauge named with the --datadog-prefix value followed by the event
name, such as concourse.ci.builds_running with the Concourse Helm chart’s default
prefix. The metric names on this page and its dashboard don’t apply to those metrics.
Configuration reference
Configure one Concourse CI integration instance with the following settings. In Helm values, place these settings undercollection.cxdot.integrations.concourse_ci. In a Collector
configuration file, place them under cxdot.integrations.concourse_ci.
Optional settings
-
enabledType:boolean. Optional. Default:true. Whether to enable this Concourse CI integration instance. If true, the Collector collects metrics from the web nodes that this instance is configured for. If false, the Collector doesn’t run this integration instance. -
endpointsType:array of object. Optional. Default:[]. Concourse web nodes to monitor, by the address of each node’s Prometheus metrics endpoint. The integration instance collects only from the endpoints listed here. Specifyhost:portto request the/metricspath over HTTP, or a full HTTP or HTTPS URL. For a URL without a path, the Collector requests/metrics. A web node serves the endpoint only when its Prometheus emitter is on, set withCONCOURSE_PROMETHEUS_BIND_IPandCONCOURSE_PROMETHEUS_BIND_PORT. The Concourse Helm chart uses port9391. Each entry is one target, so list an address that reaches one web node. An address that balances requests across replicas, such as a Kubernetes Service with several pods, reaches a different replica on each collection. -
endpoints[].endpointType:string. Required. Address of one Concourse web node’s Prometheus metrics endpoint, inhost:portformat or as a full HTTP or HTTPS URL. Forhost:port, the Collector requests the/metricspath over HTTP. A full URL is requested as given, such ashttp://concourse-web.example.com:9391/metrics. For a URL without a path, the Collector requests/metrics. -
collection_intervalType:duration. Optional. Default:15s. How often the integration collects metrics from each Concourse web node. -
timeoutType:duration. Optional. Default:10s. Maximum time the Collector waits for one collection from a web node. The value must be greater than zero. A value longer thancollection_intervalis lowered tocollection_interval.