Skip to main content
The Hasura integration requires CXDOT Collector 1.13.0 or greater. Hasura GraphQL Engine serves a GraphQL API over your databases and services. Use the Hasura integration to collect GraphQL request, subscription, event trigger, and PostgreSQL connection metrics from Hasura GraphQL Engine Enterprise Edition instances running in your environment. The Hasura integration supports Hasura GraphQL Engine 2.36 or greater.

Supported telemetry types

The Hasura integration supports these telemetry types:

Prerequisites

The Hasura integration has the following prerequisites:
  • Run Hasura GraphQL Engine Enterprise Edition with an active license. Community Edition and unlicensed Enterprise Edition don’t serve metrics.
  • Enable the metrics API on each Hasura instance by adding metrics to HASURA_GRAPHQL_ENABLED_APIS, such as HASURA_GRAPHQL_ENABLED_APIS=metadata,graphql,pgdump,config,metrics. If the instance sets HASURA_GRAPHQL_METRICS_SECRET, every request for metrics must present that secret as a bearer token. For more information, see Integrate Prometheus with Hasura. To confirm that an instance serves metrics, request /v1/metrics from a machine that can reach it and confirm that the request returns a successful status code. For example:
    If the instance sets HASURA_GRAPHQL_METRICS_SECRET, present that value as a bearer token:
  • Make port 8080, or the port that HASURA_GRAPHQL_SERVER_PORT sets, reachable on each Hasura instance from the CXDOT Collector.

Configure

The Hasura integration is enabled by default, but doesn’t collect data until you configure targets. Add a CXDOT integration annotation to your Hasura pods or to a Service in front of them, or list the Hasura instances to monitor. To configure the Hasura integration, follow these steps:
  1. To collect from Hasura pods in Kubernetes, add a CXDOT integration annotation to the pod template. The annotation key must end with the container name. For example, add the following annotation to your Hasura workload:
    The Collector replaces ${target.host} with the pod IP address and ${target.port.http} with the container port named http. Metrics from each annotated pod include that pod’s Kubernetes metadata. This annotation sets no metrics secret, so use it for instances that don’t set HASURA_GRAPHQL_METRICS_SECRET, or see the step about the metrics secret for annotated pods.
  2. Alternatively, to collect from Hasura instances outside your Kubernetes cluster or at addresses that you choose, list each instance in endpoints. For example, add the following to the Helm values file for your CXDOT Collector:
    When you list endpoints, the integration collects from exactly those endpoints instead of from annotated pods and Services. In Kubernetes, the elected cluster Collector collects from each endpoint that you list, one time rather than once per node. For more information, see architecture. List an address that reaches one Hasura instance. Each request to an address that balances across replicas, such as a Kubernetes Service in front of several pods, reaches a different replica, so the metrics mix the replicas’ values. For a Hasura deployment with several replicas, list each replica’s own address. For example, a StatefulSet with a headless Service gives each pod a stable DNS name. Metrics from listed endpoints don’t carry Kubernetes metadata from the Hasura pods. Use the server.address and server.port attributes to identify each instance.
  3. Optional: To collect from both annotated pods and listed endpoints, list the endpoints in a second integration instance. For example, add the following to the Helm values file for your CXDOT Collector:
    The hasura/edge key defines a named instance for the listed endpoints. The bare hasura key lists no endpoints, so the integration continues to collect from annotated pods and Services.
  4. Optional: If you can’t change the pod template, configure the Service in front of the instances’ pods with a CXDOT integration annotation instead. The integrations.cxdot.paloaltonetworks.com/endpoints key makes each ready pod behind the Service a separate target, so the metrics don’t mix the replicas’ values. For example, add the following annotation to that Service:
    The Collector replaces ${target.host} with the IP address of each pod and ${target.port.http} with the pod port that the Service port named http forwards to. This annotation sets no metrics secret, so use it for instances that don’t set HASURA_GRAPHQL_METRICS_SECRET. The Collector reads these annotations only when endpoints is empty.
  5. Optional: To supply the metrics secret for listed endpoints, mount the secret from a Kubernetes Secret in the namespace of your CXDOT Collector with the chart’s clusterCollector.extraVolumes and clusterCollector.extraVolumeMounts values, then set metrics_secret_file to the mounted path. For example:
    Kubernetes mounts each key of the Secret as a file named after the key, so the /etc/cxdot/hasura/secret path in this example exists only if the Secret stores the metrics secret under the key secret. For a different key, set metrics_secret_file to the path for that key. The Collector sends the same secret to every endpoint in one integration instance.
  6. Optional: Add a named integration instance for listed Hasura deployments that use a different metrics secret. Mount each instance’s Secret as in the previous step, with its own volume and mount path. For example:
    The hasura/billing key defines a second integration instance with its own endpoints and settings. Its metrics carry the cxdot.integration.instance.name attribute with the value billing.
  7. Optional: To collect from annotated pods that set a metrics secret, mount the Secret into the node Collector and allow its path. The metrics_secret_file setting takes the path of a file that the Collector reads, so an annotation can’t carry the secret itself. The Collector rejects a hasura block that names a file outside allowed_collector_file_paths, which allows no paths by default. The Secret must be in the namespace of your CXDOT Collector. For example:
    Then set metrics_secret_file: /etc/cxdot/hasura/secret in the hasura block of the pod annotation, where secret is the key that holds the metrics secret in the Kubernetes Secret. After you allow a path, an annotation on any pod in your cluster can name the file, and the Collector sends the secret to that annotation’s target. The custom_pod_host: deny setting limits each pod annotation’s target to the pod’s own IP address, so the secret reaches only annotated pods. Allow only paths that hold secrets that you intend annotated pods to use.
  8. Optional: If the OpenMetrics integration also scrapes your Hasura pods or Service, disable OpenMetrics on that workload to avoid duplicate metrics. For example, set openmetrics: {enabled: false} in a CXDOT integration annotation on the pod or Service. For more information, see autodiscovery.

Validate

To validate the Hasura integration, follow these steps:
  1. In the Live Telemetry Analyzer, filter for __name__=cxdot.integration.target.health and cxdot.integration.name=hasura. Confirm that each Hasura instance reports 1 for cxdot.integration.target.health, identified by its server.address and server.port attributes.
  2. In Metrics Explorer, run the following query while Hasura serves GraphQL requests:
    Confirm that the query returns a time series for each instance and operation type that serves requests.

Troubleshooting

  • An endpoint reports 0 for cxdot.integration.target.health: Confirm that the instance meets the requirements in the Prerequisites section and that metrics_secret_file holds the instance’s current metrics secret. If you set max_samples, a collection that exceeds the limit also reports 0.
  • Source health metrics don’t appear: Hasura reports source health only for sources that have a source health check configured.

Configuration reference

Configure one Hasura integration instance with the following settings. In Helm values, place these settings under collection.cxdot.integrations.hasura. In a Collector configuration file, place them under cxdot.integrations.hasura.

Optional settings

  • enabled Type: boolean. Optional. Default: true. Whether to enable this Hasura integration instance. If true, the Collector collects metrics from the endpoints that this instance is configured for. If false, the Collector doesn’t run this integration instance.
  • endpoints Type: array of object. Optional. Default: []. Hasura GraphQL Engine endpoints to monitor. Specify a full HTTP or HTTPS URL that includes the /v1/metrics path, or use host:port format to request the /v1/metrics path over HTTP. For a URL without a path, the Collector requests /metrics, which Hasura doesn’t serve. Hasura exposes metrics only in Enterprise Edition, with metrics in HASURA_GRAPHQL_ENABLED_APIS. Each entry is one target, so list an address that reaches one Hasura instance. An address that balances requests across replicas, such as a Kubernetes Service with several pods, reaches a different replica on each collection. The integration instance collects only from the endpoints listed here.
  • endpoints[].endpoint Type: string. Required. Hasura GraphQL Engine address, as a full HTTP or HTTPS URL or in host:port format. For host:port, the Collector requests the /v1/metrics path over HTTP. A full URL must include the /v1/metrics path, such as https://hasura.example.com/v1/metrics. For a URL without a path, the Collector requests /metrics, which Hasura doesn’t serve.
  • metrics_secret_file Type: string. Optional. Default: empty string. Path to a file that holds the value of Hasura’s HASURA_GRAPHQL_METRICS_SECRET. The Collector sends this value as a bearer token on every request. The Collector rereads the file when it changes. If Hasura has no metrics secret, omit this setting.
  • tls Type: object. Optional. Transport layer security (TLS) settings for HTTPS endpoints. The Collector verifies certificates unless insecure_skip_verify is true. These settings don’t apply to HTTP endpoints.
  • tls.ca_file Type: string. Optional. Path to a PEM-encoded certificate authority (CA) certificate file used to verify the Hasura server certificate. If set, the Collector trusts only the certificate authorities in this file instead of the system certificate authority pool.
  • tls.insecure_skip_verify Type: boolean. Optional. Whether to skip verification of the Hasura server certificate. If true, the connection is encrypted but isn’t authenticated and is vulnerable to man-in-the-middle attacks. If false, the Collector verifies the server certificate.
  • tls.server_name Type: string. Optional. Server name to verify in the Hasura server certificate instead of the endpoint hostname. The Collector also sends this name as the TLS server name indication. Use this setting when the certificate name differs from the endpoint hostname.
  • collection_interval Type: duration. Optional. Default: 15s. How often the integration collects metrics from each Hasura endpoint.
  • timeout Type: duration. Optional. Default: 10s. Maximum time allowed to collect metrics from a Hasura endpoint during one interval. The value must be greater than zero.
  • max_samples Type: integer. Optional. Default: 0. Minimum: 0. Maximum number of samples accepted from one metric collection. A value of 0, the default, means no limit. A collection that exceeds the limit fails and reports the target as unhealthy instead of returning an incomplete set of samples. If your Hasura version labels request metrics by operation, the number of samples grows with the number of distinct operations that a deployment serves.
  • metrics Type: object. Optional. Settings that control which Hasura metrics the integration collects and how it names them.
  • metrics.include Type: array of string. Optional. Default: []. Regular expressions for metrics to collect in addition to the integration’s default metric set. Each expression must match a whole metric name as Hasura exposes it, without a _total suffix. A matched metric outside the default set keeps its exposed name.
  • metrics.exclude Type: array of string. Optional. Default: []. Regular expressions for metrics to drop. Each expression must match a whole metric name as Hasura exposes it, without a _total suffix, such as hasura_otel_.*. Exclusions apply before renaming and take precedence over include.
  • metrics.rename Type: object. Optional. Default: {}. Mapping of metric names as Hasura exposes them, without a _total suffix, to the names that the Collector reports. These mappings take precedence over the integration’s default metric names. Renaming a metric also collects it.
  • labels Type: object. Optional. Settings that drop metric samples based on their label values. Rules match the label names that Hasura exposes. The integration reports operation_name as graphql.operation.name and operation_type as graphql.operation.type. It reports the source_name label of Postgres connection metrics as db.client.connection.pool.name. In a rule, use the label name that Hasura exposes, not the reported name. Other labels, such as trigger_name and the source_name label of other metrics, keep their names.
  • labels.sample_drop_rules Type: object. Optional. Default: {}. Drops a whole sample when one of its labels matches. Each key is a label name. The value is a list of regular expressions matched against the label’s value. To drop every sample the label appears on with a non-empty value, use ".+".