Set up Google Cloud integration
Successful ingestion of Google Cloud metrics requires XCOR to modify your tenant infrastructure and you to modify your Google Cloud environment. You can perform your Google Cloud setup while XCOR works to provision your tenant with the updated infrastructure.-
Contact XCOR Support to
request infrastructure changes to your tenant for Google Cloud integration.
Follow the remaining steps to set up your Google Cloud project. Metrics won’t flow into XCOR until infrastructure is provisioned for your tenant.
-
Modify the Google Cloud domain restriction constraint.
If your Google Cloud organization restricts identities by
domain,
you must add the XCOR customer identity (
C04aozwp9) as an allowed value in your policy. -
Create a Google Cloud service account.
For each Google Cloud project, create a service account with theA Google Cloud service account isn’t an XCOR service account. XCOR requires a Google Cloud service account in your Google Cloud project to provide XCOR access to metrics using service account impersonation.
monitoring.viewerrole. To implement the principle of least privilege, create a custom role with the following restricted set of permissions:monitoring.timeSeries.listmonitoring.metricDescriptors.listmonitoring.monitoredResourceDescriptors.get
-
Add the XCOR principal to the Google Cloud service account.
Each Google Cloud service account must grant access to the XCOR
principal to impersonate it.
To retrieve your tenant-specific principal, click your
profile icon from the
menu bar and select My Account. Copy the value displayed next to Google
Cloud Metrics IAM Principal.
Grant the principal the
iam.serviceAccountTokenCreatorrole. -
Configure the integration in XCOR.
In the Integrations Hub, select Google Cloud and complete the configuration form.
You can also initialize the integration using the
Google Cloud integration API
by applying the configuration with Chronoctl
or Terraform.
For each Google Cloud project, supply the Google Cloud service account email
(
SERVICE-ACCOUNT@PROJECT-ID.iam.gserviceaccount.com) in the form or request to configure the integration. Use metric prefixes to specify which metrics to ingest. See Metric Prefix Search for examples. The Integrations Hub applies the same metric prefix list to every project in one configuration. Use Chronoctl, Terraform, or the XCOR API to assign different prefix lists to different projects.
Save validation
When you save, XCOR validates the required service account, project, metric prefix, filter, and rollup rule fields. Save also rejects a project with no metric prefixes, and a project with overlapping prefixes, such ascompute.googleapis.com/ alongside
compute.googleapis.com/instance/. Save doesn’t connect to Google Cloud or verify that
XCOR can impersonate the service account or read metrics from the projects.
Example configuration
The following example creates a single service account in your Google Cloud project and enables XCOR to impersonate it.- Terraform
Terraform example
Use metrics scopes for projects
Use metrics scoping for projects in your Google Cloud environment, particularly if you manage five or more projects. XCOR recommends creating a new metrics scoped project for use with metrics ingestion. Using a new scoped project minimizes the impact on your existing workload. Metrics scopes provide the following benefits over configuring individual Google Cloud projects:- Google Cloud API cost savings: Metrics scoping lets XCOR ingest metrics from the underlying Google Cloud projects with fewer API requests by packing time series data points into fewer API responses.
- Configuration simplicity: Metrics scoping lets you manage fewer Google Cloud service accounts and XCOR configurations.
Set up XCOR to receive Google Cloud data
After configuring Google Cloud to enable XCOR to access metrics, you must configure XCOR to receive and process those metrics.View Google Cloud integrations
To list or view Google Cloud integrations, use one of the following options:- Chronoctl
- API
To list your Google Cloud integrations:To view a Google Cloud integration with Chronoctl, use the command:Replace
SLUG with the unique identifier of the Google Cloud integration.Create or update a Google Cloud integration
You can create or update your Google Cloud integration with XCOR by applying a configuration file with Chronoctl or Terraform. You must add your service account to an XCOR team with SysAdmin permissions.- Chronoctl
- Terraform
- API
To create a Google Cloud integration with Chronoctl, use the command:Replace Replace
FILENAME with the name of your Chronoctl configuration file.To update a Google Cloud integration with Chronoctl, use the command:FILENAME with the name of your Chronoctl configuration file.The input file uses the following structure:Chronoctl example
NAME: (string) The name of the Google Cloud integration.SLUG: (string) The unique identifier of the Google Cloud integration.EMAIL: (string) The email address of the customer’s Google Cloud service account of the formSERVICE_ACCOUNT@PROJECT-ID.iam.gserviceaccount.com.PROJECT_ID: (string) The ID of the customer’s Google Cloud project to ingest metrics from.PREFIX: (list(string)) A list of metric prefixes to ingest from the Google Cloud project.
Control incoming metrics
Google Cloud charges you for metrics that leave your environment, including scraped metrics you don’t need. During ingestion into XCOR, you can apply rollup and drop rules to reduce ingestion and storage costs. These XCOR rules apply after extraction from Google Cloud and don’t reduce Google Cloud API usage. Metrics that arrive after an XCOR rollup rule’s processing window can be available as raw metrics but are missing from the rule’s output. Queries on the rollup output don’t include those late metrics. Filters and rollup rules are available only through Chronoctl, Terraform, or the XCOR API, and only when these features are enabled for your tenant. The Integrations Hub form doesn’t include them. Use the following options to reduce the number of metrics extracted from Google Cloud:- Filters: Filters in Google Cloud let you target a subset of metrics. Filters don’t replace prefixes. Instead, filters apply to metric and resource labels, or other user-defined metadata.
- Rollup Rules: Google Cloud rollup rules pre-aggregate a metric by returning only the specified dimensions and the Google Cloud project ID. All other labels are dropped. Use these rules to reduce cardinality of time series returned from Google Cloud.
- Example: Exclude Region
- Example: Selective Aggregation
Exclude all metrics where the
metadata.system_labels “region” label matches
us-central1-a:AGGREGATION with MEAN, MIN, MAX, SUM, or COUNT.
Delete a Google Cloud integration
Delete a Google Cloud integration using one of the following methods:- Chronoctl
- API
To delete a Google Cloud integration with Chronoctl, your account must have
SysAdmin
permissions. Use the command:
Validate metrics ingestion
- After saving the configuration, wait for at least the metric’s Google Cloud sample period and ingest delay.
-
In Metrics Explorer,
select an ingested metric whose name begins with
gcp_, and run the following query:Use a range longer than the metric’s sample period and ingest delay. If the query returns a result, data is flowing.
Metric information
The following sections explain how XCOR uses and displays Google Cloud metrics.Metrics availability
Google metrics have different stages of availability. The integration can ingest generally available (GA) and non-GA metrics that match the configured prefixes. Non-GA metrics can change without notice and might not provide stable data. Google Cloud metrics can appear in XCOR after their metric timestamps due to the following factors:- Query latency: Most Google Cloud metrics have a delay before they’re available to query through the Google Cloud API. Metrics whose timestamps fall outside your tenant’s accepted ingestion window aren’t ingested. Contact XCOR Support if you need to ingest metrics with long delays.
- Sample rate: Google Cloud metrics sample at different rates. Most metrics sample at 60-second intervals, with some as long as once per day.
networking.googleapis.com/pod_flownetworking.googleapis.com/vm_flownetworking.googleapis.com/node_flow
Metric names
In XCOR, Google Cloud metrics have the following name structure:-
MONITORED-RESOURCE-NAMEis the monitored resource type in Google Cloud. -
FULLY-QUALIFIED-METRIC-NAMEis the metric name in Google Cloud. Characters such as.and/are converted to_to be Prometheus compatible. The following examples are fully qualified metric names in Google Cloud before metric name sanitization:bigquery.googleapis.com/job/num_in_flightcloudsql.googleapis.com/database/available_for_failoverkubernetes.io/container/accelerator/memory_total
database/active_directory/domain_reachable
to a metric name in XCOR, cloudsql_database is the monitored
resource name and cloudsql.googleapis.com/database/active_directory/domain_reachable
is the fully qualified metric name. In XCOR, the metric name is
gcp_cloudsql_database_cloudsql_googleapis_com_database_active_directory_domain_reachable.
Metric labels
You can request custom labels for your Google Cloud metrics asdefaultLabels. Contact
XCOR Support to add them.
When a configured defaultLabels name conflicts with a label from Google Cloud
(for example, job), the configured default label keeps its name and XCOR
adds the exported_ prefix to the imported Google Cloud label.
Finding Google Cloud metrics in Metrics Explorer
Use Metrics Explorer to find and review the status of your ingested metrics.-
All Google Cloud metrics start with the prefix
gcp_. Searching for this prefix displays all Google Cloud metrics in the platform. -
Substring search is possible. For example, if the original Google Cloud metric name
contains a substring like
domain_reachable, searching for the substring returns the Google Cloud metric, along with other metrics containing the substring.