Configure the Amazon Kinesis Firehose destination plugin in Palo Alto Networks Telemetry Pipeline to send logs to Amazon Kinesis Firehose.
This information is for Palo Alto Networks Telemetry Pipeline, which is a standalone
product separate from Cortex XCOR.
The Amazon Kinesis Firehose destination plugin
(name: kinesis_firehose) lets you configure your telemetry pipeline to collect
various data types such as logs, metrics, traces, and events and stream them
directly to your Kinesis Firehose delivery stream.
Supported telemetry types
The Amazon Kinesis Firehose destination plugin for Palo Alto Networks Telemetry Pipeline supports these telemetry types:
Logs
Metrics
Traces
Configuration parameters
Use the parameters in this section to configure the Amazon Kinesis Firehose destination plugin. The
Palo Alto Networks Telemetry Pipeline web interface uses the items in the Name column to
describe these parameters. Pipeline configuration files
use the items in the Key column as YAML keys.
Required
Name
Key
Description
Default
Region
region
Required. The AWS region to which to send your data.
us-east-1
Firehose delivery stream name
delivery_stream
Kinesis Firehose name.
none
AWS Authentication
Name
Key
Description
Default
IAM Role ARN
role_arn
Amazon Resource Name (ARN) of an AWS Identity and Access Management (IAM) role to assume, for uses like cross-account access.
none
Firehose API Endpoint
endpoint
Custom endpoint for the Firehose API.
none
STS API Endpoint
sts_endpoint
Custom endpoint for the STS API.
none
External ID for STS API
external_id
Specifies an external ID for the STS API. Can be used with the role_arn parameter if your role requires an external ID.
none
Advanced
Name
Key
Description
Default
Compression
compression
Compression type for Firehose records. Each log record is individually compressed and sent to Firehose. Accepted values: gzip, arrow (with arrow available only if Apache Arrow is enabled at compile time).
none
Time Key
time_key
Adds the timestamp to the record under this key. By default, the timestamp from Palo Alto Networks Telemetry Pipeline isn’t added to records sent to Kinesis.
none
Time Key Format
time_key_format
strftime-compliant format string for the timestamp. This option is used with time_key.
%Y-%m-%dT%H:%M:%S
Log Key
log_key
By default, the entire log record is sent to Kinesis. If you specify a key name with this option, only the value of that key is sent to Kinesis. For example, if you’re using the Fluentd Docker log driver, you can specify log_key log, and only the log message is sent to Kinesis.
none
Enable Auto Retry Requests
auto_retry_requests
Whether to immediately retry failed requests to AWS services one time. This option doesn’t affect the normal Palo Alto Networks Telemetry Pipeline retry mechanism with backoff. Instead, it enables an immediate retry with no delay for networking errors, which can help improve throughput when there are transient or random networking issues. Accepted values: true, false.
false
Security and TLS
Name
Key
Description
Default
TLS
tls
If true, enables TLS/SSL. If false, disables TLS/SSL. Accepted values: true, false.
false
TLS Certificate Validation
tls.verify
If on, and if tls is true, enables TLS/SSL certificate validation. If off, disables TLS/SSL certificate validation. Accepted values: on, off.