Viewing metrics and logs from AWS

Suggest edits

Cloud Service sends all metrics and logs from Postgres clusters to AWS. You can view metrics and logs directly from AWS using AWS CloudWatch.

Logs

AWS CloudWatch

When Cloud Service deploys workloads on AWS, the logs from the Postgres clusters are forwarded to AWS CloudWatch log groups as log streams. To query Cloud Service logs, you can use filter and patterns syntax or the CloudWatch Logs Insights section.

Query Postgres cluster logs

All logs from your Postgres clusters are presented as log streams in CloudWatch log groups. To find the CloudWatch log groups:

Sign in to the AWS console.
From Services, select CloudWatch.
Select the region corresponding where you choose to deploy your Cloud Service cluster.
Under Logs, select Log groups in the menu on the left.

The following log groups and log streams are available in the Customer CloudWatch space.

Log group name	Log stream name	Description	Logger
/biganimal/PostgresLogs	from-pg_cluster_logs_for_customer	Logs of the Customer clusters databases (all Postgres-related logs)	`logger = postgres`
/biganimal/PostgresAuditLogs	from-pg_cluster_audit_logs_for_customer	Audit Logs of the Customer clusters databases	`logger = pgaudit or edb_audit`

You can use the CloudWatch Logs Insights query syntax to compose your queries over these log streams.

Logs are split into structured fields matching those of the Postgres csvlog format with a record. prefix. For example, the sql_state_code is in the record.sql_state_code log field. The pg_cluster_id field identifies the specific Postgres cluster that originated the log message.

For example, with the /biganimal/PostgresLogs log group selected in CloudWatch Logs Insights, you can view all log entries with the following CloudWatch Logs Insights query:

fields record.log_time, record.error_severity, record.message
| sort record.log_time desc

With the /biganimal/PostgresAuditLogsog log group selected, you can view the pgaudit and edb_audit logs with:

fields record.log_time, record.error_severity, record.audit.statement
| filter logger == "pgaudit"
| sort record.log_time desc

fields record.log_time, record.error_severity, record.message
| filter logger == "edb_audit"
| sort record.log_time desc

Metrics

The Cloud Service instances collect a variety of metrics and make them available to your AWS subscription for dashboarding, alerting, querying, and other analytics. Most of these metrics are acquired directly from Postgres system tables, views, and functions. The Postgres documentation serves as the main reference for these metrics.

Available metrics

The following log group in AWS CloudWatch contains metrics specific to Cloud Service.

Log group name	Log stream name	Description
/biganimal/Telemetry	from-http.0	Metrics streams from Cloud Service Prometheus.

You can use the CloudWatch Logs Insights query language to compose queries that analyze time-series metrics over these log groups and report the latest samples of metrics. Query the from-http.0 log stream in the /biganimal/Telemetry log group.

The available set of metrics is subject to change. Metrics might be added, removed, or renamed. Where feasible, we will change the metric name when the meaning or type of an existing metric changes.

Currently, all metrics forwarded from Prometheus are in the /biganimal/Telemetry log group. This might change in a future release.

Metrics labels

All Postgres metrics share a common labeling scheme. Entries generally have at least the following labels.

Name	Description
instance	IP address of the host the metric originated from
postgresql	Cloud Service postgres cluster identifier, e.g., `p-abcdef012345`
role	Postgres instance role: primary or standby replica
datname	Postgres database name (where applicable)
pod	k8s pod name
aks_cluster_name	AKS cluster name

The labels field of a metrics entry is a nested field under the JSON-typed Message field. To query the field for individual values, you dot reference (Message.labels) the labels object.

Example usage:

With /biganimal/Telemetry log group selected:

fields Message.labels.postgresql,Message.name, Message.value
| filter strcontains (Message.name, 'cnp') and Message.labels.role == 'primary'
| sort @timestamp desc

See Metrics for a detailed list of available metrics.

Dashboards

You can view logs and metrics from your Postgres clusters using AWS CloudWatch dashboards.

To access AWS CloudWatch dashboards:

Sign in to the AWS console.
From Services, select CloudWatch.
Select the region where you choose to deploy your Cloud Service cluster.
In the CloudWatch menu on the left, select Dashboards.
Select Telemetry Dashboard to launch a CloudWatch dashboard for your deployed clusters.