serviceTemplateContainer Image Requirements
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Creating the External Secret
cluster-example-app-secret, which refreshes the password every 24 hours. It
uses a Merge policy to update only the specified fields (password, pgpass,
jdbc-uri and uri) in the cluster-example-app secret.
- apiVersion: external-secrets.io/v1beta1
+apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: cluster-example-app-secret
@@ -482,7 +486,7 @@ Creating the SecretStore
namespace at http://vault.vault.svc:8200, and that a Kubernetes Secret
named vault-token exists in the same namespace, containing the token used to
authenticate with Vault.
-apiVersion: external-secrets.io/v1beta1
+apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
name: vault-backend
diff --git a/assets/documentation/1.25/cnpg_i/index.html b/assets/documentation/1.25/cnpg_i/index.html
new file mode 100644
index 000000000..89cbd50d3
--- /dev/null
+++ b/assets/documentation/1.25/cnpg_i/index.html
@@ -0,0 +1,551 @@
+
+
+
+ CNPG-I - CloudNativePG v1.25
+
+
+
+
+
+
+
+ CloudNativePG v1.25
+
+
+
+
+
+
+
+
CNPG-I
+
+
+
The CloudNativePG Interface (CNPG-I )
+is a standard way to extend and customize CloudNativePG without modifying its
+core codebase.
+
Why CNPG-I?
+
CloudNativePG supports a wide range of use cases, but sometimes its built-in
+functionality isn’t enough, or adding certain features directly to the main
+project isn’t practical.
+
Before CNPG-I, users had two main options:
+
+Fork the project to add custom behavior, or
+Extend the upstream codebase by writing custom components on top of it.
+
+
Both approaches created maintenance overhead, slowed upgrades, and delayed delivery of critical features.
+
CNPG-I solves these problems by providing a stable, gRPC-based integration
+point for extending CloudNativePG at key points in a cluster’s lifecycle —such
+as backups, recovery, and sub-resource reconciliation— without disrupting the
+core project.
+
CNPG-I can extend:
+
+The operator, and/or
+The instance manager running inside PostgreSQL pods.
+
+
Registering a plugin
+
CNPG-I is inspired by the Kubernetes
+Container Storage Interface (CSI) .
+The operator communicates with registered plugins using gRPC , following the
+CNPG-I protocol .
+
CloudNativePG discovers plugins at startup . You can register them in one of two ways:
+
+Sidecar container – run the plugin inside the operator’s Deployment
+Standalone Deployment – run the plugin as a separate workload in the same
+ namespace
+
+
In both cases, the plugin must be packaged as a container image.
+
Sidecar Container
+
When running as a sidecar, the plugin must expose its gRPC server via a Unix
+domain socket . This socket must be placed in a directory shared with the
+operator container, mounted at the path set in PLUGIN_SOCKET_DIR (default:
+/plugin).
+
Example:
+
apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: controller-manager
+spec:
+ template:
+ spec:
+ containers:
+ - image: cloudnative-pg:latest
+ [...]
+ name: manager
+ volumeMounts:
+ - mountPath: /plugins
+ name: cnpg-i-plugins
+
+ - image: cnpg-i-plugin-example:latest
+ name: cnpg-i-plugin-example
+ volumeMounts:
+ - mountPath: /plugins
+ name: cnpg-i-plugins
+ volumes:
+ - name: cnpg-i-plugins
+ emptyDir: {}
+
+
Standalone Deployment (recommended)
+
Running a plugin as its own Deployment decouples its lifecycle from the
+operator’s and allows independent scaling. In this setup, the plugin exposes a
+TCP gRPC endpoint behind a Service, with mTLS for secure communication.
+
+
Warning
+
CloudNativePG does not discover plugins dynamically. If you deploy a new
+plugin, you must restart the operator to detect it.
+
+
Example Deployment:
+
apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: cnpg-i-plugin-example
+spec:
+ template:
+ [...]
+ spec:
+ containers:
+ - name: cnpg-i-plugin-example
+ image: cnpg-i-plugin-example:latest
+ ports:
+ - containerPort: 9090
+ protocol: TCP
+
+
The related Service for the plugin must include:
+
+The label cnpg.io/plugin: <plugin-name> — required for CloudNativePG to
+ discover the plugin
+The annotation cnpg.io/pluginPort: <port> — specifies the port where the
+ plugin’s gRPC server is exposed
+
+
Example Service:
+
apiVersion: v1
+kind: Service
+metadata:
+ annotations:
+ cnpg.io/pluginPort: "9090"
+ labels:
+ cnpg.io/pluginName: cnpg-i-plugin-example.my-org.io
+ name: cnpg-i-plugin-example
+spec:
+ ports:
+ - port: 9090
+ protocol: TCP
+ targetPort: 9090
+ selector:
+ app: cnpg-i-plugin-example
+
+
Configuring TLS Certificates
+
When a plugin runs as a Deployment, communication with CloudNativePG happens
+over the network. To secure it, mTLS is enforced , requiring TLS
+certificates for both sides.
+
Certificates must be stored as Kubernetes TLS Secrets
+and referenced in the plugin’s Service annotations
+(cnpg.io/pluginClientSecret and cnpg.io/pluginServerSecret):
+
apiVersion: v1
+kind: Service
+metadata:
+ annotations:
+ cnpg.io/pluginClientSecret: cnpg-i-plugin-example-client-tls
+ cnpg.io/pluginServerSecret: cnpg-i-plugin-example-server-tls
+ cnpg.io/pluginPort: "9090"
+ name: barman-cloud
+ namespace: postgresql-operator-system
+spec:
+ [...]
+
+
+
Note
+
You can provide your own certificate bundles, but the recommended method is
+to use Cert-manager .
+
+
Using a plugin
+
To enable a plugin, configure the .spec.plugins section in your Cluster
+resource. Refer to the CloudNativePG API Reference for the full
+PluginConfiguration
+specification.
+
Example:
+
apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+ name: cluster-with-plugins
+spec:
+ instances: 1
+ storage:
+ size: 1Gi
+ plugins:
+ - name: cnpg-i-plugin-example.my-org.io
+ enabled: true
+ parameters:
+ key1: value1
+ key2: value2
+
+
Each plugin may have its own parameters—check the plugin’s documentation for
+details. The name field in spec.plugins depends on how the plugin is
+deployed:
+
+Sidecar container: use the Unix socket file name
+Deployment: use the value from the Service’s cnpg.io/pluginName label
+
+
+
The CNPG-I protocol has quickly become a proven and reliable pattern for
+extending CloudNativePG while keeping the core project maintainable.
+Over time, the community has built and shared plugins that address real-world
+needs and serve as examples for developers.
+
For a complete and up-to-date list of plugins built with CNPG-I, please refer to the
+CNPG-I GitHub page .
+
+
+
+
+
+
+
+
PgBouncer configuration options
Monitoring
+
Logging
Container Image Requirements
Operator capability levels
PgBouncer configuration options
documentation for each parameter. Unless stated otherwise, the default values
are the ones directly set by PgBouncer.
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
@@ -903,6 +905,17 @@ Monitoring
podMetricsEndpoints:
- port: metrics
Deprecation of Automatic PodMonitor Creation
+
+
Feature Deprecation Notice
+
The .spec.monitoring.enablePodMonitor field in the Pooler resource is
+now deprecated and will be removed in a future version of the operator.
+
If you are currently using this feature, we strongly recommend you either
+remove or set .spec.monitoring.enablePodMonitor to false and manually
+create a PodMonitor resource for your pooler as described above.
+This change ensures that you have complete ownership of your monitoring
+configuration, preventing it from being managed or overwritten by the operator.
Logging
Logs are directly sent to standard output, in JSON format, like in the
following example:
diff --git a/assets/documentation/1.25/container_images/index.html b/assets/documentation/1.25/container_images/index.html
index d714af412..793cb08ad 100644
--- a/assets/documentation/1.25/container_images/index.html
+++ b/assets/documentation/1.25/container_images/index.html
@@ -232,6 +232,10 @@
+
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Hibernation
Cluster Summary
Name: cluster-example
Namespace: default
-PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:17.5
+PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:18.0-system-trixie
Primary instance: cluster-example-2
Status: Cluster in healthy state
Instances: 3
diff --git a/assets/documentation/1.25/declarative_role_management/index.html b/assets/documentation/1.25/declarative_role_management/index.html
index a1a474e73..9bc600677 100644
--- a/assets/documentation/1.25/declarative_role_management/index.html
+++ b/assets/documentation/1.25/declarative_role_management/index.html
@@ -242,6 +242,10 @@
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Image Catalog
Container Image Requirements
+
Operator capability levels
Image Catalog
spec:
images:
- major: 15
- image: ghcr.io/cloudnative-pg/postgresql:15.6
+ image: ghcr.io/cloudnative-pg/postgresql:15.14-system-trixie
- major: 16
- image: ghcr.io/cloudnative-pg/postgresql:16.8
+ image: ghcr.io/cloudnative-pg/postgresql:16.10-system-trixie
- major: 17
- image: ghcr.io/cloudnative-pg/postgresql:17.5
+ image: ghcr.io/cloudnative-pg/postgresql:17.6-system-trixie
+ - major: 18
+ image: ghcr.io/cloudnative-pg/postgresql:18.0-system-trixie
Example of a Cluster-Wide Catalog using ClusterImageCatalog Resource:
apiVersion: postgresql.cnpg.io/v1
@@ -355,14 +355,17 @@ Image Catalog
spec:
images:
- major: 15
- image: ghcr.io/cloudnative-pg/postgresql:15.6
+ image: ghcr.io/cloudnative-pg/postgresql:15.14-system-trixie
- major: 16
- image: ghcr.io/cloudnative-pg/postgresql:16.8
+ image: ghcr.io/cloudnative-pg/postgresql:16.10-system-trixie
- major: 17
- image: ghcr.io/cloudnative-pg/postgresql:17.5
+ image: ghcr.io/cloudnative-pg/postgresql:17.6-system-trixie
+ - major: 18
+ image: ghcr.io/cloudnative-pg/postgresql:18.0-system-trixie
A Cluster resource has the flexibility to reference either an ImageCatalog
-or a ClusterImageCatalog to precisely specify the desired image.
+(like in the following example) or a ClusterImageCatalog to precisely specify
+the desired image.
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
@@ -371,6 +374,7 @@ Image Catalog
instances: 3
imageCatalogRef:
apiGroup: postgresql.cnpg.io
+ # Change the following to `ClusterImageCatalog` if needed
kind: ImageCatalog
name: postgresql
major: 16
@@ -381,27 +385,43 @@ Image Catalog
Any alterations to the images within a catalog trigger automatic updates for
all associated clusters referencing that specific entry.
CloudNativePG Catalogs
-The CloudNativePG project maintains ClusterImageCatalogs for the images it
-provides. These catalogs are regularly updated with the latest images for each
-major version. By applying the ClusterImageCatalog.yaml file from the
-CloudNativePG project's GitHub repositories, cluster administrators can ensure
-that their clusters are automatically updated to the latest version within the
-specified major release.
-PostgreSQL Container Images
-You can install the
-latest version of the cluster catalog for the PostgreSQL Container Images
-(cloudnative-pg/postgres-containers repository)
-with:
-kubectl apply \
- -f https://raw.githubusercontent.com/cloudnative-pg/postgres-containers/main/Debian/ClusterImageCatalog-bookworm.yaml
+The CloudNativePG project maintains ClusterImageCatalog manifests for all
+supported images.
+These catalogs are regularly updated and published in the
+artifacts repository .
+Each catalog corresponds to a specific combination of image type (e.g.
+minimal) and Debian release (e.g. trixie). It lists the most up-to-date
+container images for every supported PostgreSQL major version.
+By installing these catalogs, cluster administrators can ensure that their
+PostgreSQL clusters are automatically updated to the latest patch release
+within a given PostgreSQL major version, for the selected Debian distribution
+and image type.
+For example, to install the latest catalog for the minimal PostgreSQL
+container images on Debian trixie, run:
+kubectl apply -f \
+ https://raw.githubusercontent.com/cloudnative-pg/artifacts/refs/heads/main/image-catalogs/catalog-minimal-trixie.yaml
+You can install all the available catalogs by using the kustomization file
+present in the image-catalogs directory:
+kubectl apply -k https://github.com/cloudnative-pg/artifacts//image-catalogs?ref=main
PostGIS Container Images
-You can install the
-latest version of the cluster catalog for the PostGIS Container Images
-(cloudnative-pg/postgis-containers repository)
-with:
-kubectl apply \
- -f https://raw.githubusercontent.com/cloudnative-pg/postgis-containers/main/PostGIS/ClusterImageCatalog.yaml
+You can then view all the catalogs deployed with:
+kubectl get clusterimagecatalogs.postgresql.cnpg.io
+For example, you can create a cluster with the latest minimal image for PostgreSQL 18 on trixie with:
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+ name: angus
+spec:
+ instances: 3
+ imageCatalogRef:
+ apiGroup: postgresql.cnpg.io
+ kind: ClusterImageCatalog
+ name: postgresql-minimal-trixie
+ major: 18
+ storage:
+ size: 1Gi
Container Image Requirements
Red Hat UBI images are primarily intended for OLM consumption.
Operands
-The PostgreSQL operand container images are available for all
-PGDG supported versions of PostgreSQL ,
-across multiple architectures, directly from the
-postgres-containers project's GitHub Container Registry
-The minimalstandard
-Weekly jobs ensure that critical vulnerabilities (CVEs) in the entire stack are
-promptly addressed.
-Additionally, the community provides images for the PostGIS extension .
+The CloudNativePG project provides and maintains PostgreSQL operand container
+images, built on top of the official Debian slim base image ,
+for both linux/amd64 and linux/arm64 architectures.
+Images are published for all Debian supported releases
+(stableoldstablePostgreSQL versions supported by PGDG .
+They are distributed via the postgres-containers GitHub Container Registry
+Three image flavors are available, each extending the previous one:
+
+
+
Important
+
The system images are deprecated and will be removed once in-core
+Barman Cloud support is phased out. They remain usable for now, but you may
+want to plan a future migration to minimal or standard images with the
+Barman Cloud plugin, or another supported backup solution.
+
By default, this version of CloudNativePG deploys ghcr.io/cloudnative-pg/postgresql:18.0-system-trixie.
+All images are signed and shipped with SBOM and provenance attestations.
+Weekly automated builds ensure that critical vulnerabilities (CVEs) are promptly fixed.
+For details and support, see the postgres-containers project
Main features
Direct integration with Kubernetes API server for High Availability,
@@ -507,5 +525,5 @@ About this guide
diff --git a/assets/documentation/1.25/installation_upgrade/index.html b/assets/documentation/1.25/installation_upgrade/index.html
index e34819beb..34c8ecf9e 100644
--- a/assets/documentation/1.25/installation_upgrade/index.html
+++ b/assets/documentation/1.25/installation_upgrade/index.html
@@ -264,6 +264,10 @@
Container Image Requirements
+
kubectl apply --server-side -f \
- https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.25/releases/cnpg-1.25.3.yaml
+ https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.25/releases/cnpg-1.25.4.yaml
You can verify that with:
kubectl get deployment -n cnpg-system cnpg-controller-manager
diff --git a/assets/documentation/1.25/instance_manager/index.html b/assets/documentation/1.25/instance_manager/index.html
index b00c18e39..2ca571e80 100644
--- a/assets/documentation/1.25/instance_manager/index.html
+++ b/assets/documentation/1.25/instance_manager/index.html
@@ -250,6 +250,10 @@
Container Image Requirements
Operator capability levels
Using the Debian or RedHat packages
As a result, you can follow standard practices and instructions to install
them in your systems.
Debian packages
- For example, let's install the 1.25.3 release of the plugin, for an Intel based
+
For example, let's install the 1.25.4 release of the plugin, for an Intel based
64 bit server. First, we download the right .deb file.
- wget https://github.com/cloudnative-pg/cloudnative-pg/releases/download/v1.25.3/kubectl-cnpg_1.25.3_linux_x86_64.deb \
+wget https://github.com/cloudnative-pg/cloudnative-pg/releases/download/v1.25.4/kubectl-cnpg_1.25.4_linux_x86_64.deb \
--output-document kube-plugin.deb
Then, with superuser privileges, install from the local file using dpkg:
@@ -440,13 +444,13 @@ Debian packages
Selecting previously unselected package cnpg.
(Reading database ... 6688 files and directories currently installed.)
Preparing to unpack kube-plugin.deb ...
-Unpacking cnpg (1.25.3) ...
-Setting up cnpg (1.25.3) ...
+Unpacking cnpg (1.25.4) ...
+Setting up cnpg (1.25.4) ...
RPM packages
-As in the example for .rpm packages, let's install the 1.25.3 release for an
+
As in the example for .rpm packages, let's install the 1.25.4 release for an
Intel 64 bit machine. Note the --output flag to provide a file name.
-curl -L https://github.com/cloudnative-pg/cloudnative-pg/releases/download/v1.25.3/kubectl-cnpg_1.25.3_linux_x86_64.rpm \
+curl -L https://github.com/cloudnative-pg/cloudnative-pg/releases/download/v1.25.4/kubectl-cnpg_1.25.4_linux_x86_64.rpm \
--output kube-plugin.rpm
Then, with superuser privileges, install with yum, and you're ready to use:
@@ -457,7 +461,7 @@ RPM packages
Package Architecture Version Repository Size
====================================================================================================
Installing:
- cnpg x86_64 1.25.3 @commandline 20 M
+ cnpg x86_64 1.25.4 @commandline 20 M
Transaction Summary
====================================================================================================
@@ -648,9 +652,9 @@ Status
Instances status
Name Current LSN Replication role Status QoS Manager Version Node
---- ----------- ---------------- ------ --- --------------- ----
-sandbox-1 0/604DE38 Primary OK BestEffort 1.25.3 k8s-eu-worker
-sandbox-2 0/604DE38 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker2
-sandbox-3 0/604DE38 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker
+sandbox-1 0/604DE38 Primary OK BestEffort 1.25.4 k8s-eu-worker
+sandbox-2 0/604DE38 Standby (async) OK BestEffort 1.25.4 k8s-eu-worker2
+sandbox-3 0/604DE38 Standby (async) OK BestEffort 1.25.4 k8s-eu-worker
If you require more detailed status information, use the --verbose option (or
-v for short). The level of detail increases each time the flag is repeated:
@@ -699,9 +703,9 @@ Status
Instances status
Name Current LSN Replication role Status QoS Manager Version Node
---- ----------- ---------------- ------ --- --------------- ----
-sandbox-1 0/6053720 Primary OK BestEffort 1.25.3 k8s-eu-worker
-sandbox-2 0/6053720 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker2
-sandbox-3 0/6053720 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker
+sandbox-1 0/6053720 Primary OK BestEffort 1.25.4 k8s-eu-worker
+sandbox-2 0/6053720 Standby (async) OK BestEffort 1.25.4 k8s-eu-worker2
+sandbox-3 0/6053720 Standby (async) OK BestEffort 1.25.4 k8s-eu-worker
With an additional -v (e.g. kubectl cnpg status sandbox -v -v), you can
also view PostgreSQL configuration, HBA settings, and certificates.
@@ -869,12 +873,12 @@ report Operator
and previous logs are available, it will show them both.
====== Beginning of Previous Log =====
-2023-03-28T12:56:41.251711811Z {"level":"info","ts":"2023-03-28T12:56:41Z","logger":"setup","msg":"Starting CloudNativePG Operator","version":"1.25.3","build":{"Version":"1.25.3+dev107","Commit":"cc9bab17","Date":"2023-03-28"}}
+2023-03-28T12:56:41.251711811Z {"level":"info","ts":"2023-03-28T12:56:41Z","logger":"setup","msg":"Starting CloudNativePG Operator","version":"1.25.4","build":{"Version":"1.25.4+dev107","Commit":"cc9bab17","Date":"2023-03-28"}}
2023-03-28T12:56:41.251851909Z {"level":"info","ts":"2023-03-28T12:56:41Z","logger":"setup","msg":"Starting pprof HTTP server","addr":"0.0.0.0:6060"}
<snipped …>
====== End of Previous Log =====
-2023-03-28T12:57:09.854306024Z {"level":"info","ts":"2023-03-28T12:57:09Z","logger":"setup","msg":"Starting CloudNativePG Operator","version":"1.25.3","build":{"Version":"1.25.3+dev107","Commit":"cc9bab17","Date":"2023-03-28"}}
+2023-03-28T12:57:09.854306024Z {"level":"info","ts":"2023-03-28T12:57:09Z","logger":"setup","msg":"Starting CloudNativePG Operator","version":"1.25.4","build":{"Version":"1.25.4+dev107","Commit":"cc9bab17","Date":"2023-03-28"}}
2023-03-28T12:57:09.854363943Z {"level":"info","ts":"2023-03-28T12:57:09Z","logger":"setup","msg":"Starting pprof HTTP server","addr":"0.0.0.0:6060"}
If the operator hasn't been restarted, you'll still see the ====== Begin …
@@ -1187,7 +1191,7 @@
Launching psql
$ kubectl cnpg psql cluster-example
-psql (17.5 (Debian 17.5-1.pgdg110+1))
+psql (18.0 (Debian 18.0-1.pgdg110+1))
Type "help" for help.
postgres=#
@@ -1196,7 +1200,7 @@ Launching psql
select to work against a replica by using the --replica option:
$ kubectl cnpg psql --replica cluster-example
-psql (17.5 (Debian 17.5-1.pgdg110+1))
+psql (18.0 (Debian 18.0-1.pgdg110+1))
Type "help" for help.
@@ -1553,7 +1557,7 @@ Permissions required by the plugin
status
-clusters: get
+clusters: get
subscription
@@ -1640,6 +1644,12 @@ Role examples
- policy
resources:
- poddisruptionbudgets
+ - verbs:
+ - get
+ apiGroups:
+ - barmancloud.cnpg.io
+ resources:
+ - objectstores
Important
diff --git a/assets/documentation/1.25/kubernetes_upgrade/index.html b/assets/documentation/1.25/kubernetes_upgrade/index.html
index c5f4cfe94..edf11149b 100644
--- a/assets/documentation/1.25/kubernetes_upgrade/index.html
+++ b/assets/documentation/1.25/kubernetes_upgrade/index.html
@@ -246,6 +246,10 @@
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Monitoring Instances
+
Monitoring with the Prometheus operator
-A specific PostgreSQL cluster can be monitored using the
-Prometheus Operator's resource
-PodMonitor .
-A PodMonitor that correctly points to the Cluster can be automatically created by the operator by setting
-.spec.monitoring.enablePodMonitor to true in the Cluster resource itself (default: false).
-
-
Important
-
Any change to the PodMonitor created automatically will be overridden by the Operator at the next reconciliation
-cycle, in case you need to customize it, you can do so as described below.
-
To deploy a PodMonitor for a specific Cluster manually, define it as follows and adjust as needed:
+You can monitor a specific PostgreSQL cluster using the
+Prometheus Operator's
+PodMonitor resource
+The recommended approach is to manually create and manage a PodMonitor for
+each CloudNativePG cluster. This method provides you with full control over the
+monitoring configuration and lifecycle.
+Creating a PodMonitor
+To monitor your cluster, define a PodMonitor resource as follows. Be sure to
+deploy it in the same namespace where your Prometheus Operator is configured to
+find PodMonitor resources.
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
@@ -425,21 +434,31 @@ Monitoring with the Prometheus
spec:
selector:
matchLabels:
- "cnpg.io/cluster": cluster-example
+ cnpg.io/cluster: cluster-example
podMetricsEndpoints:
- port: metrics
-
Important
-
Ensure you modify the example above with a unique name, as well as the
-correct cluster's namespace and labels (e.g., cluster-example).
+
Important Configuration Details
+
+metadata.name: Give your PodMonitor a unique name.spec.namespaceSelector: Use this to specify the namespace where
+ your PostgreSQL cluster is running.spec.selector.matchLabels: You must use the cnpg.io/cluster: <cluster-name>
+ label to correctly target the PostgreSQL instances.
-
Important
-
The postgresql label, used in previous versions of this document, is deprecated
-and will be removed in the future. Please use the cnpg.io/cluster label
-instead to select the instances.
+
Deprecation of Automatic PodMonitor Creation
+
+
Feature Deprecation Notice
+
The .spec.monitoring.enablePodMonitor field in the Cluster resource is
+now deprecated and will be removed in a future version of the operator.
+
If you are currently using this feature, we strongly recommend you either
+remove or set .spec.monitoring.enablePodMonitor to false and manually
+create a PodMonitor resource for your cluster as described above.
+This change ensures that you have complete ownership of your monitoring
+configuration, preventing it from being managed or overwritten by the operator.
Enabling TLS on the Metrics Port
To enable TLS communication on the metrics port, configure the .spec.monitoring.tls.enabled
setting to true. This setup ensures that the metrics exporter uses the same
@@ -564,7 +583,7 @@
Predefined set of metrics
# HELP cnpg_collector_postgres_version Postgres version
# TYPE cnpg_collector_postgres_version gauge
-cnpg_collector_postgres_version{cluster="cluster-example",full="17.5"} 17.5
+cnpg_collector_postgres_version{cluster="cluster-example",full="18.0"} 18.0
# HELP cnpg_collector_last_failed_backup_timestamp The last failed backup as a unix timestamp
# TYPE cnpg_collector_last_failed_backup_timestamp gauge
@@ -1106,7 +1125,7 @@
Using curl
spec:
containers:
- name: curl
- image: curlimages/curl:8.2.1
+ image: curlimages/curl:8.16.0
command: ['sleep', '3600']
EOF
diff --git a/assets/documentation/1.25/networking/index.html b/assets/documentation/1.25/networking/index.html
index 93f7b1e9d..48b069c6e 100644
--- a/assets/documentation/1.25/networking/index.html
+++ b/assets/documentation/1.25/networking/index.html
@@ -228,6 +228,10 @@
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
@@ -1023,7 +1027,7 @@ Automated recreation of a standby
- « Previous « Previous Next » Restarting the operator to reload configs
pprof HTTP Server
+ Profiling tools
Container Image Requirements
Operator capability levels
Restarting the operator to re
Following the above example, if the Cluster definition contains a categories
annotation and any of the environment, workload, or app labels, these will
be inherited by all the resources generated by the deployment.
-pprof HTTP Server
-The operator can expose a PPROF HTTP server with the following endpoints on localhost:6060:
-
-/debug/pprof/. Responds to a request for "/debug/pprof/" with an HTML page listing the available profiles/debug/pprof/cmdline. Responds with the running program's command line, with arguments separated by NULL bytes./debug/pprof/profile. Responds with the pprof-formatted cpu profile. Profiling lasts for duration specified in seconds GET parameter, or for 30 seconds if not specified./debug/pprof/symbol. Looks up the program counters listed in the request, responding with a table mapping program counters to function names./debug/pprof/trace. Responds with the execution trace in binary form. Tracing lasts for duration specified in seconds GET parameter, or for 1 second if not specified.
-To enable the operator you need to edit the operator deployment add the flag --pprof-server=true.
-You can do this by executing these commands:
+
+The operator can expose a pprof HTTP server on localhost:6060.
+To enable it, edit the operator deployment and add the flag
+--pprof-server=true to the container args:
kubectl edit deployment -n cnpg-system cnpg-controller-manager
Then on the edit page scroll down the container args and add
---pprof-server=true, as in this example:
+Add --pprof-server=true to the args list, for example:
containers:
- args:
- controller
@@ -518,14 +514,26 @@ pprof HTTP Server
command:
- /manager
Save the changes; the deployment now will execute a roll-out, and the new pod
-will have the PPROF server enabled.
-Once the pod is running you can exec inside the container by doing:
-kubectl exec -ti -n cnpg-system <pod name> -- bash
-Once inside execute:
-curl localhost:6060/debug/pprof/
+After saving, the deployment will roll out and the new pod will
+have the pprof server enabled.
+
+
Important
+
The pprof server only serves plain HTTP on port 6060.
+
To access the pprof endpoints from your local machine, use
+port-forwarding:
+kubectl port-forward -n cnpg-system deploy/cnpg-controller-manager 6060
+curl -sS http://localhost:6060/debug/pprof/
+go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30
You can also access pprof using the browser at http://localhost:6060/debug/pprof/ .
+
+
Warning
+
The example above uses kubectl port-forward for local testing only.
+This is not the intended way to expose the feature in production.
+Treat pprof as a sensitive debugging interface and never expose it publicly.
+If you must access it remotely, secure it with proper network policies and access controls.
+
diff --git a/assets/documentation/1.25/postgis/index.html b/assets/documentation/1.25/postgis/index.html
index 2fef75b38..3ae7e83ed 100644
--- a/assets/documentation/1.25/postgis/index.html
+++ b/assets/documentation/1.25/postgis/index.html
@@ -234,6 +234,10 @@
Container Image Requirements
Operator capability levels
PostGIS
in Kubernetes via CloudNativePG.
The CloudNativePG Community maintains container images that are built on top
-of the official PostGIS images hosted on DockerHub .
-For more information please visit:
+of the maintained PostgreSQL Container images .
+For more information, please visit:
The postgis-containers project in GitHub
The postgis-containers Container Registry in GitHub
@@ -345,18 +349,18 @@ Basic concepts about a PostGIS c
install it in the application database, which is the main and supposedly only
database you host in the cluster according to the microservice architecture, or
-install it in the template1 database so as to make it available for all the
+ install it in the template1 database to make it available for all the
databases you end up creating in the cluster, in case you adopt the monolith
architecture where the instance is shared by multiple databases
Create a new PostgreSQL cluster with PostGIS
- Let's suppose you want to create a new PostgreSQL 14 cluster with PostGIS 3.2.
+Let's suppose you want to create a new PostgreSQL 18 cluster with PostGIS 3.6.
The first step is to ensure you use the right PostGIS container image for the
operand, and properly set the .spec.imageName option in the Cluster
resource.
@@ -366,17 +370,18 @@ Create a new PostgreSQL cl
Warning
Please consider that, although convention over configuration applies in
CloudNativePG, you should spend time configuring and tuning your system for
-production. Also the imageName in the example below deliberately points
-to the latest available image for PostgreSQL 14 - you should use a specific
-image name or, preferably, the SHA256 digest for true immutability.
+production. Also, the imageName in the example below deliberately points
+to the latest available image for PostgreSQL 18 - you should use a specific
+image name or, preferably, the SHA256 digest for true immutability.
+Alternatively, use the provided image catalogs .
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: postgis-example
spec:
- instances: 3
- imageName: ghcr.io/cloudnative-pg/postgis:14
+ instances: 1
+ imageName: ghcr.io/cloudnative-pg/postgis:18-3.6-system-trixie
bootstrap:
initdb:
postInitTemplateSQL:
@@ -402,45 +407,44 @@ Create a new PostgreSQL cl
You can easily verify the available version of PostGIS that is in the
container, by connecting to the app database (you might obtain different
values from the ones in this document):
-$ kubectl exec -ti postgis-example-1 -- psql app
-Defaulted container "postgres" out of: postgres, bootstrap-controller (init)
-psql (17.5 (Debian 17.5-1.pgdg110+1))
+$ kubectl cnpg psql postgis-example -- app
+psql (18.0 (Debian 18.0-1.pgdg13+3))
Type "help" for help.
app=# SELECT * FROM pg_available_extensions WHERE name ~ '^postgis' ORDER BY 1;
name | default_version | installed_version | comment
--------------------------+-----------------+-------------------+------------------------------------------------------------
- postgis | 3.2.2 | 3.2.2 | PostGIS geometry and geography spatial types and functions
- postgis-3 | 3.2.2 | | PostGIS geometry and geography spatial types and functions
- postgis_raster | 3.2.2 | | PostGIS raster types and functions
- postgis_raster-3 | 3.2.2 | | PostGIS raster types and functions
- postgis_sfcgal | 3.2.2 | | PostGIS SFCGAL functions
- postgis_sfcgal-3 | 3.2.2 | | PostGIS SFCGAL functions
- postgis_tiger_geocoder | 3.2.2 | 3.2.2 | PostGIS tiger geocoder and reverse geocoder
- postgis_tiger_geocoder-3 | 3.2.2 | | PostGIS tiger geocoder and reverse geocoder
- postgis_topology | 3.2.2 | 3.2.2 | PostGIS topology spatial types and functions
- postgis_topology-3 | 3.2.2 | | PostGIS topology spatial types and functions
+ postgis | 3.6.0 | 3.6.0 | PostGIS geometry and geography spatial types and functions
+ postgis-3 | 3.6.0 | | PostGIS geometry and geography spatial types and functions
+ postgis_raster | 3.6.0 | | PostGIS raster types and functions
+ postgis_raster-3 | 3.6.0 | | PostGIS raster types and functions
+ postgis_sfcgal | 3.6.0 | | PostGIS SFCGAL functions
+ postgis_sfcgal-3 | 3.6.0 | | PostGIS SFCGAL functions
+ postgis_tiger_geocoder | 3.6.0 | 3.6.0 | PostGIS tiger geocoder and reverse geocoder
+ postgis_tiger_geocoder-3 | 3.6.0 | | PostGIS tiger geocoder and reverse geocoder
+ postgis_topology | 3.6.0 | 3.6.0 | PostGIS topology spatial types and functions
+ postgis_topology-3 | 3.6.0 | | PostGIS topology spatial types and functions
(10 rows)
The next step is to verify that the extensions listed in the
postInitTemplateSQL section have been correctly installed in the app
database.
app=# \dx
- List of installed extensions
- Name | Version | Schema | Description
-------------------------+---------+------------+------------------------------------------------------------
- fuzzystrmatch | 1.1 | public | determine similarities and distance between strings
- plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
- postgis | 3.2.2 | public | PostGIS geometry and geography spatial types and functions
- postgis_tiger_geocoder | 3.2.2 | tiger | PostGIS tiger geocoder and reverse geocoder
- postgis_topology | 3.2.2 | topology | PostGIS topology spatial types and functions
-(5 rows)
+ List of installed extensions
+ Name | Version | Default version | Schema | Description
+------------------------+---------+-----------------+------------+------------------------------------------------------------
+ fuzzystrmatch | 1.2 | 1.2 | public | determine similarities and distance between strings
+ plpgsql | 1.0 | 1.0 | pg_catalog | PL/pgSQL procedural language
+ postgis | 3.6.0 | 3.6.0 | public | PostGIS geometry and geography spatial types and functions
+ postgis_tiger_geocoder | 3.6.0 | 3.6.0 | tiger | PostGIS tiger geocoder and reverse geocoder
+ postgis_topology | 3.6.0 | 3.6.0 | topology | PostGIS topology spatial types and functions
Finally:
app=# SELECT postgis_full_version();
postgis_full_version
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- POSTGIS="3.2.2 628da50" [EXTENSION] PGSQL="140" GEOS="3.9.0-CAPI-1.16.2" PROJ="7.2.1" LIBXML="2.9.10" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY
+ POSTGIS="3.6.0 4c1967d" [EXTENSION] PGSQL="180" GEOS="3.13.1-CAPI-1.19.2" PROJ="9.6.0 NETWORK_ENABLED=OFF URL_ENDPOINT=https://cdn.proj.org USER_WRITABLE_DIRECTORY=/tmp/proj DATABASE_PATH=/usr/share/proj/proj.
+db" (compiled against PROJ 9.6.0) LIBXML="2.9.14" LIBJSON="0.18" LIBPROTOBUF="1.5.1" WAGYU="0.5.0 (Internal)" TOPOLOGY
(1 row)
Container Image Requirements
Operator capability levels
Replication Settings
Log control settings
The operator requires PostgreSQL to output its log in CSV format, and the
instance manager automatically parses it and outputs it in JSON format.
-For this reason, all log settings in PostgreSQL are fixed and cannot be
-changed.
+As a result, certain PostgreSQL log settings, listed in this section ,
+are fixed and cannot be modified.
For further information, please refer to the "Logging" section .
Shared Preload Libraries
The shared_preload_libraries option in PostgreSQL exists to specify one or
diff --git a/assets/documentation/1.25/preview_version/index.html b/assets/documentation/1.25/preview_version/index.html
index 83c23637b..ecd73c64e 100644
--- a/assets/documentation/1.25/preview_version/index.html
+++ b/assets/documentation/1.25/preview_version/index.html
@@ -228,6 +228,10 @@
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Viewing with Prometheus
storage:
size: 1Gi
-
- monitoring:
- enablePodMonitor: true
+---
+apiVersion: monitoring.coreos.com/v1
+kind: PodMonitor
+metadata:
+ name: cluster-with-metrics
+spec:
+ selector:
+ matchLabels:
+ cnpg.io/cluster: cluster-with-metrics
+ podMetricsEndpoints:
+ - port: metrics
EOF
To access Prometheus, port-forward the Prometheus service:
diff --git a/assets/documentation/1.25/recovery/index.html b/assets/documentation/1.25/recovery/index.html
index 51ca08afc..f88c2bcd5 100644
--- a/assets/documentation/1.25/recovery/index.html
+++ b/assets/documentation/1.25/recovery/index.html
@@ -254,6 +254,10 @@
Container Image Requirements
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Release notes for CloudNativePG 1.2
For a complete list of changes, please refer to the
commits
on the release branch in GitHub.
+Version 1.25.4
+Release date: Oct 23, 2025
+
+
Warning
+
This is the final release in the 1.25.x series.
+Users are strongly encouraged to upgrade to a newer minor version, as 1.25
+is no longer supported.
+
Changes
+
+
+Adopted the new format of postgres-containers and postgis-containers
+ images and image catalog artifacts, and updated the default PostgreSQL
+ version to 18.0-system-trixie (PostgreSQL 18 is now supported).
+ (#8578 ,
+ #8760 ,
+ #8558 )
+
+
+Deprecated the monitoring.enablePodMonitor field in the Cluster and
+ Pooler resources. This field will be removed in a future release. Users who
+ rely on PodMonitor resources should create them manually instead.
+ (#8753 )
+
+
+Enhancements
+
+
+Added support for overriding the PgBouncer auth_type, server_tls_sslmode,
+ and client_tls_sslmode settings, which were previously hardcoded. Default
+ values remain consistent with the former behavior but can now be customized
+ when required.
+ (#8674 )
+
+
+Added a CHECKPOINT step before PostgreSQL smart and fast shutdowns to
+ reduce shutdown duration and replica promotion time, especially on systems
+ with a high checkpoint_timeout.
+ (#8867 )
+
+
+Added a warning in the instance manager for deprecated or unsupported OS
+ versions, based on the official postgres-containers project.
+ (#8601 )
+
+
+Improved certificate parsing error reporting. Failures now log specific
+ errors instead of a generic message, aiding troubleshooting. This is
+ particularly relevant after the CVE-2025-58187 fix in Go 1.25.2 and 1.24.8,
+ which may trigger parsing failures for invalid DNS SANs.
+ (#8801 )
+
+
+Added a check to ensure the destination WAL archive path is empty when
+ bootstrapping a cluster using the pg_basebackup method, consistent with
+ other bootstrap methods.
+ (#8895 )
+
+
+Added validation to prevent backups from running on hibernated clusters.
+ Backups attempted on such clusters now fail with reason
+ ClusterIsHibernated, following the standard prerequisite check pattern.
+ (#8870 )
+
+
+Added support for pprof profiling. Instances can now enable the pprof
+ tool by adding the alpha.cnpg.io/enableInstancePprof annotation to the
+ Cluster resource for advanced debugging.
+ (#7876 )
+
+
+cnpg plugin:
+
+
+Updated the Flexible I/O Tester (FIO) image to
+ wallnerryan/fiotools-aio:v2, as provided by Ryan Wallner.
+ (#8847 )
+
+
+Enhanced the cnpg status backup command to provide more detailed status
+ information when using a barman-cloud-based backup plugin.
+ (#8780 ,
+ #8690 )
+
+
+Fixes
+
+
+Fixed backup restoration failures when using custom WAL segment sizes with
+ parallel WAL recovery. The operator no longer manages the end-of-WAL file
+ marker during restoration, preventing errors when backups span multiple WAL
+ segments.
+ (#8873 )
+
+
+Fixed a bug in major upgrades where a volume snapshot from a previous minor
+ version could be incorrectly used to optimize replica creation.
+ (#8475 )
+
+
+Fixed initdb to wait for the application user secret before bootstrapping
+ a new cluster, preventing potential race conditions.
+ (#8663 )
+
+
+Fixed the connection retry logic in the cnpgi plugin. The reconciliation
+ loop now detects connection pool changes correctly and uses exponential
+ backoff to reduce "closed pool" errors.
+ (#8554 )
+
+
+Fixed volume snapshot usage during replica scaling to work with backup plugins.
+ Previously, this optimization was only available with the in-tree backup
+ implementation, but now clusters using backup plugins can also leverage volume
+ snapshots when creating new replicas.
+ (#8506 )
+
+
+Fixed the Pooler templating to correctly inherit settings for the
+ bootstrap controller init container.
+ (#8394 )
+
+
+Fixed webhook errors to use the correct API group (postgresql.cnpg.io) in
+ Pooler and backup webhooks, ensuring consistent API error reporting.
+ (#8485 )
+
+
+Fixed a potential nil pointer dereference in the hibernation reconciler when
+ handling errors. Contributed by @PascalBourdier.
+ (#8756 )
+
+
+Fixed an issue in the environment cache where callers could inadvertently
+ modify shared data. The LoadEnv function now returns a copy of cached
+ environment slices to prevent mutations from affecting the cache.
+ (#8880 )
+
+
Version 1.25.3
Release date: Jul 25, 2025
In memory of DJ Walker-Morgan .
-Changes
+Changes
Removed 386 and ARM (v5/v6/v7) architectures from the cnpg plugin build
matrix, reducing the number of published binaries
(#7648 ).
-Enhancements
+Enhancements
-Fixes
+Fixes
-Enhancements
+Enhancements
-Fixes
+Fixes
-Changes
+Changes
Version 1.25.1
Release Date: February 28, 2025
-Enhancements
+Enhancements
Introduced a startup probe for the operator to enhance reliability and
prevent premature liveness probe failures during initialization. (#7008)
@@ -635,7 +778,7 @@ Features
feedback and contributions to shape this exciting new capability.
-Enhancements
+Enhancements
Add the dataDurability option to the .spec.postgresql.synchronous stanza,
allowing users to choose between required (default) or preferred
diff --git a/assets/documentation/1.25/replica_cluster/index.html b/assets/documentation/1.25/replica_cluster/index.html
index 766a7b958..70506a66b 100644
--- a/assets/documentation/1.25/replica_cluster/index.html
+++ b/assets/documentation/1.25/replica_cluster/index.html
@@ -272,6 +272,10 @@
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Container Image Requirements
+
Operator capability levels
Pod Affinity and Anti-Affinity
name: cluster-example
spec:
instances: 3
- imageName: ghcr.io/cloudnative-pg/postgresql:17.5
+ imageName: ghcr.io/cloudnative-pg/postgresql:18.0-system-trixie
affinity:
enablePodAntiAffinity: true # Default value
diff --git a/assets/documentation/1.25/search.html b/assets/documentation/1.25/search.html
index 6a9d2be92..065e11ec8 100644
--- a/assets/documentation/1.25/search.html
+++ b/assets/documentation/1.25/search.html
@@ -221,6 +221,10 @@
Container Image Requirements
+
Operator capability levels
node-role.kubernetes.io/postgres= To ensure that a Cluster resource is scheduled on a postgres node, you must correctly configure the .spec.affinity.nodeSelector stanza in your manifests. Here\u2019s an example: spec: # affinity: # nodeSelector: node-role.kubernetes.io/postgres: \"\" Proposed node taint CloudNativePG recommends using the node-role.kubernetes.io/postgres taint. To assign the postgres taint to a node, use the following command: kubectl taint node node-role.kubernetes.io/postgres=:NoSchedule To ensure that a Cluster resource is scheduled on a node with a postgres taint, you must correctly configure the .spec.affinity.tolerations stanza in your manifests. Here\u2019s an example: spec: # affinity: # tolerations: - key: node-role.kubernetes.io/postgres operator: Exists effect: NoSchedule PostgreSQL architecture CloudNativePG supports clusters based on asynchronous and synchronous streaming replication to manage multiple hot standby replicas within the same Kubernetes cluster, with the following specifications: One primary, with optional multiple hot standby replicas for HA Available services for applications: -rw : applications connect only to the primary instance of the cluster -ro : applications connect only to hot standby replicas for read-only-workloads (optional) -r : applications connect to any of the instances for read-only workloads (optional) Shared-nothing architecture recommended for better resilience of the PostgreSQL cluster: PostgreSQL instances should reside on different Kubernetes worker nodes and share only the network - as a result, instances should not share the storage and preferably use local volumes attached to the node they run on PostgreSQL instances should reside in different availability zones within the same Kubernetes cluster / region Important You can configure the above services through the managed.services section in the Cluster configuration. This can be done by reducing the number of services and selecting the type (default is ClusterIP ). For more details, please refer to the \"Service Management\" section below. The below diagram provides a simplistic view of the recommended shared-nothing architecture for a PostgreSQL cluster spanning across 3 different availability zones, running on separate nodes, each with dedicated local storage for PostgreSQL data. CloudNativePG automatically takes care of updating the above services if the topology of the cluster changes. For example, in case of failover, it automatically updates the -rw service to point to the promoted primary, making sure that traffic from the applications is seamlessly redirected. Replication Please refer to the \"Replication\" section for more information about how CloudNativePG relies on PostgreSQL replication, including synchronous settings. Connecting from an application Please refer to the \"Connecting from an application\" section for information about how to connect to CloudNativePG from a stateless application within the same Kubernetes cluster. Connection Pooling Please refer to the \"Connection Pooling\" section for information about how to take advantage of PgBouncer as a connection pooler, and create an access layer between your applications and the PostgreSQL clusters. Read-write workloads Applications can decide to connect to the PostgreSQL instance elected as current primary by the Kubernetes operator, as depicted in the following diagram: Applications can use the -rw suffix service. In case of temporary or permanent unavailability of the primary, for High Availability purposes CloudNativePG will trigger a failover, pointing the -rw service to another instance of the cluster. Read-only workloads Important Applications must be aware of the limitations that Hot Standby presents and familiar with the way PostgreSQL operates when dealing with these workloads. Applications can access hot standby replicas through the -ro service made available by the operator. This service enables the application to offload read-only queries from the primary node. The following diagram shows the architecture: Applications can also access any PostgreSQL instance through the -r service. Deployments across Kubernetes clusters Info CloudNativePG supports deploying PostgreSQL across multiple Kubernetes clusters through a feature that allows you to define a distributed PostgreSQL topology using replica clusters, as described in this section. In a distributed PostgreSQL cluster there can only be a single PostgreSQL instance acting as a primary at all times. This means that applications can only write inside a single Kubernetes cluster, at any time. However, for business continuity objectives it is fundamental to: reduce global recovery point objectives ( RPO ) by storing PostgreSQL backup data in multiple locations, regions and possibly using different providers (Disaster Recovery) reduce global recovery time objectives ( RTO ) by taking advantage of PostgreSQL replication beyond the primary Kubernetes cluster (High Availability) In order to address the above concerns, CloudNativePG introduces the concept of a PostgreSQL Topology that is distributed across different Kubernetes clusters and is made up of a primary PostgreSQL cluster and one or more PostgreSQL replica clusters. This feature is called distributed PostgreSQL topology with replica clusters , and it enables multi-cluster deployments in private, public, hybrid, and multi-cloud contexts. A replica cluster is a separate Cluster resource that is in continuous recovery, replicating from another source, either via WAL shipping from a WAL archive or via streaming replication from a primary or a standby (cascading). The diagram below depicts a PostgreSQL cluster spanning over two different Kubernetes clusters, where the primary cluster is in the first Kubernetes cluster and the replica cluster is in the second. The second Kubernetes cluster acts as the company's disaster recovery cluster, ready to be activated in case of disaster and unavailability of the first one. A replica cluster can have the same architecture as the primary cluster. Instead of a primary instance, a replica cluster has a designated primary instance, which is a standby server with an arbitrary number of cascading standby servers in streaming replication (symmetric architecture). The designated primary can be promoted at any time, transforming the replica cluster into a primary cluster capable of accepting write connections. This is typically triggered by: Human decision: You choose to make the other PostgreSQL cluster (or the entire Kubernetes cluster) the primary. To avoid data loss and ensure that the former primary can follow without being re-cloned (especially with large data sets), you first demote the current primary, then promote the designated primary using the API provided by CloudNativePG. Unexpected failure: If the entire Kubernetes cluster fails, you might experience data loss, but you need to fail over to the other Kubernetes cluster by promoting the PostgreSQL replica cluster. Warning CloudNativePG cannot perform any cross-cluster automated failover, as it does not have authority beyond a single Kubernetes cluster. Such operations must be performed manually or delegated to a multi-cluster/federated cluster-aware authority. Important CloudNativePG allows you to control the distributed topology via declarative configuration, enabling you to automate these procedures as part of your Infrastructure as Code (IaC) process, including GitOps. The designated primary in the above example is fed via WAL streaming ( primary_conninfo ), with fallback option for file-based WAL shipping through the restore_command and barman-cloud-wal-restore . CloudNativePG allows you to define topologies with multiple replica clusters. You can also define replica clusters with a lower number of replicas, and then increase this number when the cluster is promoted to primary. Replica clusters Please refer to the \"Replica Clusters\" section for more detailed information on how physical replica clusters operate and how to define a distributed topology with read-only clusters across different Kubernetes clusters. This approach can significantly enhance your global disaster recovery and high availability (HA) strategy.","title":"Architecture"},{"location":"architecture/#architecture","text":"Hint For a deeper understanding, we recommend reading our article on the CNCF blog post titled \"Recommended Architectures for PostgreSQL in Kubernetes\" , which provides valuable insights into best practices and design considerations for PostgreSQL deployments in Kubernetes. This documentation page provides an overview of the key architectural considerations for implementing a robust business continuity strategy when deploying PostgreSQL in Kubernetes. These considerations include: Deployments in stretched vs. non-stretched clusters : Evaluating the differences between deploying in stretched clusters (across 3 or more availability zones) versus non-stretched clusters (within a single availability zone). Reservation of postgres worker nodes : Isolating PostgreSQL workloads by dedicating specific worker nodes to postgres tasks, ensuring optimal performance and minimizing interference from other workloads. PostgreSQL architectures within a single Kubernetes cluster : Designing effective PostgreSQL deployments within a single Kubernetes cluster to meet high availability and performance requirements. PostgreSQL architectures across Kubernetes clusters for disaster recovery : Planning and implementing PostgreSQL architectures that span multiple Kubernetes clusters to provide comprehensive disaster recovery capabilities.","title":"Architecture"},{"location":"architecture/#synchronizing-the-state","text":"PostgreSQL is a database management system and, as such, it needs to be treated as a stateful workload in Kubernetes. While stateless applications mainly use traffic redirection to achieve High Availability (HA) and Disaster Recovery (DR), in the case of a database, state must be replicated in multiple locations, preferably in a continuous and instantaneous way, by adopting either of the following two strategies: storage-level replication , normally persistent volumes application-level replication , in this specific case PostgreSQL CloudNativePG relies on application-level replication, for a simple reason: the PostgreSQL database management system comes with robust and reliable built-in physical replication capabilities based on Write Ahead Log (WAL) shipping , which have been used in production by millions of users all over the world for over a decade. PostgreSQL supports both asynchronous and synchronous streaming replication over the network, as well as asynchronous file-based log shipping (normally used as a fallback option, for example, to store WAL files in an object store). Replicas are usually called standby servers and can also be used for read-only workloads, thanks to the Hot Standby feature. Important We recommend against storage-level replication with PostgreSQL , although CloudNativePG allows you to adopt that strategy. For more information, please refer to the talk given by Chris Milsted and Gabriele Bartolini at KubeCon NA 2022 entitled \"Data On Kubernetes, Deploying And Running PostgreSQL And Patterns For Databases In a Kubernetes Cluster\" where this topic was covered in detail.","title":"Synchronizing the state"},{"location":"architecture/#kubernetes-architecture","text":"Kubernetes natively provides the possibility to span separate physical locations - also known as data centers, failure zones, or more frequently availability zones - connected to each other via redundant, low-latency, private network connectivity. Being a distributed system, the recommended minimum number of availability zones for a Kubernetes cluster is three (3), in order to make the control plane resilient to the failure of a single zone. For details, please refer to \"Running in multiple zones\" . This means that each data center is active at any time and can run workloads simultaneously. Note Most of the public Cloud Providers' managed Kubernetes services already provide 3 or more availability zones in each region.","title":"Kubernetes architecture"},{"location":"architecture/#multi-availability-zone-kubernetes-clusters","text":"The multi-availability zone Kubernetes architecture with three (3) or more zones is the one that we recommend for PostgreSQL usage. This scenario is typical of Kubernetes services managed by Cloud Providers. Such an architecture enables the CloudNativePG operator to control the full lifecycle of a Cluster resource across the zones within a single Kubernetes cluster, by treating all the availability zones as active: this includes, among other operations, scheduling the workloads in a declarative manner (based on affinity rules, tolerations and node selectors), automated failover, self-healing, and updates. All will work seamlessly across the zones in a single Kubernetes cluster. Please refer to the \"PostgreSQL architecture\" section below for details on how you can design your PostgreSQL clusters within the same Kubernetes cluster through shared-nothing deployments at the storage, worker node, and availability zone levels. Additionally, you can leverage Kubernetes clusters to deploy distributed PostgreSQL topologies hosting \"passive\" PostgreSQL replica clusters in different regions and managing them via declarative configuration. This setup is ideal for disaster recovery (DR), read-only operations, or cross-region availability. Important Each operator deployment can only manage operations within its local Kubernetes cluster. For operations across Kubernetes clusters, such as controlled switchover or unexpected failover, coordination must be handled manually (through GitOps, for example) or by using a higher-level cluster management tool.","title":"Multi-availability zone Kubernetes clusters"},{"location":"architecture/#single-availability-zone-kubernetes-clusters","text":"If your Kubernetes cluster has only one availability zone, CloudNativePG still provides you with a lot of features to improve HA and DR outcomes for your PostgreSQL databases, pushing the single point of failure (SPoF) to the level of the zone as much as possible - i.e. the zone must have an outage before your CloudNativePG clusters suffer a failure. This scenario is typical of self-managed on-premise Kubernetes clusters, where only one data center is available. Single availability zone Kubernetes clusters are the only viable option when only two data centers are available within reach of a low-latency connection (typically in the same metropolitan area). Having only two zones prevents the creation of a multi-availability zone Kubernetes cluster, which requires a minimum of three zones. As a result, users must create two separate Kubernetes clusters in an active/passive configuration, with the second cluster primarily used for Disaster Recovery (see the replica cluster feature ). Hint If you are at an early stage of your Kubernetes journey, please share this document with your infrastructure team. The two data centers setup might be simply the result of a \"lift-and-shift\" transition to Kubernetes from a traditional bare-metal or VM based infrastructure, and the benefits that Kubernetes offers in a 3+ zone scenario might not have been known, or addressed at the time the infrastructure architecture was designed. Ultimately, a third physical location connected to the other two might represent a valid option to consider for organization, as it reduces the overall costs of the infrastructure by moving the day-to-day complexity from the application level down to the physical infrastructure level. Please refer to the \"PostgreSQL architecture\" section below for details on how you can design your PostgreSQL clusters within your single availability zone Kubernetes cluster through shared-nothing deployments at the storage and worker node levels only. For HA, in such a scenario it becomes even more important that the PostgreSQL instances be located on different worker nodes and do not share the same storage. For DR, you can push the SPoF above the single zone, by using additional Kubernetes clusters to define a distributed topology hosting \"passive\" PostgreSQL replica clusters . As with other Kubernetes workloads in this scenario, promotion of a Kubernetes cluster as primary must be done manually. Through the replica cluster feature , you can define a distributed PostgreSQL topology and coordinate a controlled switchover between data centers by first demoting the primary cluster and then promoting the replica cluster, without the need to re-clone the former primary. While failover is now fully declarative, automated failover across Kubernetes clusters is not within CloudNativePG's scope, as the operator can only function within a single Kubernetes cluster. Important CloudNativePG provides all the necessary primitives and probes to coordinate PostgreSQL active/passive topologies across different Kubernetes clusters through a higher-level operator or management tool.","title":"Single availability zone Kubernetes clusters"},{"location":"architecture/#reserving-nodes-for-postgresql-workloads","text":"Whether you're operating in a multi-availability zone environment or, more critically, within a single availability zone, we strongly recommend isolating PostgreSQL workloads by dedicating specific worker nodes exclusively to postgres in production. A Kubernetes worker node dedicated to running PostgreSQL workloads is referred to as a Postgres node or postgres node. This approach ensures optimal performance and resource allocation for your database operations. Hint As a general rule of thumb, deploy Postgres nodes in multiples of three\u2014ideally with one node per availability zone. Three nodes is an optimal number because it ensures that a PostgreSQL cluster with three instances (one primary and two standby replicas) is distributed across different nodes, enhancing fault tolerance and availability. In Kubernetes, this can be achieved using node labels and taints in a declarative manner, aligning with Infrastructure as Code (IaC) practices: labels ensure that a node is capable of running postgres workloads, while taints help prevent any non- postgres workloads from being scheduled on that node. Important This methodology is the most straightforward way to ensure that PostgreSQL workloads are isolated from other workloads in terms of both computing resources and, when using locally attached disks, storage. While different PostgreSQL clusters may share the same node, you can take this a step further by using labels and taints to ensure that a node is dedicated to a single instance of a specific Cluster .","title":"Reserving nodes for PostgreSQL workloads"},{"location":"architecture/#proposed-node-label","text":"CloudNativePG recommends using the node-role.kubernetes.io/postgres label. Since this is a reserved label ( *.kubernetes.io ), it can only be applied after a worker node is created. To assign the postgres label to a node, use the following command: kubectl label node node-role.kubernetes.io/postgres= To ensure that a Cluster resource is scheduled on a postgres node, you must correctly configure the .spec.affinity.nodeSelector stanza in your manifests. Here\u2019s an example: spec: # affinity: # nodeSelector: node-role.kubernetes.io/postgres: \"\"","title":"Proposed node label"},{"location":"architecture/#proposed-node-taint","text":"CloudNativePG recommends using the node-role.kubernetes.io/postgres taint. To assign the postgres taint to a node, use the following command: kubectl taint node node-role.kubernetes.io/postgres=:NoSchedule To ensure that a Cluster resource is scheduled on a node with a postgres taint, you must correctly configure the .spec.affinity.tolerations stanza in your manifests. Here\u2019s an example: spec: # affinity: # tolerations: - key: node-role.kubernetes.io/postgres operator: Exists effect: NoSchedule","title":"Proposed node taint"},{"location":"architecture/#postgresql-architecture","text":"CloudNativePG supports clusters based on asynchronous and synchronous streaming replication to manage multiple hot standby replicas within the same Kubernetes cluster, with the following specifications: One primary, with optional multiple hot standby replicas for HA Available services for applications: -rw : applications connect only to the primary instance of the cluster -ro : applications connect only to hot standby replicas for read-only-workloads (optional) -r : applications connect to any of the instances for read-only workloads (optional) Shared-nothing architecture recommended for better resilience of the PostgreSQL cluster: PostgreSQL instances should reside on different Kubernetes worker nodes and share only the network - as a result, instances should not share the storage and preferably use local volumes attached to the node they run on PostgreSQL instances should reside in different availability zones within the same Kubernetes cluster / region Important You can configure the above services through the managed.services section in the Cluster configuration. This can be done by reducing the number of services and selecting the type (default is ClusterIP ). For more details, please refer to the \"Service Management\" section below. The below diagram provides a simplistic view of the recommended shared-nothing architecture for a PostgreSQL cluster spanning across 3 different availability zones, running on separate nodes, each with dedicated local storage for PostgreSQL data. CloudNativePG automatically takes care of updating the above services if the topology of the cluster changes. For example, in case of failover, it automatically updates the -rw service to point to the promoted primary, making sure that traffic from the applications is seamlessly redirected. Replication Please refer to the \"Replication\" section for more information about how CloudNativePG relies on PostgreSQL replication, including synchronous settings. Connecting from an application Please refer to the \"Connecting from an application\" section for information about how to connect to CloudNativePG from a stateless application within the same Kubernetes cluster. Connection Pooling Please refer to the \"Connection Pooling\" section for information about how to take advantage of PgBouncer as a connection pooler, and create an access layer between your applications and the PostgreSQL clusters.","title":"PostgreSQL architecture"},{"location":"architecture/#read-write-workloads","text":"Applications can decide to connect to the PostgreSQL instance elected as current primary by the Kubernetes operator, as depicted in the following diagram: Applications can use the -rw suffix service. In case of temporary or permanent unavailability of the primary, for High Availability purposes CloudNativePG will trigger a failover, pointing the -rw service to another instance of the cluster.","title":"Read-write workloads"},{"location":"architecture/#read-only-workloads","text":"Important Applications must be aware of the limitations that Hot Standby presents and familiar with the way PostgreSQL operates when dealing with these workloads. Applications can access hot standby replicas through the -ro service made available by the operator. This service enables the application to offload read-only queries from the primary node. The following diagram shows the architecture: Applications can also access any PostgreSQL instance through the -r service.","title":"Read-only workloads"},{"location":"architecture/#deployments-across-kubernetes-clusters","text":"Info CloudNativePG supports deploying PostgreSQL across multiple Kubernetes clusters through a feature that allows you to define a distributed PostgreSQL topology using replica clusters, as described in this section. In a distributed PostgreSQL cluster there can only be a single PostgreSQL instance acting as a primary at all times. This means that applications can only write inside a single Kubernetes cluster, at any time. However, for business continuity objectives it is fundamental to: reduce global recovery point objectives ( RPO ) by storing PostgreSQL backup data in multiple locations, regions and possibly using different providers (Disaster Recovery) reduce global recovery time objectives ( RTO ) by taking advantage of PostgreSQL replication beyond the primary Kubernetes cluster (High Availability) In order to address the above concerns, CloudNativePG introduces the concept of a PostgreSQL Topology that is distributed across different Kubernetes clusters and is made up of a primary PostgreSQL cluster and one or more PostgreSQL replica clusters. This feature is called distributed PostgreSQL topology with replica clusters , and it enables multi-cluster deployments in private, public, hybrid, and multi-cloud contexts. A replica cluster is a separate Cluster resource that is in continuous recovery, replicating from another source, either via WAL shipping from a WAL archive or via streaming replication from a primary or a standby (cascading). The diagram below depicts a PostgreSQL cluster spanning over two different Kubernetes clusters, where the primary cluster is in the first Kubernetes cluster and the replica cluster is in the second. The second Kubernetes cluster acts as the company's disaster recovery cluster, ready to be activated in case of disaster and unavailability of the first one. A replica cluster can have the same architecture as the primary cluster. Instead of a primary instance, a replica cluster has a designated primary instance, which is a standby server with an arbitrary number of cascading standby servers in streaming replication (symmetric architecture). The designated primary can be promoted at any time, transforming the replica cluster into a primary cluster capable of accepting write connections. This is typically triggered by: Human decision: You choose to make the other PostgreSQL cluster (or the entire Kubernetes cluster) the primary. To avoid data loss and ensure that the former primary can follow without being re-cloned (especially with large data sets), you first demote the current primary, then promote the designated primary using the API provided by CloudNativePG. Unexpected failure: If the entire Kubernetes cluster fails, you might experience data loss, but you need to fail over to the other Kubernetes cluster by promoting the PostgreSQL replica cluster. Warning CloudNativePG cannot perform any cross-cluster automated failover, as it does not have authority beyond a single Kubernetes cluster. Such operations must be performed manually or delegated to a multi-cluster/federated cluster-aware authority. Important CloudNativePG allows you to control the distributed topology via declarative configuration, enabling you to automate these procedures as part of your Infrastructure as Code (IaC) process, including GitOps. The designated primary in the above example is fed via WAL streaming ( primary_conninfo ), with fallback option for file-based WAL shipping through the restore_command and barman-cloud-wal-restore . CloudNativePG allows you to define topologies with multiple replica clusters. You can also define replica clusters with a lower number of replicas, and then increase this number when the cluster is promoted to primary. Replica clusters Please refer to the \"Replica Clusters\" section for more detailed information on how physical replica clusters operate and how to define a distributed topology with read-only clusters across different Kubernetes clusters. This approach can significantly enhance your global disaster recovery and high availability (HA) strategy.","title":"Deployments across Kubernetes clusters"},{"location":"backup/","text":"Backup PostgreSQL natively provides first class backup and recovery capabilities based on file system level (physical) copy. These have been successfully used for more than 15 years in mission critical production databases, helping organizations all over the world achieve their disaster recovery goals with Postgres. Note There's another way to backup databases in PostgreSQL, through the pg_dump utility - which relies on logical backups instead of physical ones. However, logical backups are not suitable for business continuity use cases and as such are not covered by CloudNativePG (yet, at least). If you want to use the pg_dump utility, let yourself be inspired by the \"Troubleshooting / Emergency backup\" section . In CloudNativePG, the backup infrastructure for each PostgreSQL cluster is made up of the following resources: WAL archive : a location containing the WAL files (transactional logs) that are continuously written by Postgres and archived for data durability Physical base backups : a copy of all the files that PostgreSQL uses to store the data in the database (primarily the PGDATA and any tablespace) The WAL archive can only be stored on object stores at the moment. On the other hand, CloudNativePG supports two ways to store physical base backups: on object stores , as tarballs - optionally compressed on Kubernetes Volume Snapshots , if supported by the underlying storage class Important Before choosing your backup strategy with CloudNativePG, it is important that you take some time to familiarize with some basic concepts, like WAL archive, hot and cold backups. Important Please refer to the official Kubernetes documentation for a list of all the supported Container Storage Interface (CSI) drivers that provide snapshotting capabilities. Info Starting with version 1.25, CloudNativePG includes experimental support for backup and recovery using plugins, such as the Barman Cloud plugin . WAL archive The WAL archive in PostgreSQL is at the heart of continuous backup , and it is fundamental for the following reasons: Hot backups : the possibility to take physical base backups from any instance in the Postgres cluster (either primary or standby) without shutting down the server; they are also known as online backups Point in Time recovery (PITR): to possibility to recover at any point in time from the first available base backup in your system Warning WAL archive alone is useless. Without a physical base backup, you cannot restore a PostgreSQL cluster. In general, the presence of a WAL archive enhances the resilience of a PostgreSQL cluster, allowing each instance to fetch any required WAL file from the archive if needed (normally the WAL archive has higher retention periods than any Postgres instance that normally recycles those files). This use case can also be extended to replica clusters , as they can simply rely on the WAL archive to synchronize across long distances, extending disaster recovery goals across different regions. When you configure a WAL archive , CloudNativePG provides out-of-the-box an RPO <= 5 minutes for disaster recovery, even across regions. Important Our recommendation is to always setup the WAL archive in production. There are known use cases - normally involving staging and development environments - where none of the above benefits are needed and the WAL archive is not necessary. RPO in this case can be any value, such as 24 hours (daily backups) or infinite (no backup at all). Cold and Hot backups Hot backups have already been defined in the previous section. They require the presence of a WAL archive and they are the norm in any modern database management system. Cold backups , also known as offline backups, are instead physical base backups taken when the PostgreSQL instance (standby or primary) is shut down. They are consistent per definition and they represent a snapshot of the database at the time it was shut down. As a result, PostgreSQL instances can be restarted from a cold backup without the need of a WAL archive, even though they can take advantage of it, if available (with all the benefits on the recovery side highlighted in the previous section). In those situations with a higher RPO (for example, 1 hour or 24 hours), and shorter retention periods, cold backups represent a viable option to be considered for your disaster recovery plans. Object stores or volume snapshots: which one to use? In CloudNativePG, object store based backups: always require the WAL archive support hot backup only don't support incremental copy don't support differential copy VolumeSnapshots instead: don't require the WAL archive, although in production it is always recommended support incremental copy, depending on the underlying storage classes support differential copy, depending on the underlying storage classes also support cold backup Which one to use depends on your specific requirements and environment, including: availability of a viable object store solution in your Kubernetes cluster availability of a trusted storage class that supports volume snapshots size of the database: with object stores, the larger your database, the longer backup and, most importantly, recovery procedures take (the latter impacts RTO ); in presence of Very Large Databases (VLDB), the general advice is to rely on Volume Snapshots as, thanks to copy-on-write, they provide faster recovery data mobility and possibility to store or relay backup files on a secondary location in a different region, or any subsequent one other factors, mostly based on the confidence and familiarity with the underlying storage solutions The summary table below highlights some of the main differences between the two available methods for storing physical base backups. Object store Volume Snapshots WAL archiving Required Recommended (1) Cold backup \u2717 \u2713 Hot backup \u2713 \u2713 Incremental copy \u2717 \u2713 (2) Differential copy \u2717 \u2713 (2) Backup from a standby \u2713 \u2713 Snapshot recovery \u2717 (3) \u2713 Point In Time Recovery (PITR) \u2713 Requires WAL archive Underlying technology Barman Cloud Kubernetes API See the explanation below for the notes in the above table: WAL archive must be on an object store at the moment If supported by the underlying storage classes of the PostgreSQL volumes Snapshot recovery can be emulated using the bootstrap.recovery.recoveryTarget.targetImmediate option Scheduled backups Scheduled backups are the recommended way to configure your backup strategy in CloudNativePG. They are managed by the ScheduledBackup resource. Info Please refer to ScheduledBackupSpec in the API reference for a full list of options. The schedule field allows you to define a six-term cron schedule specification, which includes seconds, as expressed in the Go cron package format . Warning Beware that this format accepts also the seconds field, and it is different from the crontab format in Unix/Linux systems. This is an example of a scheduled backup: apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: backup-example spec: schedule: \"0 0 0 * * *\" backupOwnerReference: self cluster: name: pg-backup The above example will schedule a backup every day at midnight because the schedule specifies zero for the second, minute, and hour, while specifying wildcard, meaning all, for day of the month, month, and day of the week. In Kubernetes CronJobs, the equivalent expression is 0 0 * * * because seconds are not included. Hint Backup frequency might impact your recovery time objective ( RTO ) after a disaster which requires a full or Point-In-Time recovery operation. Our advice is that you regularly test your backups by recovering them, and then measuring the time it takes to recover from scratch so that you can refine your RTO predictability. Recovery time is influenced by the size of the base backup and the amount of WAL files that need to be fetched from the archive and replayed during recovery (remember that WAL archiving is what enables continuous backup in PostgreSQL!). Based on our experience, a weekly base backup is more than enough for most cases - while it is extremely rare to schedule backups more frequently than once a day. You can choose whether to schedule a backup on a defined object store or a volume snapshot via the .spec.method attribute, by default set to barmanObjectStore . If you have properly defined volume snapshots in the backup stanza of the cluster, you can set method: volumeSnapshot to start scheduling base backups on volume snapshots. ScheduledBackups can be suspended, if needed, by setting .spec.suspend: true . This will stop any new backup from being scheduled until the option is removed or set back to false . In case you want to issue a backup as soon as the ScheduledBackup resource is created you can set .spec.immediate: true . Note .spec.backupOwnerReference indicates which ownerReference should be put inside the created backup resources. none: no owner reference for created backup objects (same behavior as before the field was introduced) self: sets the Scheduled backup object as owner of the backup cluster: set the cluster as owner of the backup On-demand backups Info Please refer to BackupSpec in the API reference for a full list of options. To request a new backup, you need to create a new Backup resource like the following one: apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: name: backup-example spec: method: barmanObjectStore cluster: name: pg-backup In this case, the operator will start to orchestrate the cluster to take the required backup on an object store, using barman-cloud-backup . You can check the backup status using the plain kubectl describe backup command: Name: backup-example Namespace: default Labels: Annotations: API Version: postgresql.cnpg.io/v1 Kind: Backup Metadata: Creation Timestamp: 2020-10-26T13:57:40Z Self Link: /apis/postgresql.cnpg.io/v1/namespaces/default/backups/backup-example UID: ad5f855c-2ffd-454a-a157-900d5f1f6584 Spec: Cluster: Name: pg-backup Status: Phase: running Started At: 2020-10-26T13:57:40Z Events: When the backup has been completed, the phase will be completed like in the following example: Name: backup-example Namespace: default Labels: Annotations: API Version: postgresql.cnpg.io/v1 Kind: Backup Metadata: Creation Timestamp: 2020-10-26T13:57:40Z Self Link: /apis/postgresql.cnpg.io/v1/namespaces/default/backups/backup-example UID: ad5f855c-2ffd-454a-a157-900d5f1f6584 Spec: Cluster: Name: pg-backup Status: Backup Id: 20201026T135740 Destination Path: s3://backups/ Endpoint URL: http://minio:9000 Phase: completed s3Credentials: Access Key Id: Key: ACCESS_KEY_ID Name: minio Secret Access Key: Key: ACCESS_SECRET_KEY Name: minio Server Name: pg-backup Started At: 2020-10-26T13:57:40Z Stopped At: 2020-10-26T13:57:44Z Events: Important This feature will not backup the secrets for the superuser and the application user. The secrets are supposed to be backed up as part of the standard backup procedures for the Kubernetes cluster. Backup from a standby Taking a base backup requires to scrape the whole data content of the PostgreSQL instance on disk, possibly resulting in I/O contention with the actual workload of the database. For this reason, CloudNativePG allows you to take advantage of a feature which is directly available in PostgreSQL: backup from a standby . By default, backups will run on the most aligned replica of a Cluster . If no replicas are available, backups will run on the primary instance. Info Although the standby might not always be up to date with the primary, in the time continuum from the first available backup to the last archived WAL this is normally irrelevant. The base backup indeed represents the starting point from which to begin a recovery operation, including PITR. Similarly to what happens with pg_basebackup , when backing up from an online standby we do not force a switch of the WAL on the primary. This might produce unexpected results in the short term (before archive_timeout kicks in) in deployments with low write activity. If you prefer to always run backups on the primary, you can set the backup target to primary as outlined in the example below: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: [...] spec: backup: target: \"primary\" Warning Beware of setting the target to primary when performing a cold backup with volume snapshots, as this will shut down the primary for the time needed to take the snapshot, impacting write operations. This also applies to taking a cold backup in a single-instance cluster, even if you did not explicitly set the primary as the target. When the backup target is set to prefer-standby , such policy will ensure backups are run on the most up-to-date available secondary instance, or if no other instance is available, on the primary instance. By default, when not otherwise specified, target is automatically set to take backups from a standby. The backup target specified in the Cluster can be overridden in the Backup and ScheduledBackup types, like in the following example: apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: [...] spec: cluster: name: [...] target: \"primary\" In the previous example, CloudNativePG will invariably choose the primary instance even if the Cluster is set to prefer replicas.","title":"Backup"},{"location":"backup/#backup","text":"PostgreSQL natively provides first class backup and recovery capabilities based on file system level (physical) copy. These have been successfully used for more than 15 years in mission critical production databases, helping organizations all over the world achieve their disaster recovery goals with Postgres. Note There's another way to backup databases in PostgreSQL, through the pg_dump utility - which relies on logical backups instead of physical ones. However, logical backups are not suitable for business continuity use cases and as such are not covered by CloudNativePG (yet, at least). If you want to use the pg_dump utility, let yourself be inspired by the \"Troubleshooting / Emergency backup\" section . In CloudNativePG, the backup infrastructure for each PostgreSQL cluster is made up of the following resources: WAL archive : a location containing the WAL files (transactional logs) that are continuously written by Postgres and archived for data durability Physical base backups : a copy of all the files that PostgreSQL uses to store the data in the database (primarily the PGDATA and any tablespace) The WAL archive can only be stored on object stores at the moment. On the other hand, CloudNativePG supports two ways to store physical base backups: on object stores , as tarballs - optionally compressed on Kubernetes Volume Snapshots , if supported by the underlying storage class Important Before choosing your backup strategy with CloudNativePG, it is important that you take some time to familiarize with some basic concepts, like WAL archive, hot and cold backups. Important Please refer to the official Kubernetes documentation for a list of all the supported Container Storage Interface (CSI) drivers that provide snapshotting capabilities. Info Starting with version 1.25, CloudNativePG includes experimental support for backup and recovery using plugins, such as the Barman Cloud plugin .","title":"Backup"},{"location":"backup/#wal-archive","text":"The WAL archive in PostgreSQL is at the heart of continuous backup , and it is fundamental for the following reasons: Hot backups : the possibility to take physical base backups from any instance in the Postgres cluster (either primary or standby) without shutting down the server; they are also known as online backups Point in Time recovery (PITR): to possibility to recover at any point in time from the first available base backup in your system Warning WAL archive alone is useless. Without a physical base backup, you cannot restore a PostgreSQL cluster. In general, the presence of a WAL archive enhances the resilience of a PostgreSQL cluster, allowing each instance to fetch any required WAL file from the archive if needed (normally the WAL archive has higher retention periods than any Postgres instance that normally recycles those files). This use case can also be extended to replica clusters , as they can simply rely on the WAL archive to synchronize across long distances, extending disaster recovery goals across different regions. When you configure a WAL archive , CloudNativePG provides out-of-the-box an RPO <= 5 minutes for disaster recovery, even across regions. Important Our recommendation is to always setup the WAL archive in production. There are known use cases - normally involving staging and development environments - where none of the above benefits are needed and the WAL archive is not necessary. RPO in this case can be any value, such as 24 hours (daily backups) or infinite (no backup at all).","title":"WAL archive"},{"location":"backup/#cold-and-hot-backups","text":"Hot backups have already been defined in the previous section. They require the presence of a WAL archive and they are the norm in any modern database management system. Cold backups , also known as offline backups, are instead physical base backups taken when the PostgreSQL instance (standby or primary) is shut down. They are consistent per definition and they represent a snapshot of the database at the time it was shut down. As a result, PostgreSQL instances can be restarted from a cold backup without the need of a WAL archive, even though they can take advantage of it, if available (with all the benefits on the recovery side highlighted in the previous section). In those situations with a higher RPO (for example, 1 hour or 24 hours), and shorter retention periods, cold backups represent a viable option to be considered for your disaster recovery plans.","title":"Cold and Hot backups"},{"location":"backup/#object-stores-or-volume-snapshots-which-one-to-use","text":"In CloudNativePG, object store based backups: always require the WAL archive support hot backup only don't support incremental copy don't support differential copy VolumeSnapshots instead: don't require the WAL archive, although in production it is always recommended support incremental copy, depending on the underlying storage classes support differential copy, depending on the underlying storage classes also support cold backup Which one to use depends on your specific requirements and environment, including: availability of a viable object store solution in your Kubernetes cluster availability of a trusted storage class that supports volume snapshots size of the database: with object stores, the larger your database, the longer backup and, most importantly, recovery procedures take (the latter impacts RTO ); in presence of Very Large Databases (VLDB), the general advice is to rely on Volume Snapshots as, thanks to copy-on-write, they provide faster recovery data mobility and possibility to store or relay backup files on a secondary location in a different region, or any subsequent one other factors, mostly based on the confidence and familiarity with the underlying storage solutions The summary table below highlights some of the main differences between the two available methods for storing physical base backups. Object store Volume Snapshots WAL archiving Required Recommended (1) Cold backup \u2717 \u2713 Hot backup \u2713 \u2713 Incremental copy \u2717 \u2713 (2) Differential copy \u2717 \u2713 (2) Backup from a standby \u2713 \u2713 Snapshot recovery \u2717 (3) \u2713 Point In Time Recovery (PITR) \u2713 Requires WAL archive Underlying technology Barman Cloud Kubernetes API See the explanation below for the notes in the above table: WAL archive must be on an object store at the moment If supported by the underlying storage classes of the PostgreSQL volumes Snapshot recovery can be emulated using the bootstrap.recovery.recoveryTarget.targetImmediate option","title":"Object stores or volume snapshots: which one to use?"},{"location":"backup/#scheduled-backups","text":"Scheduled backups are the recommended way to configure your backup strategy in CloudNativePG. They are managed by the ScheduledBackup resource. Info Please refer to ScheduledBackupSpec in the API reference for a full list of options. The schedule field allows you to define a six-term cron schedule specification, which includes seconds, as expressed in the Go cron package format . Warning Beware that this format accepts also the seconds field, and it is different from the crontab format in Unix/Linux systems. This is an example of a scheduled backup: apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: backup-example spec: schedule: \"0 0 0 * * *\" backupOwnerReference: self cluster: name: pg-backup The above example will schedule a backup every day at midnight because the schedule specifies zero for the second, minute, and hour, while specifying wildcard, meaning all, for day of the month, month, and day of the week. In Kubernetes CronJobs, the equivalent expression is 0 0 * * * because seconds are not included. Hint Backup frequency might impact your recovery time objective ( RTO ) after a disaster which requires a full or Point-In-Time recovery operation. Our advice is that you regularly test your backups by recovering them, and then measuring the time it takes to recover from scratch so that you can refine your RTO predictability. Recovery time is influenced by the size of the base backup and the amount of WAL files that need to be fetched from the archive and replayed during recovery (remember that WAL archiving is what enables continuous backup in PostgreSQL!). Based on our experience, a weekly base backup is more than enough for most cases - while it is extremely rare to schedule backups more frequently than once a day. You can choose whether to schedule a backup on a defined object store or a volume snapshot via the .spec.method attribute, by default set to barmanObjectStore . If you have properly defined volume snapshots in the backup stanza of the cluster, you can set method: volumeSnapshot to start scheduling base backups on volume snapshots. ScheduledBackups can be suspended, if needed, by setting .spec.suspend: true . This will stop any new backup from being scheduled until the option is removed or set back to false . In case you want to issue a backup as soon as the ScheduledBackup resource is created you can set .spec.immediate: true . Note .spec.backupOwnerReference indicates which ownerReference should be put inside the created backup resources. none: no owner reference for created backup objects (same behavior as before the field was introduced) self: sets the Scheduled backup object as owner of the backup cluster: set the cluster as owner of the backup","title":"Scheduled backups"},{"location":"backup/#on-demand-backups","text":"Info Please refer to BackupSpec in the API reference for a full list of options. To request a new backup, you need to create a new Backup resource like the following one: apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: name: backup-example spec: method: barmanObjectStore cluster: name: pg-backup In this case, the operator will start to orchestrate the cluster to take the required backup on an object store, using barman-cloud-backup . You can check the backup status using the plain kubectl describe backup command: Name: backup-example Namespace: default Labels: Annotations: API Version: postgresql.cnpg.io/v1 Kind: Backup Metadata: Creation Timestamp: 2020-10-26T13:57:40Z Self Link: /apis/postgresql.cnpg.io/v1/namespaces/default/backups/backup-example UID: ad5f855c-2ffd-454a-a157-900d5f1f6584 Spec: Cluster: Name: pg-backup Status: Phase: running Started At: 2020-10-26T13:57:40Z Events: When the backup has been completed, the phase will be completed like in the following example: Name: backup-example Namespace: default Labels: Annotations: API Version: postgresql.cnpg.io/v1 Kind: Backup Metadata: Creation Timestamp: 2020-10-26T13:57:40Z Self Link: /apis/postgresql.cnpg.io/v1/namespaces/default/backups/backup-example UID: ad5f855c-2ffd-454a-a157-900d5f1f6584 Spec: Cluster: Name: pg-backup Status: Backup Id: 20201026T135740 Destination Path: s3://backups/ Endpoint URL: http://minio:9000 Phase: completed s3Credentials: Access Key Id: Key: ACCESS_KEY_ID Name: minio Secret Access Key: Key: ACCESS_SECRET_KEY Name: minio Server Name: pg-backup Started At: 2020-10-26T13:57:40Z Stopped At: 2020-10-26T13:57:44Z Events: Important This feature will not backup the secrets for the superuser and the application user. The secrets are supposed to be backed up as part of the standard backup procedures for the Kubernetes cluster.","title":"On-demand backups"},{"location":"backup/#backup-from-a-standby","text":"Taking a base backup requires to scrape the whole data content of the PostgreSQL instance on disk, possibly resulting in I/O contention with the actual workload of the database. For this reason, CloudNativePG allows you to take advantage of a feature which is directly available in PostgreSQL: backup from a standby . By default, backups will run on the most aligned replica of a Cluster . If no replicas are available, backups will run on the primary instance. Info Although the standby might not always be up to date with the primary, in the time continuum from the first available backup to the last archived WAL this is normally irrelevant. The base backup indeed represents the starting point from which to begin a recovery operation, including PITR. Similarly to what happens with pg_basebackup , when backing up from an online standby we do not force a switch of the WAL on the primary. This might produce unexpected results in the short term (before archive_timeout kicks in) in deployments with low write activity. If you prefer to always run backups on the primary, you can set the backup target to primary as outlined in the example below: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: [...] spec: backup: target: \"primary\" Warning Beware of setting the target to primary when performing a cold backup with volume snapshots, as this will shut down the primary for the time needed to take the snapshot, impacting write operations. This also applies to taking a cold backup in a single-instance cluster, even if you did not explicitly set the primary as the target. When the backup target is set to prefer-standby , such policy will ensure backups are run on the most up-to-date available secondary instance, or if no other instance is available, on the primary instance. By default, when not otherwise specified, target is automatically set to take backups from a standby. The backup target specified in the Cluster can be overridden in the Backup and ScheduledBackup types, like in the following example: apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: [...] spec: cluster: name: [...] target: \"primary\" In the previous example, CloudNativePG will invariably choose the primary instance even if the Cluster is set to prefer replicas.","title":"Backup from a standby"},{"location":"backup_barmanobjectstore/","text":"Backup on object stores CloudNativePG natively supports online/hot backup of PostgreSQL clusters through continuous physical backup and WAL archiving on an object store. This means that the database is always up (no downtime required) and that Point In Time Recovery is available. The operator can orchestrate a continuous backup infrastructure that is based on the Barman Cloud tool. Instead of using the classical architecture with a Barman server, which backs up many PostgreSQL instances, the operator relies on the barman-cloud-wal-archive , barman-cloud-check-wal-archive , barman-cloud-backup , barman-cloud-backup-list , and barman-cloud-backup-delete tools. As a result, base backups will be tarballs . Both base backups and WAL files can be compressed and encrypted. For this, it is required to use an image with barman-cli-cloud included. You can use the image ghcr.io/cloudnative-pg/postgresql for this scope, as it is composed of a community PostgreSQL image and the latest barman-cli-cloud package. Important Always ensure that you are running the latest version of the operands in your system to take advantage of the improvements introduced in Barman cloud (as well as improve the security aspects of your cluster). A backup is performed from a primary or a designated primary instance in a Cluster (please refer to replica clusters for more information about designated primary instances), or alternatively on a standby . Common object stores If you are looking for a specific object store such as AWS S3 , Microsoft Azure Blob Storage , Google Cloud Storage , or MinIO Gateway , or a compatible provider, please refer to Appendix A - Common object stores . Retention policies Important Retention policies are not currently available on volume snapshots. CloudNativePG can manage the automated deletion of backup files from the backup object store, using retention policies based on the recovery window. Internally, the retention policy feature uses barman-cloud-backup-delete with --retention-policy \u201cRECOVERY WINDOW OF {{ retention policy value }} {{ retention policy unit }}\u201d . For example, you can define your backups with a retention policy of 30 days as follows: apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: destinationPath: \"\" s3Credentials: accessKeyId: name: aws-creds key: ACCESS_KEY_ID secretAccessKey: name: aws-creds key: ACCESS_SECRET_KEY retentionPolicy: \"30d\" There's more ... The recovery window retention policy is focused on the concept of Point of Recoverability ( PoR ), a moving point in time determined by current time - recovery window . The first valid backup is the first available backup before PoR (in reverse chronological order). CloudNativePG must ensure that we can recover the cluster at any point in time between PoR and the latest successfully archived WAL file, starting from the first valid backup. Base backups that are older than the first valid backup will be marked as obsolete and permanently removed after the next backup is completed. Compression algorithms CloudNativePG by default archives backups and WAL files in an uncompressed fashion. However, it also supports the following compression algorithms via barman-cloud-backup (for backups) and barman-cloud-wal-archive (for WAL files): bzip2 gzip lz4 snappy xz zstd The compression settings for backups and WALs are independent. See the DataBackupConfiguration and WALBackupConfiguration sections in the barman-cloud API reference. It is important to note that archival time, restore time, and size change between the algorithms, so the compression algorithm should be chosen according to your use case. The Barman team has performed an evaluation of the performance of the supported algorithms for Barman Cloud. The following table summarizes a scenario where a backup is taken on a local MinIO deployment. The Barman GitHub project includes a deeper analysis . Compression Backup Time (ms) Restore Time (ms) Uncompressed size (MB) Compressed size (MB) Approx ratio None 10927 7553 395 395 1:1 bzip2 25404 13886 395 67 5.9:1 gzip 116281 3077 395 91 4.3:1 snappy 8134 8341 395 166 2.4:1 Tagging of backup objects Barman 2.18 introduces support for tagging backup resources when saving them in object stores via barman-cloud-backup and barman-cloud-wal-archive . As a result, if your PostgreSQL container image includes Barman with version 2.18 or higher, CloudNativePG enables you to specify tags as key-value pairs for backup objects, namely base backups, WAL files and history files. You can use two properties in the .spec.backup.barmanObjectStore definition: tags : key-value pair tags to be added to backup objects and archived WAL file in the backup object store historyTags : key-value pair tags to be added to archived history files in the backup object store The excerpt of a YAML manifest below provides an example of usage of this feature: apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: [...] tags: backupRetentionPolicy: \"expire\" historyTags: backupRetentionPolicy: \"keep\" Extra options for the backup and WAL commands You can append additional options to the barman-cloud-backup and barman-cloud-wal-archive commands by using the additionalCommandArgs property in the .spec.backup.barmanObjectStore.data and .spec.backup.barmanObjectStore.wal sections respectively. This properties are lists of strings that will be appended to the barman-cloud-backup and barman-cloud-wal-archive commands. For example, you can use the --read-timeout=60 to customize the connection reading timeout. For additional options supported by barman-cloud-backup and barman-cloud-wal-archive commands you can refer to the official barman documentation here . If an option provided in additionalCommandArgs is already present in the declared options in its section ( .spec.backup.barmanObjectStore.data or .spec.backup.barmanObjectStore.wal ), the extra option will be ignored. The following is an example of how to use this property: For backups: apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: [...] data: additionalCommandArgs: - \"--min-chunk-size=5MB\" - \"--read-timeout=60\" For WAL files: apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: [...] wal: additionalCommandArgs: - \"--max-concurrency=1\" - \"--read-timeout=60\"","title":"Backup on object stores"},{"location":"backup_barmanobjectstore/#backup-on-object-stores","text":"CloudNativePG natively supports online/hot backup of PostgreSQL clusters through continuous physical backup and WAL archiving on an object store. This means that the database is always up (no downtime required) and that Point In Time Recovery is available. The operator can orchestrate a continuous backup infrastructure that is based on the Barman Cloud tool. Instead of using the classical architecture with a Barman server, which backs up many PostgreSQL instances, the operator relies on the barman-cloud-wal-archive , barman-cloud-check-wal-archive , barman-cloud-backup , barman-cloud-backup-list , and barman-cloud-backup-delete tools. As a result, base backups will be tarballs . Both base backups and WAL files can be compressed and encrypted. For this, it is required to use an image with barman-cli-cloud included. You can use the image ghcr.io/cloudnative-pg/postgresql for this scope, as it is composed of a community PostgreSQL image and the latest barman-cli-cloud package. Important Always ensure that you are running the latest version of the operands in your system to take advantage of the improvements introduced in Barman cloud (as well as improve the security aspects of your cluster). A backup is performed from a primary or a designated primary instance in a Cluster (please refer to replica clusters for more information about designated primary instances), or alternatively on a standby .","title":"Backup on object stores"},{"location":"backup_barmanobjectstore/#common-object-stores","text":"If you are looking for a specific object store such as AWS S3 , Microsoft Azure Blob Storage , Google Cloud Storage , or MinIO Gateway , or a compatible provider, please refer to Appendix A - Common object stores .","title":"Common object stores"},{"location":"backup_barmanobjectstore/#retention-policies","text":"Important Retention policies are not currently available on volume snapshots. CloudNativePG can manage the automated deletion of backup files from the backup object store, using retention policies based on the recovery window. Internally, the retention policy feature uses barman-cloud-backup-delete with --retention-policy \u201cRECOVERY WINDOW OF {{ retention policy value }} {{ retention policy unit }}\u201d . For example, you can define your backups with a retention policy of 30 days as follows: apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: destinationPath: \"\" s3Credentials: accessKeyId: name: aws-creds key: ACCESS_KEY_ID secretAccessKey: name: aws-creds key: ACCESS_SECRET_KEY retentionPolicy: \"30d\" There's more ... The recovery window retention policy is focused on the concept of Point of Recoverability ( PoR ), a moving point in time determined by current time - recovery window . The first valid backup is the first available backup before PoR (in reverse chronological order). CloudNativePG must ensure that we can recover the cluster at any point in time between PoR and the latest successfully archived WAL file, starting from the first valid backup. Base backups that are older than the first valid backup will be marked as obsolete and permanently removed after the next backup is completed.","title":"Retention policies"},{"location":"backup_barmanobjectstore/#compression-algorithms","text":"CloudNativePG by default archives backups and WAL files in an uncompressed fashion. However, it also supports the following compression algorithms via barman-cloud-backup (for backups) and barman-cloud-wal-archive (for WAL files): bzip2 gzip lz4 snappy xz zstd The compression settings for backups and WALs are independent. See the DataBackupConfiguration and WALBackupConfiguration sections in the barman-cloud API reference. It is important to note that archival time, restore time, and size change between the algorithms, so the compression algorithm should be chosen according to your use case. The Barman team has performed an evaluation of the performance of the supported algorithms for Barman Cloud. The following table summarizes a scenario where a backup is taken on a local MinIO deployment. The Barman GitHub project includes a deeper analysis . Compression Backup Time (ms) Restore Time (ms) Uncompressed size (MB) Compressed size (MB) Approx ratio None 10927 7553 395 395 1:1 bzip2 25404 13886 395 67 5.9:1 gzip 116281 3077 395 91 4.3:1 snappy 8134 8341 395 166 2.4:1","title":"Compression algorithms"},{"location":"backup_barmanobjectstore/#tagging-of-backup-objects","text":"Barman 2.18 introduces support for tagging backup resources when saving them in object stores via barman-cloud-backup and barman-cloud-wal-archive . As a result, if your PostgreSQL container image includes Barman with version 2.18 or higher, CloudNativePG enables you to specify tags as key-value pairs for backup objects, namely base backups, WAL files and history files. You can use two properties in the .spec.backup.barmanObjectStore definition: tags : key-value pair tags to be added to backup objects and archived WAL file in the backup object store historyTags : key-value pair tags to be added to archived history files in the backup object store The excerpt of a YAML manifest below provides an example of usage of this feature: apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: [...] tags: backupRetentionPolicy: \"expire\" historyTags: backupRetentionPolicy: \"keep\"","title":"Tagging of backup objects"},{"location":"backup_barmanobjectstore/#extra-options-for-the-backup-and-wal-commands","text":"You can append additional options to the barman-cloud-backup and barman-cloud-wal-archive commands by using the additionalCommandArgs property in the .spec.backup.barmanObjectStore.data and .spec.backup.barmanObjectStore.wal sections respectively. This properties are lists of strings that will be appended to the barman-cloud-backup and barman-cloud-wal-archive commands. For example, you can use the --read-timeout=60 to customize the connection reading timeout. For additional options supported by barman-cloud-backup and barman-cloud-wal-archive commands you can refer to the official barman documentation here . If an option provided in additionalCommandArgs is already present in the declared options in its section ( .spec.backup.barmanObjectStore.data or .spec.backup.barmanObjectStore.wal ), the extra option will be ignored. The following is an example of how to use this property: For backups: apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: [...] data: additionalCommandArgs: - \"--min-chunk-size=5MB\" - \"--read-timeout=60\" For WAL files: apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: [...] wal: additionalCommandArgs: - \"--max-concurrency=1\" - \"--read-timeout=60\"","title":"Extra options for the backup and WAL commands"},{"location":"backup_recovery/","text":"Backup and Recovery Backup and recovery are in two separate sections.","title":"Backup and Recovery"},{"location":"backup_recovery/#backup-and-recovery","text":"Backup and recovery are in two separate sections.","title":"Backup and Recovery"},{"location":"backup_volumesnapshot/","text":"Backup on volume snapshots Warning As noted in the backup document , a cold snapshot explicitly set to target the primary will result in the primary being fenced for the duration of the backup, rendering the cluster read-only during that For safety, in a cluster already containing fenced instances, a cold snapshot is rejected. CloudNativePG is one of the first known cases of database operators that directly leverages the Kubernetes native Volume Snapshot API for both backup and recovery operations, in an entirely declarative way. About standard Volume Snapshots Volume snapshotting was first introduced in Kubernetes 1.12 (2018) as alpha , promoted to beta in 1.17 (2019) , and moved to GA in 1.20 (2020) . It\u2019s now stable, widely available, and standard, providing 3 custom resource definitions: VolumeSnapshot , VolumeSnapshotContent and VolumeSnapshotClass . This Kubernetes feature defines a generic interface for: the creation of a new volume snapshot, starting from a PVC the deletion of an existing snapshot the creation of a new volume from a snapshot Kubernetes delegates the actual implementation to the underlying CSI drivers (not all of them support volume snapshots). Normally, storage classes that provide volume snapshotting support incremental and differential block level backup in a transparent way for the application , which can delegate the complexity and the independent management down the stack, including cross-cluster availability of the snapshots. Requirements For Volume Snapshots to work with a CloudNativePG cluster, you need to ensure that each storage class used to dynamically provision the PostgreSQL volumes (namely, storage and walStorage sections) support volume snapshots. Given that instructions vary from storage class to storage class, please refer to the documentation of the specific storage class and related CSI drivers you have deployed in your Kubernetes system. Normally, it is the VolumeSnapshotClass that is responsible to ensure that snapshots can be taken from persistent volumes of a given storage class, and managed as VolumeSnapshot and VolumeSnapshotContent resources. Important It is your responsibility to verify with the third party vendor that volume snapshots are supported. CloudNativePG only interacts with the Kubernetes API on this matter and we cannot support issues at the storage level for each specific CSI driver. How to configure Volume Snapshot backups CloudNativePG allows you to configure a given Postgres cluster for Volume Snapshot backups through the backup.volumeSnapshot stanza. Info Please refer to VolumeSnapshotConfiguration in the API reference for a full list of options. A generic example with volume snapshots (assuming that PGDATA and WALs share the same storage class) is the following: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: snapshot-cluster spec: instances: 3 storage: storageClass: @STORAGE_CLASS@ size: 10Gi walStorage: storageClass: @STORAGE_CLASS@ size: 10Gi backup: # Volume snapshot backups volumeSnapshot: className: @VOLUME_SNAPSHOT_CLASS_NAME@ # WAL archive barmanObjectStore: # ... As you can see, the backup section contains both the volumeSnapshot stanza (controlling physical base backups on volume snapshots) and the barmanObjectStore one (controlling the WAL archive ). Info Once you have defined the barmanObjectStore , you can decide to use both volume snapshot and object store backup strategies simultaneously to take physical backups. The volumeSnapshot.className option allows you to reference the default VolumeSnapshotClass object used for all the storage volumes you have defined in your PostgreSQL cluster. Info In case you are using a different storage class for PGDATA and WAL files, you can specify a separate VolumeSnapshotClass for that volume through the walClassName option (which defaults to the same value as className ). Once a cluster is defined for volume snapshot backups, you need to define a ScheduledBackup resource that requests such backups on a periodic basis. Hot and cold backups By default, CloudNativePG requests an online/hot backup on volume snapshots, using the PostgreSQL defaults of the low-level API for base backups : it doesn't request an immediate checkpoint when starting the backup procedure it waits for the WAL archiver to archive the last segment of the backup when terminating the backup procedure Important The default values are suitable for most production environments. Hot backups are consistent and can be used to perform snapshot recovery, as we ensure WAL retention from the start of the backup through a temporary replication slot. However, our recommendation is to rely on cold backups for that purpose. You can explicitly change the default behavior through the following options in the .spec.backup.volumeSnapshot stanza of the Cluster resource: online : accepting true (default) or false as a value onlineConfiguration.immediateCheckpoint : whether you want to request an immediate checkpoint before you start the backup procedure or not; technically, it corresponds to the fast argument you pass to the pg_backup_start / pg_start_backup() function in PostgreSQL, accepting true (default) or false onlineConfiguration.waitForArchive : whether you want to wait for the archiver to process the last segment of the backup or not; technically, it corresponds to the wait_for_archive argument you pass to the pg_backup_stop / pg_stop_backup() function in PostgreSQL, accepting true (default) or false If you want to change the default behavior of your Postgres cluster to take cold backups by default, all you need to do is add the online: false option to your manifest, as follows: # ... backup: volumeSnapshot: online: false # ... If you are instead requesting an immediate checkpoint as the default behavior, you can add this section: # ... backup: volumeSnapshot: online: true onlineConfiguration: immediateCheckpoint: true # ... Overriding the default behavior You can change the default behavior defined in the cluster resource by setting different values for online and, if needed, onlineConfiguration in the Backup or ScheduledBackup objects. For example, in case you want to issue an on-demand cold backup, you can create a Backup object with .spec.online: false : apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: name: snapshot-cluster-cold-backup-example spec: cluster: name: snapshot-cluster method: volumeSnapshot online: false Similarly, for the ScheduledBackup: apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: snapshot-cluster-cold-backup-example spec: schedule: \"0 0 0 * * *\" backupOwnerReference: self cluster: name: snapshot-cluster method: volumeSnapshot online: false Persistence of volume snapshot objects By default, VolumeSnapshot objects created by CloudNativePG are retained after deleting the Backup object that originated them, or the Cluster they refer to. Such behavior is controlled by the .spec.backup.volumeSnapshot.snapshotOwnerReference option which accepts the following values: none : no ownership is set, meaning that VolumeSnapshot objects persist after the Backup and/or the Cluster resources are removed backup : the VolumeSnapshot object is owned by the Backup resource that originated it, and when the backup object is removed, the volume snapshot is also removed cluster : the VolumeSnapshot object is owned by the Cluster resource that is backed up, and when the Postgres cluster is removed, the volume snapshot is also removed In case a VolumeSnapshot is deleted, the deletionPolicy specified in the VolumeSnapshotContent is evaluated: if set to Retain , the VolumeSnapshotContent object is kept if set to Delete , the VolumeSnapshotContent object is removed as well Warning VolumeSnapshotContent objects do not keep all the information regarding the backup and the cluster they refer to (like the annotations and labels that are contained in the VolumeSnapshot object). Although possible, restoring from just this kind of object might not be straightforward. For this reason, our recommendation is to always backup the VolumeSnapshot definitions, even using a Kubernetes level data protection solution. The value in VolumeSnapshotContent is determined by the deletionPolicy set in the corresponding VolumeSnapshotClass definition, which is referenced in the .spec.backup.volumeSnapshot.className option. Please refer to the Kubernetes documentation on Volume Snapshot Classes for details on this standard behavior. Backup Volume Snapshot Deadlines CloudNativePG supports backups using the volume snapshot method. In some environments, volume snapshots may encounter temporary issues that can be retried. The backup.cnpg.io/volumeSnapshotDeadline annotation defines how long CloudNativePG should continue retrying recoverable errors before marking the backup as failed. You can add the backup.cnpg.io/volumeSnapshotDeadline annotation to both Backup and ScheduledBackup resources. For ScheduledBackup resources, this annotation is automatically inherited by any Backup resources created from the schedule. If not specified, the default retry deadline is 10 minutes . Error Handling When a retryable error occurs during a volume snapshot operation: CloudNativePG records the time of the first error. The system retries the operation every 10 seconds . If the error persists beyond the specified deadline (or the default 10 minutes), the backup is marked as failed . Retryable Errors CloudNativePG treats the following types of errors as retryable: Server timeout errors (HTTP 408, 429, 500, 502, 503, 504) Conflicts (optimistic locking errors) Internal errors Context deadline exceeded errors Timeout errors from the CSI snapshot controller Examples You can add the annotation to a ScheduledBackup resource as follows: apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: daily-backup-schedule annotations: backup.cnpg.io/volumeSnapshotDeadline: \"20\" spec: schedule: \"0 0 * * *\" backupOwnerReference: self method: volumeSnapshot # other configuration... When you define a ScheduledBackup with the annotation, any Backup resources created from this schedule automatically inherit the specified timeout value. In the following example, all backups created from the schedule will have a 30-minute timeout for retrying recoverable snapshot errors. apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: weekly-backup annotations: backup.cnpg.io/volumeSnapshotDeadline: \"30\" spec: schedule: \"0 0 * * 0\" # Weekly backup on Sunday method: volumeSnapshot cluster: name: my-postgresql-cluster Alternatively, you can add the annotation directly to a Backup Resource: apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: name: my-backup annotations: backup.cnpg.io/volumeSnapshotDeadline: \"15\" spec: method: volumeSnapshot # other backup configuration... Example of Volume Snapshot Backup The following example shows how to configure volume snapshot base backups on an EKS cluster on AWS using the ebs-sc storage class and the csi-aws-vsc volume snapshot class. Important If you are interested in testing the example, please read \"Volume Snapshots\" for the Amazon Elastic Block Store (EBS) CSI driver for detailed instructions on the installation process for the storage class and the snapshot class. The following manifest creates a Cluster that is ready to be used for volume snapshots and that stores the WAL archive in a S3 bucket via IAM role for the Service Account (IRSA, see AWS S3 ): apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: hendrix spec: instances: 3 storage: storageClass: ebs-sc size: 10Gi walStorage: storageClass: ebs-sc size: 10Gi backup: volumeSnapshot: className: csi-aws-vsc barmanObjectStore: destinationPath: s3://@BUCKET_NAME@/ s3Credentials: inheritFromIAMRole: true wal: compression: gzip maxParallel: 2 serviceAccountTemplate: metadata: annotations: eks.amazonaws.com/role-arn: \"@ARN@\" --- apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: hendrix-vs-backup spec: cluster: name: hendrix method: volumeSnapshot schedule: '0 0 0 * * *' backupOwnerReference: cluster immediate: true The last resource defines daily volume snapshot backups at midnight, requesting one immediately after the cluster is created.","title":"Backup on volume snapshots"},{"location":"backup_volumesnapshot/#backup-on-volume-snapshots","text":"Warning As noted in the backup document , a cold snapshot explicitly set to target the primary will result in the primary being fenced for the duration of the backup, rendering the cluster read-only during that For safety, in a cluster already containing fenced instances, a cold snapshot is rejected. CloudNativePG is one of the first known cases of database operators that directly leverages the Kubernetes native Volume Snapshot API for both backup and recovery operations, in an entirely declarative way.","title":"Backup on volume snapshots"},{"location":"backup_volumesnapshot/#about-standard-volume-snapshots","text":"Volume snapshotting was first introduced in Kubernetes 1.12 (2018) as alpha , promoted to beta in 1.17 (2019) , and moved to GA in 1.20 (2020) . It\u2019s now stable, widely available, and standard, providing 3 custom resource definitions: VolumeSnapshot , VolumeSnapshotContent and VolumeSnapshotClass . This Kubernetes feature defines a generic interface for: the creation of a new volume snapshot, starting from a PVC the deletion of an existing snapshot the creation of a new volume from a snapshot Kubernetes delegates the actual implementation to the underlying CSI drivers (not all of them support volume snapshots). Normally, storage classes that provide volume snapshotting support incremental and differential block level backup in a transparent way for the application , which can delegate the complexity and the independent management down the stack, including cross-cluster availability of the snapshots.","title":"About standard Volume Snapshots"},{"location":"backup_volumesnapshot/#requirements","text":"For Volume Snapshots to work with a CloudNativePG cluster, you need to ensure that each storage class used to dynamically provision the PostgreSQL volumes (namely, storage and walStorage sections) support volume snapshots. Given that instructions vary from storage class to storage class, please refer to the documentation of the specific storage class and related CSI drivers you have deployed in your Kubernetes system. Normally, it is the VolumeSnapshotClass that is responsible to ensure that snapshots can be taken from persistent volumes of a given storage class, and managed as VolumeSnapshot and VolumeSnapshotContent resources. Important It is your responsibility to verify with the third party vendor that volume snapshots are supported. CloudNativePG only interacts with the Kubernetes API on this matter and we cannot support issues at the storage level for each specific CSI driver.","title":"Requirements"},{"location":"backup_volumesnapshot/#how-to-configure-volume-snapshot-backups","text":"CloudNativePG allows you to configure a given Postgres cluster for Volume Snapshot backups through the backup.volumeSnapshot stanza. Info Please refer to VolumeSnapshotConfiguration in the API reference for a full list of options. A generic example with volume snapshots (assuming that PGDATA and WALs share the same storage class) is the following: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: snapshot-cluster spec: instances: 3 storage: storageClass: @STORAGE_CLASS@ size: 10Gi walStorage: storageClass: @STORAGE_CLASS@ size: 10Gi backup: # Volume snapshot backups volumeSnapshot: className: @VOLUME_SNAPSHOT_CLASS_NAME@ # WAL archive barmanObjectStore: # ... As you can see, the backup section contains both the volumeSnapshot stanza (controlling physical base backups on volume snapshots) and the barmanObjectStore one (controlling the WAL archive ). Info Once you have defined the barmanObjectStore , you can decide to use both volume snapshot and object store backup strategies simultaneously to take physical backups. The volumeSnapshot.className option allows you to reference the default VolumeSnapshotClass object used for all the storage volumes you have defined in your PostgreSQL cluster. Info In case you are using a different storage class for PGDATA and WAL files, you can specify a separate VolumeSnapshotClass for that volume through the walClassName option (which defaults to the same value as className ). Once a cluster is defined for volume snapshot backups, you need to define a ScheduledBackup resource that requests such backups on a periodic basis.","title":"How to configure Volume Snapshot backups"},{"location":"backup_volumesnapshot/#hot-and-cold-backups","text":"By default, CloudNativePG requests an online/hot backup on volume snapshots, using the PostgreSQL defaults of the low-level API for base backups : it doesn't request an immediate checkpoint when starting the backup procedure it waits for the WAL archiver to archive the last segment of the backup when terminating the backup procedure Important The default values are suitable for most production environments. Hot backups are consistent and can be used to perform snapshot recovery, as we ensure WAL retention from the start of the backup through a temporary replication slot. However, our recommendation is to rely on cold backups for that purpose. You can explicitly change the default behavior through the following options in the .spec.backup.volumeSnapshot stanza of the Cluster resource: online : accepting true (default) or false as a value onlineConfiguration.immediateCheckpoint : whether you want to request an immediate checkpoint before you start the backup procedure or not; technically, it corresponds to the fast argument you pass to the pg_backup_start / pg_start_backup() function in PostgreSQL, accepting true (default) or false onlineConfiguration.waitForArchive : whether you want to wait for the archiver to process the last segment of the backup or not; technically, it corresponds to the wait_for_archive argument you pass to the pg_backup_stop / pg_stop_backup() function in PostgreSQL, accepting true (default) or false If you want to change the default behavior of your Postgres cluster to take cold backups by default, all you need to do is add the online: false option to your manifest, as follows: # ... backup: volumeSnapshot: online: false # ... If you are instead requesting an immediate checkpoint as the default behavior, you can add this section: # ... backup: volumeSnapshot: online: true onlineConfiguration: immediateCheckpoint: true # ...","title":"Hot and cold backups"},{"location":"backup_volumesnapshot/#overriding-the-default-behavior","text":"You can change the default behavior defined in the cluster resource by setting different values for online and, if needed, onlineConfiguration in the Backup or ScheduledBackup objects. For example, in case you want to issue an on-demand cold backup, you can create a Backup object with .spec.online: false : apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: name: snapshot-cluster-cold-backup-example spec: cluster: name: snapshot-cluster method: volumeSnapshot online: false Similarly, for the ScheduledBackup: apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: snapshot-cluster-cold-backup-example spec: schedule: \"0 0 0 * * *\" backupOwnerReference: self cluster: name: snapshot-cluster method: volumeSnapshot online: false","title":"Overriding the default behavior"},{"location":"backup_volumesnapshot/#persistence-of-volume-snapshot-objects","text":"By default, VolumeSnapshot objects created by CloudNativePG are retained after deleting the Backup object that originated them, or the Cluster they refer to. Such behavior is controlled by the .spec.backup.volumeSnapshot.snapshotOwnerReference option which accepts the following values: none : no ownership is set, meaning that VolumeSnapshot objects persist after the Backup and/or the Cluster resources are removed backup : the VolumeSnapshot object is owned by the Backup resource that originated it, and when the backup object is removed, the volume snapshot is also removed cluster : the VolumeSnapshot object is owned by the Cluster resource that is backed up, and when the Postgres cluster is removed, the volume snapshot is also removed In case a VolumeSnapshot is deleted, the deletionPolicy specified in the VolumeSnapshotContent is evaluated: if set to Retain , the VolumeSnapshotContent object is kept if set to Delete , the VolumeSnapshotContent object is removed as well Warning VolumeSnapshotContent objects do not keep all the information regarding the backup and the cluster they refer to (like the annotations and labels that are contained in the VolumeSnapshot object). Although possible, restoring from just this kind of object might not be straightforward. For this reason, our recommendation is to always backup the VolumeSnapshot definitions, even using a Kubernetes level data protection solution. The value in VolumeSnapshotContent is determined by the deletionPolicy set in the corresponding VolumeSnapshotClass definition, which is referenced in the .spec.backup.volumeSnapshot.className option. Please refer to the Kubernetes documentation on Volume Snapshot Classes for details on this standard behavior.","title":"Persistence of volume snapshot objects"},{"location":"backup_volumesnapshot/#backup-volume-snapshot-deadlines","text":"CloudNativePG supports backups using the volume snapshot method. In some environments, volume snapshots may encounter temporary issues that can be retried. The backup.cnpg.io/volumeSnapshotDeadline annotation defines how long CloudNativePG should continue retrying recoverable errors before marking the backup as failed. You can add the backup.cnpg.io/volumeSnapshotDeadline annotation to both Backup and ScheduledBackup resources. For ScheduledBackup resources, this annotation is automatically inherited by any Backup resources created from the schedule. If not specified, the default retry deadline is 10 minutes .","title":"Backup Volume Snapshot Deadlines"},{"location":"backup_volumesnapshot/#error-handling","text":"When a retryable error occurs during a volume snapshot operation: CloudNativePG records the time of the first error. The system retries the operation every 10 seconds . If the error persists beyond the specified deadline (or the default 10 minutes), the backup is marked as failed .","title":"Error Handling"},{"location":"backup_volumesnapshot/#retryable-errors","text":"CloudNativePG treats the following types of errors as retryable: Server timeout errors (HTTP 408, 429, 500, 502, 503, 504) Conflicts (optimistic locking errors) Internal errors Context deadline exceeded errors Timeout errors from the CSI snapshot controller","title":"Retryable Errors"},{"location":"backup_volumesnapshot/#examples","text":"You can add the annotation to a ScheduledBackup resource as follows: apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: daily-backup-schedule annotations: backup.cnpg.io/volumeSnapshotDeadline: \"20\" spec: schedule: \"0 0 * * *\" backupOwnerReference: self method: volumeSnapshot # other configuration... When you define a ScheduledBackup with the annotation, any Backup resources created from this schedule automatically inherit the specified timeout value. In the following example, all backups created from the schedule will have a 30-minute timeout for retrying recoverable snapshot errors. apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: weekly-backup annotations: backup.cnpg.io/volumeSnapshotDeadline: \"30\" spec: schedule: \"0 0 * * 0\" # Weekly backup on Sunday method: volumeSnapshot cluster: name: my-postgresql-cluster Alternatively, you can add the annotation directly to a Backup Resource: apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: name: my-backup annotations: backup.cnpg.io/volumeSnapshotDeadline: \"15\" spec: method: volumeSnapshot # other backup configuration...","title":"Examples"},{"location":"backup_volumesnapshot/#example-of-volume-snapshot-backup","text":"The following example shows how to configure volume snapshot base backups on an EKS cluster on AWS using the ebs-sc storage class and the csi-aws-vsc volume snapshot class. Important If you are interested in testing the example, please read \"Volume Snapshots\" for the Amazon Elastic Block Store (EBS) CSI driver for detailed instructions on the installation process for the storage class and the snapshot class. The following manifest creates a Cluster that is ready to be used for volume snapshots and that stores the WAL archive in a S3 bucket via IAM role for the Service Account (IRSA, see AWS S3 ): apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: hendrix spec: instances: 3 storage: storageClass: ebs-sc size: 10Gi walStorage: storageClass: ebs-sc size: 10Gi backup: volumeSnapshot: className: csi-aws-vsc barmanObjectStore: destinationPath: s3://@BUCKET_NAME@/ s3Credentials: inheritFromIAMRole: true wal: compression: gzip maxParallel: 2 serviceAccountTemplate: metadata: annotations: eks.amazonaws.com/role-arn: \"@ARN@\" --- apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: hendrix-vs-backup spec: cluster: name: hendrix method: volumeSnapshot schedule: '0 0 0 * * *' backupOwnerReference: cluster immediate: true The last resource defines daily volume snapshot backups at midnight, requesting one immediately after the cluster is created.","title":"Example of Volume Snapshot Backup"},{"location":"before_you_start/","text":"Before You Start Before we get started, it is essential to go over some terminology that is specific to Kubernetes and PostgreSQL. Kubernetes terminology Node A node is a worker machine in Kubernetes, either virtual or physical, where all services necessary to run pods are managed by the control plane node(s). Postgres Node A Postgres node is a Kubernetes worker node dedicated to running PostgreSQL workloads. This is achieved by applying the node-role.kubernetes.io label and taint, as proposed by CloudNativePG . It is also referred to as a postgres node. Pod A pod is the smallest computing unit that can be deployed in a Kubernetes cluster and is composed of one or more containers that share network and storage. Service A service is an abstraction that exposes as a network service an application that runs on a group of pods and standardizes important features such as service discovery across applications, load balancing, failover, and so on. Secret A secret is an object that is designed to store small amounts of sensitive data such as passwords, access keys, or tokens, and use them in pods. Storage Class A storage class allows an administrator to define the classes of storage in a cluster, including provisioner (such as AWS EBS), reclaim policies, mount options, volume expansion, and so on. Persistent Volume A persistent volume (PV) is a resource in a Kubernetes cluster that represents storage that has been either manually provisioned by an administrator or dynamically provisioned by a storage class controller. A PV is associated with a pod using a persistent volume claim and its lifecycle is independent of any pod that uses it. Normally, a PV is a network volume, especially in the public cloud. A local persistent volume (LPV) is a persistent volume that exists only on the particular node where the pod that uses it is running. Persistent Volume Claim A persistent volume claim (PVC) represents a request for storage, which might include size, access mode, or a particular storage class. Similar to how a pod consumes node resources, a PVC consumes the resources of a PV. Namespace A namespace is a logical and isolated subset of a Kubernetes cluster and can be seen as a virtual cluster within the wider physical cluster. Namespaces allow administrators to create separated environments based on projects, departments, teams, and so on. RBAC Role Based Access Control (RBAC), also known as role-based security , is a method used in computer systems security to restrict access to the network and resources of a system to authorized users only. Kubernetes has a native API to control roles at the namespace and cluster level and associate them with specific resources and individuals. CRD A custom resource definition (CRD) is an extension of the Kubernetes API and allows developers to create new data types and objects, called custom resources . Operator An operator is a custom resource that automates those steps that are normally performed by a human operator when managing one or more applications or given services. An operator assists Kubernetes in making sure that the resource's defined state always matches the observed one. kubectl kubectl is the command-line tool used to manage a Kubernetes cluster. CloudNativePG requires a Kubernetes version supported by the community. Please refer to the \"Supported releases\" page for details. PostgreSQL terminology Instance A Postgres server process running and listening on a pair \"IP address(es)\" and \"TCP port\" (usually 5432). Primary A PostgreSQL instance that can accept both read and write operations. Replica A PostgreSQL instance replicating from the only primary instance in a cluster and is kept updated by reading a stream of Write-Ahead Log (WAL) records. A replica is also known as standby or secondary server. PostgreSQL relies on physical streaming replication (async/sync) and file-based log shipping (async). Hot Standby PostgreSQL feature that allows a replica to accept read-only workloads. Cluster To be intended as High Availability (HA) Cluster: a set of PostgreSQL instances made up by a single primary and an optional arbitrary number of replicas. Replica Cluster A CloudNativePG Cluster that is in continuous recovery mode from a selected PostgreSQL cluster, normally residing outside the Kubernetes cluster. It is a feature that enables multi-cluster deployments in private, public, hybrid, and multi-cloud contexts. Designated Primary A PostgreSQL standby instance in a replica cluster that is in continuous recovery from another PostgreSQL cluster and that is designated to become primary in case the replica cluster becomes primary. Superuser In PostgreSQL a superuser is any role with both LOGIN and SUPERUSER privileges. For security reasons, CloudNativePG performs administrative tasks by connecting to the postgres database as the postgres user via peer authentication over the local Unix Domain Socket. WAL Write-Ahead Logging (WAL) is a standard method for ensuring data integrity in database management systems. PVC group A PVC group in CloudNativePG's terminology is a group of related PVCs belonging to the same PostgreSQL instance, namely the main volume containing the PGDATA ( storage ) and the volume for WALs ( walStorage ). RTO Acronym for \"recovery time objective\", the amount of time a system can be unavailable without adversely impacting the application. RPO Acronym for \"recovery point objective\", a calculation of the level of acceptable data loss following a disaster recovery scenario. Cloud terminology Region A region in the Cloud is an isolated and independent geographic area organized in availability zones . Zones within a region have very little round-trip network latency. Zone An availability zone in the Cloud (also known as zone ) is an area in a region where resources can be deployed. Usually, an availability zone corresponds to a data center or an isolated building of the same data center. What to do next Now that you have familiarized with the terminology, you can decide to test CloudNativePG on your laptop using a local cluster before deploying the operator in your selected cloud environment.","title":"Before You Start"},{"location":"before_you_start/#before-you-start","text":"Before we get started, it is essential to go over some terminology that is specific to Kubernetes and PostgreSQL.","title":"Before You Start"},{"location":"before_you_start/#kubernetes-terminology","text":"Node A node is a worker machine in Kubernetes, either virtual or physical, where all services necessary to run pods are managed by the control plane node(s). Postgres Node A Postgres node is a Kubernetes worker node dedicated to running PostgreSQL workloads. This is achieved by applying the node-role.kubernetes.io label and taint, as proposed by CloudNativePG . It is also referred to as a postgres node. Pod A pod is the smallest computing unit that can be deployed in a Kubernetes cluster and is composed of one or more containers that share network and storage. Service A service is an abstraction that exposes as a network service an application that runs on a group of pods and standardizes important features such as service discovery across applications, load balancing, failover, and so on. Secret A secret is an object that is designed to store small amounts of sensitive data such as passwords, access keys, or tokens, and use them in pods. Storage Class A storage class allows an administrator to define the classes of storage in a cluster, including provisioner (such as AWS EBS), reclaim policies, mount options, volume expansion, and so on. Persistent Volume A persistent volume (PV) is a resource in a Kubernetes cluster that represents storage that has been either manually provisioned by an administrator or dynamically provisioned by a storage class controller. A PV is associated with a pod using a persistent volume claim and its lifecycle is independent of any pod that uses it. Normally, a PV is a network volume, especially in the public cloud. A local persistent volume (LPV) is a persistent volume that exists only on the particular node where the pod that uses it is running. Persistent Volume Claim A persistent volume claim (PVC) represents a request for storage, which might include size, access mode, or a particular storage class. Similar to how a pod consumes node resources, a PVC consumes the resources of a PV. Namespace A namespace is a logical and isolated subset of a Kubernetes cluster and can be seen as a virtual cluster within the wider physical cluster. Namespaces allow administrators to create separated environments based on projects, departments, teams, and so on. RBAC Role Based Access Control (RBAC), also known as role-based security , is a method used in computer systems security to restrict access to the network and resources of a system to authorized users only. Kubernetes has a native API to control roles at the namespace and cluster level and associate them with specific resources and individuals. CRD A custom resource definition (CRD) is an extension of the Kubernetes API and allows developers to create new data types and objects, called custom resources . Operator An operator is a custom resource that automates those steps that are normally performed by a human operator when managing one or more applications or given services. An operator assists Kubernetes in making sure that the resource's defined state always matches the observed one. kubectl kubectl is the command-line tool used to manage a Kubernetes cluster. CloudNativePG requires a Kubernetes version supported by the community. Please refer to the \"Supported releases\" page for details.","title":"Kubernetes terminology"},{"location":"before_you_start/#postgresql-terminology","text":"Instance A Postgres server process running and listening on a pair \"IP address(es)\" and \"TCP port\" (usually 5432). Primary A PostgreSQL instance that can accept both read and write operations. Replica A PostgreSQL instance replicating from the only primary instance in a cluster and is kept updated by reading a stream of Write-Ahead Log (WAL) records. A replica is also known as standby or secondary server. PostgreSQL relies on physical streaming replication (async/sync) and file-based log shipping (async). Hot Standby PostgreSQL feature that allows a replica to accept read-only workloads. Cluster To be intended as High Availability (HA) Cluster: a set of PostgreSQL instances made up by a single primary and an optional arbitrary number of replicas. Replica Cluster A CloudNativePG Cluster that is in continuous recovery mode from a selected PostgreSQL cluster, normally residing outside the Kubernetes cluster. It is a feature that enables multi-cluster deployments in private, public, hybrid, and multi-cloud contexts. Designated Primary A PostgreSQL standby instance in a replica cluster that is in continuous recovery from another PostgreSQL cluster and that is designated to become primary in case the replica cluster becomes primary. Superuser In PostgreSQL a superuser is any role with both LOGIN and SUPERUSER privileges. For security reasons, CloudNativePG performs administrative tasks by connecting to the postgres database as the postgres user via peer authentication over the local Unix Domain Socket. WAL Write-Ahead Logging (WAL) is a standard method for ensuring data integrity in database management systems. PVC group A PVC group in CloudNativePG's terminology is a group of related PVCs belonging to the same PostgreSQL instance, namely the main volume containing the PGDATA ( storage ) and the volume for WALs ( walStorage ). RTO Acronym for \"recovery time objective\", the amount of time a system can be unavailable without adversely impacting the application. RPO Acronym for \"recovery point objective\", a calculation of the level of acceptable data loss following a disaster recovery scenario.","title":"PostgreSQL terminology"},{"location":"before_you_start/#cloud-terminology","text":"Region A region in the Cloud is an isolated and independent geographic area organized in availability zones . Zones within a region have very little round-trip network latency. Zone An availability zone in the Cloud (also known as zone ) is an area in a region where resources can be deployed. Usually, an availability zone corresponds to a data center or an isolated building of the same data center.","title":"Cloud terminology"},{"location":"before_you_start/#what-to-do-next","text":"Now that you have familiarized with the terminology, you can decide to test CloudNativePG on your laptop using a local cluster before deploying the operator in your selected cloud environment.","title":"What to do next"},{"location":"benchmarking/","text":"Benchmarking The CNPG kubectl plugin provides an easy way for benchmarking a PostgreSQL deployment in Kubernetes using CloudNativePG. Benchmarking is focused on two aspects: the database , by relying on pgbench the storage , by relying on fio Important pgbench and fio must be run in a staging or pre-production environment. Do not use these plugins in a production environment, as it might have catastrophic consequences on your databases and the other workloads/applications that run in the same shared environment. pgbench The kubectl CNPG plugin command pgbench executes a user-defined pgbench job against an existing Postgres Cluster. Through the --dry-run flag you can generate the manifest of the job for later modification/execution. A common command structure with pgbench is the following: kubectl cnpg pgbench \\ -n \\ --job-name \\ --db-name \\ -- Important Please refer to the pgbench documentation for information about the specific options to be used in your jobs. This example creates a job called pgbench-init that initializes for pgbench OLTP-like purposes the app database in a Cluster named cluster-example , using a scale factor of 1000: kubectl cnpg pgbench \\ --job-name pgbench-init \\ cluster-example \\ -- --initialize --scale 1000 Note This will generate a database with 100000000 records, taking approximately 13GB of space on disk. You can see the progress of the job with: kubectl logs jobs/pgbench-run The following example creates a job called pgbench-run executing pgbench against the previously initialized database for 30 seconds, using a single connection: kubectl cnpg pgbench \\ --job-name pgbench-run \\ cluster-example \\ -- --time 30 --client 1 --jobs 1 The next example runs pgbench against an existing database by using the --db-name flag and the pgbench namespace: kubectl cnpg pgbench \\ --db-name pgbench \\ --job-name pgbench-job \\ cluster-example \\ -- --time 30 --client 1 --jobs 1 By default, jobs do not expire. You can enable automatic deletion with the --ttl flag. The job will be deleted after the specified duration (in seconds). kubectl cnpg pgbench \\ --job-name pgbench-run \\ --ttl 600 \\ cluster-example \\ -- --time 30 --client 1 --jobs 1 If you want to run a pgbench job on a specific worker node, you can use the --node-selector option. Suppose you want to run the previous initialization job on a node having the workload=pgbench label, you can run: kubectl cnpg pgbench \\ --db-name pgbench \\ --job-name pgbench-init \\ --node-selector workload=pgbench \\ cluster-example \\ -- --initialize --scale 1000 The job status can be fetched by running: kubectl get job/pgbench-job -n NAME COMPLETIONS DURATION AGE job-name 1/1 15s 41s Once the job is completed the results can be gathered by executing: kubectl logs job/pgbench-job -n fio The kubectl CNPG plugin command fio executes a fio job with default values and read operations. Through the --dry-run flag you can generate the manifest of the job for later modification/execution. Note The kubectl plugin command fio will create a deployment with predefined fio job values using a ConfigMap. If you want to provide custom job values, we recommend generating a manifest using the --dry-run flag and providing your custom job values in the generated ConfigMap. Example of default usage: kubectl cnpg fio Example with custom values: kubectl cnpg fio \\ -n \\ --storageClass \\ --pvcSize Example of how to run the fio command against a StorageClass named standard and pvcSize: 2Gi in the fio namespace: kubectl cnpg fio fio-job \\ -n fio \\ --storageClass standard \\ --pvcSize 2Gi The deployment status can be fetched by running: kubectl get deployment/fio-job -n fio NAME READY UP-TO-DATE AVAILABLE AGE fio-job 1/1 1 1 14s After running kubectl plugin command fio . It will: Create a PVC Create a ConfigMap representing the configuration of a fio job Create a fio deployment composed by a single Pod, which will run fio on the PVC, create graphs after completing the benchmark and start serving the generated files with a webserver. We use the fio-tools image for that. The Pod created by the deployment will be ready when it starts serving the results. You can forward the port of the pod created by the deployment kubectl port-forward -n deployment/ 8000 and then use a browser and connect to http://localhost:8000/ to get the data. The default 8k block size has been chosen to emulate a PostgreSQL workload. Disks that cap the amount of available IOPS can show very different throughput values when changing this parameter. Below is an example diagram of sequential writes on a local disk mounted on a dedicated Kubernetes node (1 hour benchmark): After all testing is done, fio deployment and resources can be deleted by: kubectl cnpg fio --dry-run | kubectl delete -f - make sure use the same name which was used to create the fio deployment and add namespace if applicable.","title":"Benchmarking"},{"location":"benchmarking/#benchmarking","text":"The CNPG kubectl plugin provides an easy way for benchmarking a PostgreSQL deployment in Kubernetes using CloudNativePG. Benchmarking is focused on two aspects: the database , by relying on pgbench the storage , by relying on fio Important pgbench and fio must be run in a staging or pre-production environment. Do not use these plugins in a production environment, as it might have catastrophic consequences on your databases and the other workloads/applications that run in the same shared environment.","title":"Benchmarking"},{"location":"benchmarking/#pgbench","text":"The kubectl CNPG plugin command pgbench executes a user-defined pgbench job against an existing Postgres Cluster. Through the --dry-run flag you can generate the manifest of the job for later modification/execution. A common command structure with pgbench is the following: kubectl cnpg pgbench \\ -n \\ --job-name \\ --db-name \\ -- Important Please refer to the pgbench documentation for information about the specific options to be used in your jobs. This example creates a job called pgbench-init that initializes for pgbench OLTP-like purposes the app database in a Cluster named cluster-example , using a scale factor of 1000: kubectl cnpg pgbench \\ --job-name pgbench-init \\ cluster-example \\ -- --initialize --scale 1000 Note This will generate a database with 100000000 records, taking approximately 13GB of space on disk. You can see the progress of the job with: kubectl logs jobs/pgbench-run The following example creates a job called pgbench-run executing pgbench against the previously initialized database for 30 seconds, using a single connection: kubectl cnpg pgbench \\ --job-name pgbench-run \\ cluster-example \\ -- --time 30 --client 1 --jobs 1 The next example runs pgbench against an existing database by using the --db-name flag and the pgbench namespace: kubectl cnpg pgbench \\ --db-name pgbench \\ --job-name pgbench-job \\ cluster-example \\ -- --time 30 --client 1 --jobs 1 By default, jobs do not expire. You can enable automatic deletion with the --ttl flag. The job will be deleted after the specified duration (in seconds). kubectl cnpg pgbench \\ --job-name pgbench-run \\ --ttl 600 \\ cluster-example \\ -- --time 30 --client 1 --jobs 1 If you want to run a pgbench job on a specific worker node, you can use the --node-selector option. Suppose you want to run the previous initialization job on a node having the workload=pgbench label, you can run: kubectl cnpg pgbench \\ --db-name pgbench \\ --job-name pgbench-init \\ --node-selector workload=pgbench \\ cluster-example \\ -- --initialize --scale 1000 The job status can be fetched by running: kubectl get job/pgbench-job -n NAME COMPLETIONS DURATION AGE job-name 1/1 15s 41s Once the job is completed the results can be gathered by executing: kubectl logs job/pgbench-job -n ","title":"pgbench"},{"location":"benchmarking/#fio","text":"The kubectl CNPG plugin command fio executes a fio job with default values and read operations. Through the --dry-run flag you can generate the manifest of the job for later modification/execution. Note The kubectl plugin command fio will create a deployment with predefined fio job values using a ConfigMap. If you want to provide custom job values, we recommend generating a manifest using the --dry-run flag and providing your custom job values in the generated ConfigMap. Example of default usage: kubectl cnpg fio Example with custom values: kubectl cnpg fio \\ -n \\ --storageClass \\ --pvcSize Example of how to run the fio command against a StorageClass named standard and pvcSize: 2Gi in the fio namespace: kubectl cnpg fio fio-job \\ -n fio \\ --storageClass standard \\ --pvcSize 2Gi The deployment status can be fetched by running: kubectl get deployment/fio-job -n fio NAME READY UP-TO-DATE AVAILABLE AGE fio-job 1/1 1 1 14s After running kubectl plugin command fio . It will: Create a PVC Create a ConfigMap representing the configuration of a fio job Create a fio deployment composed by a single Pod, which will run fio on the PVC, create graphs after completing the benchmark and start serving the generated files with a webserver. We use the fio-tools image for that. The Pod created by the deployment will be ready when it starts serving the results. You can forward the port of the pod created by the deployment kubectl port-forward -n deployment/ 8000 and then use a browser and connect to http://localhost:8000/ to get the data. The default 8k block size has been chosen to emulate a PostgreSQL workload. Disks that cap the amount of available IOPS can show very different throughput values when changing this parameter. Below is an example diagram of sequential writes on a local disk mounted on a dedicated Kubernetes node (1 hour benchmark): After all testing is done, fio deployment and resources can be deleted by: kubectl cnpg fio --dry-run | kubectl delete -f - make sure use the same name which was used to create the fio deployment and add namespace if applicable.","title":"fio"},{"location":"bootstrap/","text":"Bootstrap This section describes the options available to create a new PostgreSQL cluster and the design rationale behind them. There are primarily two ways to bootstrap a new cluster: from scratch ( initdb ) from an existing PostgreSQL cluster, either directly ( pg_basebackup ) or indirectly through a physical base backup ( recovery ) The initdb bootstrap also provides the option to import one or more databases from an existing PostgreSQL cluster, even if it's outside Kubernetes or running a different major version of PostgreSQL. For more detailed information about this feature, please refer to the \"Importing Postgres databases\" section. Important Bootstrapping from an existing cluster enables the creation of a replica cluster \u2014an independent PostgreSQL cluster that remains in continuous recovery, stays synchronized with the source cluster, and accepts read-only connections. For more details, refer to the Replica Cluster section . Warning CloudNativePG requires both the postgres user and database to always exist. Using the local Unix Domain Socket, it needs to connect as the postgres user to the postgres database via peer authentication in order to perform administrative tasks on the cluster. DO NOT DELETE the postgres user or the postgres database!!! Info CloudNativePG is gradually introducing support for Kubernetes' native VolumeSnapshot API for both incremental and differential copy in backup and recovery operations - if supported by the underlying storage classes. Please see \"Recovery from Volume Snapshot objects\" for details. The bootstrap section The bootstrap method can be defined in the bootstrap section of the cluster specification. CloudNativePG currently supports the following bootstrap methods: initdb : initialize a new PostgreSQL cluster (default) recovery : create a PostgreSQL cluster by restoring from a base backup of an existing cluster and, if needed, replaying all the available WAL files or up to a given point in time pg_basebackup : create a PostgreSQL cluster by cloning an existing one of the same major version using pg_basebackup through the streaming replication protocol. This method is particularly useful for migrating databases to CloudNativePG, although meeting all requirements can be challenging. Be sure to review the warnings in the pg_basebackup subsection carefully. Only one bootstrap method can be specified in the manifest. Attempting to define multiple bootstrap methods will result in validation errors. In contrast to the initdb method, both recovery and pg_basebackup create a new cluster based on another one (either offline or online) and can be used to spin up replica clusters. They both rely on the definition of external clusters. Refer to the replica cluster section for more information. Given the amount of possible backup methods and combinations of backup storage that the CloudNativePG operator provides for recovery , please refer to the dedicated \"Recovery\" section for guidance on each method. API reference Please refer to the \"API reference for the bootstrap section for more information. The externalClusters section The externalClusters section of the cluster manifest can be used to configure access to one or more PostgreSQL clusters as sources . The primary use cases include: Importing Databases: Specify an external source to be utilized during the importation of databases via logical backup and restore, as part of the initdb bootstrap method. Cross-Region Replication: Define a cross-region PostgreSQL cluster employing physical replication, capable of extending across distinct Kubernetes clusters or traditional VM/bare-metal environments. Recovery from Physical Base Backup: Recover, fully or at a given Point-In-Time, a PostgreSQL cluster by referencing a physical base backup. Info Ongoing development will extend the functionality of externalClusters to accommodate additional use cases, such as logical replication and foreign servers in future releases. As far as bootstrapping is concerned, externalClusters can be used to define the source PostgreSQL cluster for either the pg_basebackup method or the recovery one. An external cluster needs to have: a name that identifies the external cluster, to be used as a reference via the source option at least one of the following: information about streaming connection information about the recovery object store , which is a Barman Cloud compatible object store that contains: the WAL archive (required for Point In Time Recovery) the catalog of physical base backups for the Postgres cluster Note A recovery object store is normally an AWS S3, Azure Blob Storage, or Google Cloud Storage source that is managed by Barman Cloud. When only the streaming connection is defined, the source can be used for the pg_basebackup method. When only the recovery object store is defined, the source can be used for the recovery method. When both are defined, any of the two bootstrap methods can be chosen. The following table summarizes your options: Content of externalClusters pg_basebackup recovery Only streaming \u2713 Only object store \u2713 Streaming and object store \u2713 \u2713 Furthermore, in case of pg_basebackup or full recovery point in time, the cluster is eligible for replica cluster mode. This means that the cluster is continuously fed from the source, either via streaming, via WAL shipping through the PostgreSQL's restore_command , or any of the two. API reference Please refer to the \"API reference for the externalClusters section for more information. Password files Whenever a password is supplied within an externalClusters entry, CloudNativePG autonomously manages a PostgreSQL password file for it, residing at /controller/external/NAME/pgpass in each instance. This approach enables CloudNativePG to securely establish connections with an external server without exposing any passwords in the connection string. Instead, the connection safely references the aforementioned file through the passfile connection parameter. Bootstrap an empty cluster ( initdb ) The initdb bootstrap method is used to create a new PostgreSQL cluster from scratch. It is the default one unless specified differently. The following example contains the full structure of the initdb configuration: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-initdb spec: instances: 3 bootstrap: initdb: database: app owner: app secret: name: app-secret storage: size: 1Gi The above example of bootstrap will: create a new PGDATA folder using PostgreSQL's native initdb command create an unprivileged user named app set the password of the latter ( app ) using the one in the app-secret secret (make sure that username matches the same name of the owner ) create a database called app owned by the app user. Thanks to the convention over configuration paradigm , you can let the operator choose a default database name ( app ) and a default application user name (same as the database name), as well as randomly generate a secure password for both the superuser and the application user in PostgreSQL. Alternatively, you can generate your password, store it as a secret, and use it in the PostgreSQL cluster - as described in the above example. The supplied secret must comply with the specifications of the kubernetes.io/basic-auth type . As a result, the username in the secret must match the one of the owner (for the application secret) and postgres for the superuser one. The following is an example of a basic-auth secret: apiVersion: v1 data: username: YXBw password: cGFzc3dvcmQ= kind: Secret metadata: name: app-secret type: kubernetes.io/basic-auth The application database is the one that should be used to store application data. Applications should connect to the cluster with the user that owns the application database. Important If you need to create additional users, please refer to \"Declarative database role management\" . In case you don't supply any database name, the operator will proceed by convention and create the app database, and adds it to the cluster definition using a defaulting webhook . The user that owns the database defaults to the database name instead. The application user is not used internally by the operator, which instead relies on the superuser to reconcile the cluster with the desired status. Passing Options to initdb The PostgreSQL data directory is initialized using the initdb PostgreSQL command . CloudNativePG enables you to customize the behavior of initdb to modify settings such as default locale configurations and data checksums. Warning CloudNativePG acts only as a direct proxy to initdb for locale-related options, due to the ongoing and significant enhancements in PostgreSQL's locale support. It is your responsibility to ensure that the correct options are provided, following the PostgreSQL documentation, and to verify that the bootstrap process completes successfully. To include custom options in the initdb command, you can use the following parameters: builtinLocale When builtinLocale is set to a value, CloudNativePG passes it to the --builtin-locale option in initdb . This option controls the builtin locale, as defined in \"Locale Support\" from the PostgreSQL documentation (default: empty). Note that this option requires localeProvider to be set to builtin . Available from PostgreSQL 17. dataChecksums When dataChecksums is set to true , CloudNativePG invokes the -k option in initdb to enable checksums on data pages and help detect corruption by the I/O system - that would otherwise be silent (default: false ). encoding When encoding set to a value, CloudNativePG passes it to the --encoding option in initdb , which selects the encoding of the template database (default: UTF8 ). icuLocale When icuLocale is set to a value, CloudNativePG passes it to the --icu-locale option in initdb . This option controls the ICU locale, as defined in \"Locale Support\" from the PostgreSQL documentation (default: empty). Note that this option requires localeProvider to be set to icu . Available from PostgreSQL 15. icuRules When icuRules is set to a value, CloudNativePG passes it to the --icu-rules option in initdb . This option controls the ICU locale, as defined in \"Locale Support\" from the PostgreSQL documentation (default: empty). Note that this option requires localeProvider to be set to icu . Available from PostgreSQL 16. locale When locale is set to a value, CloudNativePG passes it to the --locale option in initdb . This option controls the locale, as defined in \"Locale Support\" from the PostgreSQL documentation. By default, the locale parameter is empty. In this case, environment variables such as LANG are used to determine the locale. Be aware that these variables can vary between container images, potentially leading to inconsistent behavior. localeCollate When localeCollate is set to a value, CloudNativePG passes it to the --lc-collate option in initdb . This option controls the collation order ( LC_COLLATE subcategory), as defined in \"Locale Support\" from the PostgreSQL documentation (default: C ). localeCType When localeCType is set to a value, CloudNativePG passes it to the --lc-ctype option in initdb . This option controls the collation order ( LC_CTYPE subcategory), as defined in \"Locale Support\" from the PostgreSQL documentation (default: C ). localeProvider When localeProvider is set to a value, CloudNativePG passes it to the --locale-provider option in initdb . This option controls the locale provider, as defined in \"Locale Support\" from the PostgreSQL documentation (default: empty, which means libc for PostgreSQL). Available from PostgreSQL 15. walSegmentSize When walSegmentSize is set to a value, CloudNativePG passes it to the --wal-segsize option in initdb (default: not set - defined by PostgreSQL as 16 megabytes). Note The only two locale options that CloudNativePG implements during the initdb bootstrap refer to the LC_COLLATE and LC_TYPE subcategories. The remaining locale subcategories can be configured directly in the PostgreSQL configuration, using the lc_messages , lc_monetary , lc_numeric , and lc_time parameters. The following example enables data checksums and sets the default encoding to LATIN1 : apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-initdb spec: instances: 3 bootstrap: initdb: database: app owner: app dataChecksums: true encoding: 'LATIN1' storage: size: 1Gi Warning CloudNativePG supports another way to customize the behavior of the initdb invocation, using the options subsection. However, given that there are options that can break the behavior of the operator (such as --auth or -d ), this technique is deprecated and will be removed from future versions of the API. Executing Queries After Initialization You can specify a custom list of queries that will be executed once, immediately after the cluster is created and configured. These queries will be executed as the superuser ( postgres ) against three different databases, in this specific order: The postgres database ( postInit section) The template1 database ( postInitTemplate section) The application database ( postInitApplication section) For each of these sections, CloudNativePG provides two ways to specify custom queries, executed in the following order: As a list of SQL queries in the cluster's definition ( postInitSQL , postInitTemplateSQL , and postInitApplicationSQL stanzas) As a list of Secrets and/or ConfigMaps, each containing a SQL script to be executed ( postInitSQLRefs , postInitTemplateSQLRefs , and postInitApplicationSQLRefs stanzas). Secrets are processed before ConfigMaps. Objects in each list will be processed sequentially. Warning Use the postInit , postInitTemplate , and postInitApplication options with extreme care, as queries are run as a superuser and can disrupt the entire cluster. An error in any of those queries will interrupt the bootstrap phase, leaving the cluster incomplete and requiring manual intervention. Important Ensure the existence of entries inside the ConfigMaps or Secrets specified in postInitSQLRefs , postInitTemplateSQLRefs , and postInitApplicationSQLRefs , otherwise the bootstrap will fail. Errors in any of those SQL files will prevent the bootstrap phase from completing successfully. The following example runs a single SQL query as part of the postInitSQL stanza: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-initdb spec: instances: 3 bootstrap: initdb: database: app owner: app dataChecksums: true localeCollate: 'en_US' localeCType: 'en_US' postInitSQL: - CREATE DATABASE angus storage: size: 1Gi The example below relies on postInitApplicationSQLRefs to specify a secret and a ConfigMap containing the queries to run after the initialization on the application database: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-initdb spec: instances: 3 bootstrap: initdb: database: app owner: app postInitApplicationSQLRefs: secretRefs: - name: my-secret key: secret.sql configMapRefs: - name: my-configmap key: configmap.sql storage: size: 1Gi Note Within SQL scripts, each SQL statement is executed in a single exec on the server according to the PostgreSQL semantics . Comments can be included, but internal commands like psql cannot. Bootstrap from another cluster CloudNativePG enables bootstrapping a cluster starting from another one of the same major version. This operation can be carried out either connecting directly to the source cluster via streaming replication ( pg_basebackup ), or indirectly via an existing physical base backup ( recovery ). The source cluster must be defined in the externalClusters section, identified by name (our recommendation is to use the same name of the origin cluster). Important By default the recovery method strictly uses the name of the cluster in the externalClusters section to locate the main folder of the backup data within the object store, which is normally reserved for the name of the server. You can specify a different one with the barmanObjectStore.serverName property (by default assigned to the value of name in the external cluster definition). Bootstrap from a backup ( recovery ) Given the variety of backup methods and combinations of backup storage options provided by the CloudNativePG operator for recovery , please refer to the dedicated \"Recovery\" section for detailed guidance on each method. Bootstrap from a live cluster ( pg_basebackup ) The pg_basebackup bootstrap mode allows you to create a new cluster ( target ) as an exact physical copy of an existing and binary-compatible PostgreSQL instance ( source ) managed by CloudNativePG, using a valid streaming replication connection. The source instance can either be a primary or a standby PostgreSQL server. It\u2019s crucial to thoroughly review the requirements section below, as the pros and cons of PostgreSQL physical replication fully apply. The primary use cases for this method include: Reporting and business intelligence clusters that need to be regenerated periodically (daily, weekly) Test databases containing live data that require periodic regeneration (daily, weekly, monthly) and anonymization Rapid spin-up of a standalone replica cluster Physical migrations of CloudNativePG clusters to different namespaces or Kubernetes clusters Important Avoid using this method, based on physical replication, to migrate an existing PostgreSQL cluster outside of Kubernetes into CloudNativePG, unless you are completely certain that all requirements are met and the operation has been thoroughly tested. The CloudNativePG community does not endorse this approach for such use cases, and recommends using logical import instead. It is exceedingly rare that all requirements for physical replication are met in a way that seamlessly works with CloudNativePG. Warning In its current implementation, this method clones the source PostgreSQL instance, thereby creating a snapshot . Once the cloning process has finished, the new cluster is immediately started. Refer to \"Current limitations\" for more details. Similar to the recovery bootstrap method, once the cloning operation is complete, the operator takes full ownership of the target cluster, starting from the first instance. This includes overriding certain configuration parameters as required by CloudNativePG, resetting the superuser password, creating the streaming_replica user, managing replicas, and more. The resulting cluster operates independently from the source instance. Important Configuring the network connection between the target and source instances lies outside the scope of CloudNativePG documentation, as it depends heavily on the specific context and environment. The streaming replication client on the target instance, managed transparently by pg_basebackup , can authenticate on the source instance using one of the following methods: Username/password TLS client certificate Both authentication methods are detailed below. Requirements The following requirements apply to the pg_basebackup bootstrap method: target and source must have the same hardware architecture target and source must have the same major PostgreSQL version target and source must have the same tablespaces source must be configured with enough max_wal_senders to grant access from the target for this one-off operation by providing at least one walsender for the backup plus one for WAL streaming the network between source and target must be configured to enable the target instance to connect to the PostgreSQL port on the source instance source must have a role with REPLICATION LOGIN privileges and must accept connections from the target instance for this role in pg_hba.conf , preferably via TLS (see \"About the replication user\" below) target must be able to successfully connect to the source PostgreSQL instance using a role with REPLICATION LOGIN privileges Seealso For further information, please refer to the \"Planning\" section for Warm Standby , the pg_basebackup page and the \"High Availability, Load Balancing, and Replication\" chapter in the PostgreSQL documentation. About the replication user As explained in the requirements section, you need to have a user with either the SUPERUSER or, preferably, just the REPLICATION privilege in the source instance. If the source database is created with CloudNativePG, you can reuse the streaming_replica user and take advantage of client TLS certificates authentication (which, by default, is the only allowed connection method for streaming_replica ). For all other cases, including outside Kubernetes, please verify that you already have a user with the REPLICATION privilege, or create a new one by following the instructions below. As postgres user on the source system, please run: createuser -P --replication streaming_replica Enter the password at the prompt and save it for later, as you will need to add it to a secret in the target instance. Note Although the name is not important, we will use streaming_replica for the sake of simplicity. Feel free to change it as you like, provided you adapt the instructions in the following sections. Username/Password authentication The first authentication method supported by CloudNativePG with the pg_basebackup bootstrap is based on username and password matching. Make sure you have the following information before you start the procedure: location of the source instance, identified by a hostname or an IP address and a TCP port replication username ( streaming_replica for simplicity) password You might need to add a line similar to the following to the pg_hba.conf file on the source PostgreSQL instance: # A more restrictive rule for TLS and IP of origin is recommended host replication streaming_replica all md5 The following manifest creates a new PostgreSQL 17.5 cluster, called target-db , using the pg_basebackup bootstrap method to clone an external PostgreSQL cluster defined as source-db (in the externalClusters array). As you can see, the source-db definition points to the source-db.foo.com host and connects as the streaming_replica user, whose password is stored in the password key of the source-db-replica-user secret. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: target-db spec: instances: 3 imageName: ghcr.io/cloudnative-pg/postgresql:17.5 bootstrap: pg_basebackup: source: source-db storage: size: 1Gi externalClusters: - name: source-db connectionParameters: host: source-db.foo.com user: streaming_replica password: name: source-db-replica-user key: password All the requirements must be met for the clone operation to work, including the same PostgreSQL version (in our case 17.5). TLS certificate authentication The second authentication method supported by CloudNativePG with the pg_basebackup bootstrap is based on TLS client certificates. This is the recommended approach from a security standpoint. The following example clones an existing PostgreSQL cluster ( cluster-example ) in the same Kubernetes cluster. Note This example can be easily adapted to cover an instance that resides outside the Kubernetes cluster. The manifest defines a new PostgreSQL 17.5 cluster called cluster-clone-tls , which is bootstrapped using the pg_basebackup method from the cluster-example external cluster. The host is identified by the read/write service in the same cluster, while the streaming_replica user is authenticated thanks to the provided keys, certificate, and certification authority information (respectively in the cluster-example-replication and cluster-example-ca secrets). apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-clone-tls spec: instances: 3 imageName: ghcr.io/cloudnative-pg/postgresql:17.5 bootstrap: pg_basebackup: source: cluster-example storage: size: 1Gi externalClusters: - name: cluster-example connectionParameters: host: cluster-example-rw.default.svc user: streaming_replica sslmode: verify-full sslKey: name: cluster-example-replication key: tls.key sslCert: name: cluster-example-replication key: tls.crt sslRootCert: name: cluster-example-ca key: ca.crt Configure the application database We also support to configure the application database for cluster which bootstrap from a live cluster, just like the case of initdb and recovery bootstrap method. If the new cluster is created as a replica cluster (with replica mode enabled), application database configuration will be skipped. Important While the Cluster is in recovery mode, no changes to the database, including the catalog, are permitted. This restriction includes any role overrides, which are deferred until the Cluster transitions to primary. During the recovery phase, roles remain as defined in the source cluster. The example below configures the app database with the owner app and the password stored in the provided secret app-secret , following the bootstrap from a live cluster. apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: bootstrap: pg_basebackup: database: app owner: app secret: name: app-secret source: cluster-example With the above configuration, the following will happen only after recovery is completed : If the app database does not exist, it will be created. If the app user does not exist, it will be created. If the app user is not the owner of the app database, ownership will be granted to the app user. If the username value matches the owner value in the secret, the password for the application user (the app user in this case) will be updated to the password value in the secret. Current limitations Snapshot copy The pg_basebackup method takes a snapshot of the source instance in the form of a PostgreSQL base backup. All transactions written from the start of the backup to the correct termination of the backup will be streamed to the target instance using a second connection (see the --wal-method=stream option for pg_basebackup ). Once the backup is completed, the new instance will be started on a new timeline and diverge from the source. For this reason, it is advised to stop all write operations to the source database before migrating to the target database. Note that this limitation applies only if the target cluster is not defined as a replica cluster. Important Before you attempt a migration, you must test both the procedure and the applications. In particular, it is fundamental that you run the migration procedure as many times as needed to systematically measure the downtime of your applications in production.","title":"Bootstrap"},{"location":"bootstrap/#bootstrap","text":"This section describes the options available to create a new PostgreSQL cluster and the design rationale behind them. There are primarily two ways to bootstrap a new cluster: from scratch ( initdb ) from an existing PostgreSQL cluster, either directly ( pg_basebackup ) or indirectly through a physical base backup ( recovery ) The initdb bootstrap also provides the option to import one or more databases from an existing PostgreSQL cluster, even if it's outside Kubernetes or running a different major version of PostgreSQL. For more detailed information about this feature, please refer to the \"Importing Postgres databases\" section. Important Bootstrapping from an existing cluster enables the creation of a replica cluster \u2014an independent PostgreSQL cluster that remains in continuous recovery, stays synchronized with the source cluster, and accepts read-only connections. For more details, refer to the Replica Cluster section . Warning CloudNativePG requires both the postgres user and database to always exist. Using the local Unix Domain Socket, it needs to connect as the postgres user to the postgres database via peer authentication in order to perform administrative tasks on the cluster. DO NOT DELETE the postgres user or the postgres database!!! Info CloudNativePG is gradually introducing support for Kubernetes' native VolumeSnapshot API for both incremental and differential copy in backup and recovery operations - if supported by the underlying storage classes. Please see \"Recovery from Volume Snapshot objects\" for details.","title":"Bootstrap"},{"location":"bootstrap/#the-bootstrap-section","text":"The bootstrap method can be defined in the bootstrap section of the cluster specification. CloudNativePG currently supports the following bootstrap methods: initdb : initialize a new PostgreSQL cluster (default) recovery : create a PostgreSQL cluster by restoring from a base backup of an existing cluster and, if needed, replaying all the available WAL files or up to a given point in time pg_basebackup : create a PostgreSQL cluster by cloning an existing one of the same major version using pg_basebackup through the streaming replication protocol. This method is particularly useful for migrating databases to CloudNativePG, although meeting all requirements can be challenging. Be sure to review the warnings in the pg_basebackup subsection carefully. Only one bootstrap method can be specified in the manifest. Attempting to define multiple bootstrap methods will result in validation errors. In contrast to the initdb method, both recovery and pg_basebackup create a new cluster based on another one (either offline or online) and can be used to spin up replica clusters. They both rely on the definition of external clusters. Refer to the replica cluster section for more information. Given the amount of possible backup methods and combinations of backup storage that the CloudNativePG operator provides for recovery , please refer to the dedicated \"Recovery\" section for guidance on each method. API reference Please refer to the \"API reference for the bootstrap section for more information.","title":"The bootstrap section"},{"location":"bootstrap/#the-externalclusters-section","text":"The externalClusters section of the cluster manifest can be used to configure access to one or more PostgreSQL clusters as sources . The primary use cases include: Importing Databases: Specify an external source to be utilized during the importation of databases via logical backup and restore, as part of the initdb bootstrap method. Cross-Region Replication: Define a cross-region PostgreSQL cluster employing physical replication, capable of extending across distinct Kubernetes clusters or traditional VM/bare-metal environments. Recovery from Physical Base Backup: Recover, fully or at a given Point-In-Time, a PostgreSQL cluster by referencing a physical base backup. Info Ongoing development will extend the functionality of externalClusters to accommodate additional use cases, such as logical replication and foreign servers in future releases. As far as bootstrapping is concerned, externalClusters can be used to define the source PostgreSQL cluster for either the pg_basebackup method or the recovery one. An external cluster needs to have: a name that identifies the external cluster, to be used as a reference via the source option at least one of the following: information about streaming connection information about the recovery object store , which is a Barman Cloud compatible object store that contains: the WAL archive (required for Point In Time Recovery) the catalog of physical base backups for the Postgres cluster Note A recovery object store is normally an AWS S3, Azure Blob Storage, or Google Cloud Storage source that is managed by Barman Cloud. When only the streaming connection is defined, the source can be used for the pg_basebackup method. When only the recovery object store is defined, the source can be used for the recovery method. When both are defined, any of the two bootstrap methods can be chosen. The following table summarizes your options: Content of externalClusters pg_basebackup recovery Only streaming \u2713 Only object store \u2713 Streaming and object store \u2713 \u2713 Furthermore, in case of pg_basebackup or full recovery point in time, the cluster is eligible for replica cluster mode. This means that the cluster is continuously fed from the source, either via streaming, via WAL shipping through the PostgreSQL's restore_command , or any of the two. API reference Please refer to the \"API reference for the externalClusters section for more information.","title":"The externalClusters section"},{"location":"bootstrap/#password-files","text":"Whenever a password is supplied within an externalClusters entry, CloudNativePG autonomously manages a PostgreSQL password file for it, residing at /controller/external/NAME/pgpass in each instance. This approach enables CloudNativePG to securely establish connections with an external server without exposing any passwords in the connection string. Instead, the connection safely references the aforementioned file through the passfile connection parameter.","title":"Password files"},{"location":"bootstrap/#bootstrap-an-empty-cluster-initdb","text":"The initdb bootstrap method is used to create a new PostgreSQL cluster from scratch. It is the default one unless specified differently. The following example contains the full structure of the initdb configuration: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-initdb spec: instances: 3 bootstrap: initdb: database: app owner: app secret: name: app-secret storage: size: 1Gi The above example of bootstrap will: create a new PGDATA folder using PostgreSQL's native initdb command create an unprivileged user named app set the password of the latter ( app ) using the one in the app-secret secret (make sure that username matches the same name of the owner ) create a database called app owned by the app user. Thanks to the convention over configuration paradigm , you can let the operator choose a default database name ( app ) and a default application user name (same as the database name), as well as randomly generate a secure password for both the superuser and the application user in PostgreSQL. Alternatively, you can generate your password, store it as a secret, and use it in the PostgreSQL cluster - as described in the above example. The supplied secret must comply with the specifications of the kubernetes.io/basic-auth type . As a result, the username in the secret must match the one of the owner (for the application secret) and postgres for the superuser one. The following is an example of a basic-auth secret: apiVersion: v1 data: username: YXBw password: cGFzc3dvcmQ= kind: Secret metadata: name: app-secret type: kubernetes.io/basic-auth The application database is the one that should be used to store application data. Applications should connect to the cluster with the user that owns the application database. Important If you need to create additional users, please refer to \"Declarative database role management\" . In case you don't supply any database name, the operator will proceed by convention and create the app database, and adds it to the cluster definition using a defaulting webhook . The user that owns the database defaults to the database name instead. The application user is not used internally by the operator, which instead relies on the superuser to reconcile the cluster with the desired status.","title":"Bootstrap an empty cluster (initdb)"},{"location":"bootstrap/#passing-options-to-initdb","text":"The PostgreSQL data directory is initialized using the initdb PostgreSQL command . CloudNativePG enables you to customize the behavior of initdb to modify settings such as default locale configurations and data checksums. Warning CloudNativePG acts only as a direct proxy to initdb for locale-related options, due to the ongoing and significant enhancements in PostgreSQL's locale support. It is your responsibility to ensure that the correct options are provided, following the PostgreSQL documentation, and to verify that the bootstrap process completes successfully. To include custom options in the initdb command, you can use the following parameters: builtinLocale When builtinLocale is set to a value, CloudNativePG passes it to the --builtin-locale option in initdb . This option controls the builtin locale, as defined in \"Locale Support\" from the PostgreSQL documentation (default: empty). Note that this option requires localeProvider to be set to builtin . Available from PostgreSQL 17. dataChecksums When dataChecksums is set to true , CloudNativePG invokes the -k option in initdb to enable checksums on data pages and help detect corruption by the I/O system - that would otherwise be silent (default: false ). encoding When encoding set to a value, CloudNativePG passes it to the --encoding option in initdb , which selects the encoding of the template database (default: UTF8 ). icuLocale When icuLocale is set to a value, CloudNativePG passes it to the --icu-locale option in initdb . This option controls the ICU locale, as defined in \"Locale Support\" from the PostgreSQL documentation (default: empty). Note that this option requires localeProvider to be set to icu . Available from PostgreSQL 15. icuRules When icuRules is set to a value, CloudNativePG passes it to the --icu-rules option in initdb . This option controls the ICU locale, as defined in \"Locale Support\" from the PostgreSQL documentation (default: empty). Note that this option requires localeProvider to be set to icu . Available from PostgreSQL 16. locale When locale is set to a value, CloudNativePG passes it to the --locale option in initdb . This option controls the locale, as defined in \"Locale Support\" from the PostgreSQL documentation. By default, the locale parameter is empty. In this case, environment variables such as LANG are used to determine the locale. Be aware that these variables can vary between container images, potentially leading to inconsistent behavior. localeCollate When localeCollate is set to a value, CloudNativePG passes it to the --lc-collate option in initdb . This option controls the collation order ( LC_COLLATE subcategory), as defined in \"Locale Support\" from the PostgreSQL documentation (default: C ). localeCType When localeCType is set to a value, CloudNativePG passes it to the --lc-ctype option in initdb . This option controls the collation order ( LC_CTYPE subcategory), as defined in \"Locale Support\" from the PostgreSQL documentation (default: C ). localeProvider When localeProvider is set to a value, CloudNativePG passes it to the --locale-provider option in initdb . This option controls the locale provider, as defined in \"Locale Support\" from the PostgreSQL documentation (default: empty, which means libc for PostgreSQL). Available from PostgreSQL 15. walSegmentSize When walSegmentSize is set to a value, CloudNativePG passes it to the --wal-segsize option in initdb (default: not set - defined by PostgreSQL as 16 megabytes). Note The only two locale options that CloudNativePG implements during the initdb bootstrap refer to the LC_COLLATE and LC_TYPE subcategories. The remaining locale subcategories can be configured directly in the PostgreSQL configuration, using the lc_messages , lc_monetary , lc_numeric , and lc_time parameters. The following example enables data checksums and sets the default encoding to LATIN1 : apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-initdb spec: instances: 3 bootstrap: initdb: database: app owner: app dataChecksums: true encoding: 'LATIN1' storage: size: 1Gi Warning CloudNativePG supports another way to customize the behavior of the initdb invocation, using the options subsection. However, given that there are options that can break the behavior of the operator (such as --auth or -d ), this technique is deprecated and will be removed from future versions of the API.","title":"Passing Options to initdb"},{"location":"bootstrap/#executing-queries-after-initialization","text":"You can specify a custom list of queries that will be executed once, immediately after the cluster is created and configured. These queries will be executed as the superuser ( postgres ) against three different databases, in this specific order: The postgres database ( postInit section) The template1 database ( postInitTemplate section) The application database ( postInitApplication section) For each of these sections, CloudNativePG provides two ways to specify custom queries, executed in the following order: As a list of SQL queries in the cluster's definition ( postInitSQL , postInitTemplateSQL , and postInitApplicationSQL stanzas) As a list of Secrets and/or ConfigMaps, each containing a SQL script to be executed ( postInitSQLRefs , postInitTemplateSQLRefs , and postInitApplicationSQLRefs stanzas). Secrets are processed before ConfigMaps. Objects in each list will be processed sequentially. Warning Use the postInit , postInitTemplate , and postInitApplication options with extreme care, as queries are run as a superuser and can disrupt the entire cluster. An error in any of those queries will interrupt the bootstrap phase, leaving the cluster incomplete and requiring manual intervention. Important Ensure the existence of entries inside the ConfigMaps or Secrets specified in postInitSQLRefs , postInitTemplateSQLRefs , and postInitApplicationSQLRefs , otherwise the bootstrap will fail. Errors in any of those SQL files will prevent the bootstrap phase from completing successfully. The following example runs a single SQL query as part of the postInitSQL stanza: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-initdb spec: instances: 3 bootstrap: initdb: database: app owner: app dataChecksums: true localeCollate: 'en_US' localeCType: 'en_US' postInitSQL: - CREATE DATABASE angus storage: size: 1Gi The example below relies on postInitApplicationSQLRefs to specify a secret and a ConfigMap containing the queries to run after the initialization on the application database: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-initdb spec: instances: 3 bootstrap: initdb: database: app owner: app postInitApplicationSQLRefs: secretRefs: - name: my-secret key: secret.sql configMapRefs: - name: my-configmap key: configmap.sql storage: size: 1Gi Note Within SQL scripts, each SQL statement is executed in a single exec on the server according to the PostgreSQL semantics . Comments can be included, but internal commands like psql cannot.","title":"Executing Queries After Initialization"},{"location":"bootstrap/#bootstrap-from-another-cluster","text":"CloudNativePG enables bootstrapping a cluster starting from another one of the same major version. This operation can be carried out either connecting directly to the source cluster via streaming replication ( pg_basebackup ), or indirectly via an existing physical base backup ( recovery ). The source cluster must be defined in the externalClusters section, identified by name (our recommendation is to use the same name of the origin cluster). Important By default the recovery method strictly uses the name of the cluster in the externalClusters section to locate the main folder of the backup data within the object store, which is normally reserved for the name of the server. You can specify a different one with the barmanObjectStore.serverName property (by default assigned to the value of name in the external cluster definition).","title":"Bootstrap from another cluster"},{"location":"bootstrap/#bootstrap-from-a-backup-recovery","text":"Given the variety of backup methods and combinations of backup storage options provided by the CloudNativePG operator for recovery , please refer to the dedicated \"Recovery\" section for detailed guidance on each method.","title":"Bootstrap from a backup (recovery)"},{"location":"bootstrap/#bootstrap-from-a-live-cluster-pg_basebackup","text":"The pg_basebackup bootstrap mode allows you to create a new cluster ( target ) as an exact physical copy of an existing and binary-compatible PostgreSQL instance ( source ) managed by CloudNativePG, using a valid streaming replication connection. The source instance can either be a primary or a standby PostgreSQL server. It\u2019s crucial to thoroughly review the requirements section below, as the pros and cons of PostgreSQL physical replication fully apply. The primary use cases for this method include: Reporting and business intelligence clusters that need to be regenerated periodically (daily, weekly) Test databases containing live data that require periodic regeneration (daily, weekly, monthly) and anonymization Rapid spin-up of a standalone replica cluster Physical migrations of CloudNativePG clusters to different namespaces or Kubernetes clusters Important Avoid using this method, based on physical replication, to migrate an existing PostgreSQL cluster outside of Kubernetes into CloudNativePG, unless you are completely certain that all requirements are met and the operation has been thoroughly tested. The CloudNativePG community does not endorse this approach for such use cases, and recommends using logical import instead. It is exceedingly rare that all requirements for physical replication are met in a way that seamlessly works with CloudNativePG. Warning In its current implementation, this method clones the source PostgreSQL instance, thereby creating a snapshot . Once the cloning process has finished, the new cluster is immediately started. Refer to \"Current limitations\" for more details. Similar to the recovery bootstrap method, once the cloning operation is complete, the operator takes full ownership of the target cluster, starting from the first instance. This includes overriding certain configuration parameters as required by CloudNativePG, resetting the superuser password, creating the streaming_replica user, managing replicas, and more. The resulting cluster operates independently from the source instance. Important Configuring the network connection between the target and source instances lies outside the scope of CloudNativePG documentation, as it depends heavily on the specific context and environment. The streaming replication client on the target instance, managed transparently by pg_basebackup , can authenticate on the source instance using one of the following methods: Username/password TLS client certificate Both authentication methods are detailed below.","title":"Bootstrap from a live cluster (pg_basebackup)"},{"location":"bootstrap/#requirements","text":"The following requirements apply to the pg_basebackup bootstrap method: target and source must have the same hardware architecture target and source must have the same major PostgreSQL version target and source must have the same tablespaces source must be configured with enough max_wal_senders to grant access from the target for this one-off operation by providing at least one walsender for the backup plus one for WAL streaming the network between source and target must be configured to enable the target instance to connect to the PostgreSQL port on the source instance source must have a role with REPLICATION LOGIN privileges and must accept connections from the target instance for this role in pg_hba.conf , preferably via TLS (see \"About the replication user\" below) target must be able to successfully connect to the source PostgreSQL instance using a role with REPLICATION LOGIN privileges Seealso For further information, please refer to the \"Planning\" section for Warm Standby , the pg_basebackup page and the \"High Availability, Load Balancing, and Replication\" chapter in the PostgreSQL documentation.","title":"Requirements"},{"location":"bootstrap/#about-the-replication-user","text":"As explained in the requirements section, you need to have a user with either the SUPERUSER or, preferably, just the REPLICATION privilege in the source instance. If the source database is created with CloudNativePG, you can reuse the streaming_replica user and take advantage of client TLS certificates authentication (which, by default, is the only allowed connection method for streaming_replica ). For all other cases, including outside Kubernetes, please verify that you already have a user with the REPLICATION privilege, or create a new one by following the instructions below. As postgres user on the source system, please run: createuser -P --replication streaming_replica Enter the password at the prompt and save it for later, as you will need to add it to a secret in the target instance. Note Although the name is not important, we will use streaming_replica for the sake of simplicity. Feel free to change it as you like, provided you adapt the instructions in the following sections.","title":"About the replication user"},{"location":"bootstrap/#usernamepassword-authentication","text":"The first authentication method supported by CloudNativePG with the pg_basebackup bootstrap is based on username and password matching. Make sure you have the following information before you start the procedure: location of the source instance, identified by a hostname or an IP address and a TCP port replication username ( streaming_replica for simplicity) password You might need to add a line similar to the following to the pg_hba.conf file on the source PostgreSQL instance: # A more restrictive rule for TLS and IP of origin is recommended host replication streaming_replica all md5 The following manifest creates a new PostgreSQL 17.5 cluster, called target-db , using the pg_basebackup bootstrap method to clone an external PostgreSQL cluster defined as source-db (in the externalClusters array). As you can see, the source-db definition points to the source-db.foo.com host and connects as the streaming_replica user, whose password is stored in the password key of the source-db-replica-user secret. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: target-db spec: instances: 3 imageName: ghcr.io/cloudnative-pg/postgresql:17.5 bootstrap: pg_basebackup: source: source-db storage: size: 1Gi externalClusters: - name: source-db connectionParameters: host: source-db.foo.com user: streaming_replica password: name: source-db-replica-user key: password All the requirements must be met for the clone operation to work, including the same PostgreSQL version (in our case 17.5).","title":"Username/Password authentication"},{"location":"bootstrap/#tls-certificate-authentication","text":"The second authentication method supported by CloudNativePG with the pg_basebackup bootstrap is based on TLS client certificates. This is the recommended approach from a security standpoint. The following example clones an existing PostgreSQL cluster ( cluster-example ) in the same Kubernetes cluster. Note This example can be easily adapted to cover an instance that resides outside the Kubernetes cluster. The manifest defines a new PostgreSQL 17.5 cluster called cluster-clone-tls , which is bootstrapped using the pg_basebackup method from the cluster-example external cluster. The host is identified by the read/write service in the same cluster, while the streaming_replica user is authenticated thanks to the provided keys, certificate, and certification authority information (respectively in the cluster-example-replication and cluster-example-ca secrets). apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-clone-tls spec: instances: 3 imageName: ghcr.io/cloudnative-pg/postgresql:17.5 bootstrap: pg_basebackup: source: cluster-example storage: size: 1Gi externalClusters: - name: cluster-example connectionParameters: host: cluster-example-rw.default.svc user: streaming_replica sslmode: verify-full sslKey: name: cluster-example-replication key: tls.key sslCert: name: cluster-example-replication key: tls.crt sslRootCert: name: cluster-example-ca key: ca.crt","title":"TLS certificate authentication"},{"location":"bootstrap/#configure-the-application-database","text":"We also support to configure the application database for cluster which bootstrap from a live cluster, just like the case of initdb and recovery bootstrap method. If the new cluster is created as a replica cluster (with replica mode enabled), application database configuration will be skipped. Important While the Cluster is in recovery mode, no changes to the database, including the catalog, are permitted. This restriction includes any role overrides, which are deferred until the Cluster transitions to primary. During the recovery phase, roles remain as defined in the source cluster. The example below configures the app database with the owner app and the password stored in the provided secret app-secret , following the bootstrap from a live cluster. apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: bootstrap: pg_basebackup: database: app owner: app secret: name: app-secret source: cluster-example With the above configuration, the following will happen only after recovery is completed : If the app database does not exist, it will be created. If the app user does not exist, it will be created. If the app user is not the owner of the app database, ownership will be granted to the app user. If the username value matches the owner value in the secret, the password for the application user (the app user in this case) will be updated to the password value in the secret.","title":"Configure the application database"},{"location":"bootstrap/#current-limitations","text":"","title":"Current limitations"},{"location":"bootstrap/#snapshot-copy","text":"The pg_basebackup method takes a snapshot of the source instance in the form of a PostgreSQL base backup. All transactions written from the start of the backup to the correct termination of the backup will be streamed to the target instance using a second connection (see the --wal-method=stream option for pg_basebackup ). Once the backup is completed, the new instance will be started on a new timeline and diverge from the source. For this reason, it is advised to stop all write operations to the source database before migrating to the target database. Note that this limitation applies only if the target cluster is not defined as a replica cluster. Important Before you attempt a migration, you must test both the procedure and the applications. In particular, it is fundamental that you run the migration procedure as many times as needed to systematically measure the downtime of your applications in production.","title":"Snapshot copy"},{"location":"certificates/","text":"Certificates CloudNativePG was designed to natively support TLS certificates. To set up a cluster, the operator requires: A server certification authority (CA) certificate A server TLS certificate signed by the server CA A client CA certificate A streaming replication client certificate generated by the client CA Note You can find all the secrets used by the cluster and their expiration dates in the cluster's status. CloudNativePG is very flexible when it comes to TLS certificates. It primarily operates in two modes: Operator managed \u2013 Certificates are internally managed by the operator in a fully automated way and signed using a CA created by CloudNativePG. User provided \u2013 Certificates are generated outside the operator and imported in the cluster definition as secrets. CloudNativePG integrates itself with cert-manager (See Cert-manager example .) You can also choose a hybrid approach, where only part of the certificates is generated outside CNPG. Note The operator and instances verify server certificates against the CA only, disregarding the DNS name. This approach is due to the typical absence of DNS names in user-provided certificates for the -rw service used for communication within the cluster. Operator-Managed Mode By default, the operator automatically generates a single Certificate Authority (CA) to issue both client and server certificates. These certificates are managed continuously by the operator, with automatic renewal 7 days before expiration (within a 90-day validity period). Info You can adjust this default behavior by configuring the CERTIFICATE_DURATION and EXPIRING_CHECK_THRESHOLD environment variables. For detailed guidance, refer to the Operator Configuration . Important Certificate renewal does not cause any downtime for the PostgreSQL server, as a simple reload operation is sufficient. However, any user-managed certificates not controlled by CloudNativePG must be re-issued following the renewal process. When generating certificates, the operator assumes that the Kubernetes cluster's DNS zone is set to cluster.local by default. This behavior can be customized by setting the KUBERNETES_CLUSTER_DOMAIN environment variable. A convenient alternative is to use the operator's configuration capability . Server certificates Server CA secret The operator generates a self-signed CA and stores it in a generic secret containing the following keys: ca.crt \u2013 CA certificate used to validate the server certificate, used as sslrootcert in clients' connection strings. ca.key \u2013 The key used to sign the server SSL certificate automatically. Server TLS secret The operator uses the generated self-signed CA to sign a server TLS certificate. It's stored in a secret of type kubernetes.io/tls and configured to be used as ssl_cert_file and ssl_key_file by the instances. This approach enables clients to verify their identity and connect securely. Server alternative DNS names In addition to the default ones, you can specify DNS server alternative names as part of the generated server TLS secret. Client certificates Client CA secret By default, the same self-signed CA as the server CA is used. The public part is passed as ssl_ca_file to all the instances so it can verify client certificates it signed. The private key is stored in the same secret and used to sign client certificates generated by the kubectl cnpg plugin. Client streaming_replica certificate The operator uses the generated self-signed CA to sign a client certificate for the user streaming_replica , storing it in a secret of type kubernetes.io/tls . To allow secure connection to the primary instance, this certificate is passed as sslcert and sslkey in the replicas' connection strings. User-provided certificates mode Server certificates If required, you can also provide the two server certificates, generating them using a separate component such as cert-manager . To use a custom server TLS certificate for a cluster, you must specify the following parameters: serverTLSSecret \u2013 The name of a secret of type kubernetes.io/tls containing the server TLS certificate. It must contain both the standard tls.crt and tls.key keys. serverCASecret \u2013 The name of a secret containing the ca.crt key. Note The operator still creates and manages the two secrets related to client certificates. Note The operator and instances verify server certificates against the CA only, disregarding the DNS name. This approach is due to the typical absence of DNS names in user-provided certificates for the -rw service used for communication within the cluster. Note If you want ConfigMaps and secrets to be reloaded by instances, you can add a label with the key cnpg.io/reload to it. Otherwise you must reload the instances using the kubectl cnpg reload subcommand. Example Given the following files: server-ca.crt \u2013 The certificate of the CA that signed the server TLS certificate. server.crt \u2013 The certificate of the server TLS certificate. server.key \u2013 The private key of the server TLS certificate. Create a secret containing the CA certificate: kubectl create secret generic my-postgresql-server-ca \\ --from-file=ca.crt=./server-ca.crt Create a secret with the TLS certificate: kubectl create secret tls my-postgresql-server \\ --cert=./server.crt --key=./server.key Create a PostgreSQL cluster referencing those secrets: kubectl apply -f - <-rw service used for communication within the cluster.","title":"Certificates"},{"location":"certificates/#operator-managed-mode","text":"By default, the operator automatically generates a single Certificate Authority (CA) to issue both client and server certificates. These certificates are managed continuously by the operator, with automatic renewal 7 days before expiration (within a 90-day validity period). Info You can adjust this default behavior by configuring the CERTIFICATE_DURATION and EXPIRING_CHECK_THRESHOLD environment variables. For detailed guidance, refer to the Operator Configuration . Important Certificate renewal does not cause any downtime for the PostgreSQL server, as a simple reload operation is sufficient. However, any user-managed certificates not controlled by CloudNativePG must be re-issued following the renewal process. When generating certificates, the operator assumes that the Kubernetes cluster's DNS zone is set to cluster.local by default. This behavior can be customized by setting the KUBERNETES_CLUSTER_DOMAIN environment variable. A convenient alternative is to use the operator's configuration capability .","title":"Operator-Managed Mode"},{"location":"certificates/#server-certificates","text":"","title":"Server certificates"},{"location":"certificates/#server-ca-secret","text":"The operator generates a self-signed CA and stores it in a generic secret containing the following keys: ca.crt \u2013 CA certificate used to validate the server certificate, used as sslrootcert in clients' connection strings. ca.key \u2013 The key used to sign the server SSL certificate automatically.","title":"Server CA secret"},{"location":"certificates/#server-tls-secret","text":"The operator uses the generated self-signed CA to sign a server TLS certificate. It's stored in a secret of type kubernetes.io/tls and configured to be used as ssl_cert_file and ssl_key_file by the instances. This approach enables clients to verify their identity and connect securely.","title":"Server TLS secret"},{"location":"certificates/#server-alternative-dns-names","text":"In addition to the default ones, you can specify DNS server alternative names as part of the generated server TLS secret.","title":"Server alternative DNS names"},{"location":"certificates/#client-certificates","text":"","title":"Client certificates"},{"location":"certificates/#client-ca-secret","text":"By default, the same self-signed CA as the server CA is used. The public part is passed as ssl_ca_file to all the instances so it can verify client certificates it signed. The private key is stored in the same secret and used to sign client certificates generated by the kubectl cnpg plugin.","title":"Client CA secret"},{"location":"certificates/#client-streaming_replica-certificate","text":"The operator uses the generated self-signed CA to sign a client certificate for the user streaming_replica , storing it in a secret of type kubernetes.io/tls . To allow secure connection to the primary instance, this certificate is passed as sslcert and sslkey in the replicas' connection strings.","title":"Client streaming_replica certificate"},{"location":"certificates/#user-provided-certificates-mode","text":"","title":"User-provided certificates mode"},{"location":"certificates/#server-certificates_1","text":"If required, you can also provide the two server certificates, generating them using a separate component such as cert-manager . To use a custom server TLS certificate for a cluster, you must specify the following parameters: serverTLSSecret \u2013 The name of a secret of type kubernetes.io/tls containing the server TLS certificate. It must contain both the standard tls.crt and tls.key keys. serverCASecret \u2013 The name of a secret containing the ca.crt key. Note The operator still creates and manages the two secrets related to client certificates. Note The operator and instances verify server certificates against the CA only, disregarding the DNS name. This approach is due to the typical absence of DNS names in user-provided certificates for the -rw service used for communication within the cluster. Note If you want ConfigMaps and secrets to be reloaded by instances, you can add a label with the key cnpg.io/reload to it. Otherwise you must reload the instances using the kubectl cnpg reload subcommand.","title":"Server certificates"},{"location":"certificates/#example","text":"Given the following files: server-ca.crt \u2013 The certificate of the CA that signed the server TLS certificate. server.crt \u2013 The certificate of the server TLS certificate. server.key \u2013 The private key of the server TLS certificate. Create a secret containing the CA certificate: kubectl create secret generic my-postgresql-server-ca \\ --from-file=ca.crt=./server-ca.crt Create a secret with the TLS certificate: kubectl create secret tls my-postgresql-server \\ --cert=./server.crt --key=./server.key Create a PostgreSQL cluster referencing those secrets: kubectl apply -f - < Once the cluster exits recovery, the password for the superuser will be changed through the provided secret. Refer to the Bootstrap page of the documentation for more information. Field Description backup BackupSource The backup object containing the physical base backup from which to initiate the recovery procedure. Mutually exclusive with source and volumeSnapshots . source string The external cluster whose backup we will restore. This is also used as the name of the folder under which the backup is stored, so it must be set to the name of the source cluster Mutually exclusive with backup . volumeSnapshots DataSource The static PVC data source(s) from which to initiate the recovery procedure. Currently supporting VolumeSnapshot and PersistentVolumeClaim resources that map an existing PVC group, compatible with CloudNativePG, and taken with a cold backup copy on a fenced Postgres instance (limitation which will be removed in the future when online backup will be implemented). Mutually exclusive with backup . recoveryTarget RecoveryTarget By default, the recovery process applies all the available WAL files in the archive (full recovery). However, you can also end the recovery as soon as a consistent state is reached or recover to a point-in-time (PITR) by specifying a RecoveryTarget object, as expected by PostgreSQL (i.e., timestamp, transaction Id, LSN, ...). More info: https://www.postgresql.org/docs/current/runtime-config-wal.html#RUNTIME-CONFIG-WAL-RECOVERY-TARGET database string Name of the database used by the application. Default: app . owner string Name of the owner of the database in the instance to be used by applications. Defaults to the value of the database key. secret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference Name of the secret containing the initial credentials for the owner of the user database. If empty a new secret will be created from scratch CatalogImage Appears in: ImageCatalogSpec CatalogImage defines the image and major version Field Description image [Required] string The image reference major [Required] int The PostgreSQL major version of the image. Must be unique within the catalog. CertificatesConfiguration Appears in: CertificatesStatus ClusterSpec CertificatesConfiguration contains the needed configurations to handle server certificates. Field Description serverCASecret string The secret containing the Server CA certificate. If not defined, a new secret will be created with a self-signed CA and will be used to generate the TLS certificate ServerTLSSecret. Contains: ca.crt : CA that should be used to validate the server certificate, used as sslrootcert in client connection strings. ca.key : key used to generate Server SSL certs, if ServerTLSSecret is provided, this can be omitted. serverTLSSecret string The secret of type kubernetes.io/tls containing the server TLS certificate and key that will be set as ssl_cert_file and ssl_key_file so that clients can connect to postgres securely. If not defined, ServerCASecret must provide also ca.key and a new secret will be created using the provided CA. replicationTLSSecret string The secret of type kubernetes.io/tls containing the client certificate to authenticate as the streaming_replica user. If not defined, ClientCASecret must provide also ca.key , and a new secret will be created using the provided CA. clientCASecret string The secret containing the Client CA certificate. If not defined, a new secret will be created with a self-signed CA and will be used to generate all the client certificates. Contains: ca.crt : CA that should be used to validate the client certificates, used as ssl_ca_file of all the instances. ca.key : key used to generate client certificates, if ReplicationTLSSecret is provided, this can be omitted. serverAltDNSNames []string The list of the server alternative DNS names to be added to the generated server TLS certificates, when required. CertificatesStatus Appears in: ClusterStatus CertificatesStatus contains configuration certificates and related expiration dates. Field Description CertificatesConfiguration CertificatesConfiguration (Members of CertificatesConfiguration are embedded into this type.) Needed configurations to handle server certificates, initialized with default values, if needed. expirations map[string]string Expiration dates for all certificates. ClusterMonitoringTLSConfiguration Appears in: MonitoringConfiguration ClusterMonitoringTLSConfiguration is the type containing the TLS configuration for the cluster's monitoring Field Description enabled bool Enable TLS for the monitoring endpoint. Changing this option will force a rollout of all instances. ClusterSpec Appears in: Cluster ClusterSpec defines the desired state of Cluster Field Description description string Description of this PostgreSQL cluster inheritedMetadata EmbeddedObjectMetadata Metadata that will be inherited by all objects related to the Cluster imageName string Name of the container image, supporting both tags ( : ) and digests for deterministic and repeatable deployments ( :@sha256: ) imageCatalogRef ImageCatalogRef Defines the major PostgreSQL version we want to use within an ImageCatalog imagePullPolicy core/v1.PullPolicy Image pull policy. One of Always , Never or IfNotPresent . If not defined, it defaults to IfNotPresent . Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images schedulerName string If specified, the pod will be dispatched by specified Kubernetes scheduler. If not specified, the pod will be dispatched by the default scheduler. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/ postgresUID int64 The UID of the postgres user inside the image, defaults to 26 postgresGID int64 The GID of the postgres user inside the image, defaults to 26 instances [Required] int Number of instances required in the cluster minSyncReplicas int Minimum number of instances required in synchronous replication with the primary. Undefined or 0 allow writes to complete when no standby is available. maxSyncReplicas int The target value for the synchronous replication quorum, that can be decreased if the number of ready standbys is lower than this. Undefined or 0 disable synchronous replication. postgresql PostgresConfiguration Configuration of the PostgreSQL server replicationSlots ReplicationSlotsConfiguration Replication slots management configuration bootstrap BootstrapConfiguration Instructions to bootstrap this cluster replica ReplicaClusterConfiguration Replica cluster configuration superuserSecret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The secret containing the superuser password. If not defined a new secret will be created with a randomly generated password enableSuperuserAccess bool When this option is enabled, the operator will use the SuperuserSecret to update the postgres user password (if the secret is not present, the operator will automatically create one). When this option is disabled, the operator will ignore the SuperuserSecret content, delete it when automatically created, and then blank the password of the postgres user by setting it to NULL . Disabled by default. certificates CertificatesConfiguration The configuration for the CA and related certificates imagePullSecrets []github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The list of pull secrets to be used to pull the images storage StorageConfiguration Configuration of the storage of the instances serviceAccountTemplate ServiceAccountTemplate Configure the generation of the service account walStorage StorageConfiguration Configuration of the storage for PostgreSQL WAL (Write-Ahead Log) ephemeralVolumeSource core/v1.EphemeralVolumeSource EphemeralVolumeSource allows the user to configure the source of ephemeral volumes. startDelay int32 The time in seconds that is allowed for a PostgreSQL instance to successfully start up (default 3600). The startup probe failure threshold is derived from this value using the formula: ceiling(startDelay / 10). stopDelay int32 The time in seconds that is allowed for a PostgreSQL instance to gracefully shutdown (default 1800) smartShutdownTimeout int32 The time in seconds that controls the window of time reserved for the smart shutdown of Postgres to complete. Make sure you reserve enough time for the operator to request a fast shutdown of Postgres (that is: stopDelay - smartShutdownTimeout ). switchoverDelay int32 The time in seconds that is allowed for a primary PostgreSQL instance to gracefully shutdown during a switchover. Default value is 3600 seconds (1 hour). failoverDelay int32 The amount of time (in seconds) to wait before triggering a failover after the primary PostgreSQL instance in the cluster was detected to be unhealthy livenessProbeTimeout int32 LivenessProbeTimeout is the time (in seconds) that is allowed for a PostgreSQL instance to successfully respond to the liveness probe (default 30). The Liveness probe failure threshold is derived from this value using the formula: ceiling(livenessProbe / 10). affinity AffinityConfiguration Affinity/Anti-affinity rules for Pods topologySpreadConstraints []core/v1.TopologySpreadConstraint TopologySpreadConstraints specifies how to spread matching pods among the given topology. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/ resources core/v1.ResourceRequirements Resources requirements of every generated Pod. Please refer to https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ for more information. ephemeralVolumesSizeLimit EphemeralVolumesSizeLimitConfiguration EphemeralVolumesSizeLimit allows the user to set the limits for the ephemeral volumes priorityClassName string Name of the priority class which will be used in every generated Pod, if the PriorityClass specified does not exist, the pod will not be able to schedule. Please refer to https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass for more information primaryUpdateStrategy PrimaryUpdateStrategy Deployment strategy to follow to upgrade the primary server during a rolling update procedure, after all replicas have been successfully updated: it can be automated ( unsupervised - default) or manual ( supervised ) primaryUpdateMethod PrimaryUpdateMethod Method to follow to upgrade the primary server during a rolling update procedure, after all replicas have been successfully updated: it can be with a switchover ( switchover ) or in-place ( restart - default) backup BackupConfiguration The configuration to be used for backups nodeMaintenanceWindow NodeMaintenanceWindow Define a maintenance window for the Kubernetes nodes monitoring MonitoringConfiguration The configuration of the monitoring infrastructure of this cluster externalClusters []ExternalCluster The list of external clusters which are used in the configuration logLevel string The instances' log level, one of the following values: error, warning, info (default), debug, trace projectedVolumeTemplate core/v1.ProjectedVolumeSource Template to be used to define projected volumes, projected volumes will be mounted under /projected base folder env []core/v1.EnvVar Env follows the Env format to pass environment variables to the pods created in the cluster envFrom []core/v1.EnvFromSource EnvFrom follows the EnvFrom format to pass environment variables sources to the pods to be used by Env managed ManagedConfiguration The configuration that is used by the portions of PostgreSQL that are managed by the instance manager seccompProfile core/v1.SeccompProfile The SeccompProfile applied to every Pod and Container. Defaults to: RuntimeDefault tablespaces []TablespaceConfiguration The tablespaces configuration enablePDB bool Manage the PodDisruptionBudget resources within the cluster. When configured as true (default setting), the pod disruption budgets will safeguard the primary node from being terminated. Conversely, setting it to false will result in the absence of any PodDisruptionBudget resource, permitting the shutdown of all nodes hosting the PostgreSQL cluster. This latter configuration is advisable for any PostgreSQL cluster employed for development/staging purposes. plugins []PluginConfiguration The plugins configuration, containing any plugin to be loaded with the corresponding configuration probes ProbesConfiguration The configuration of the probes to be injected in the PostgreSQL Pods. ClusterStatus Appears in: Cluster ClusterStatus defines the observed state of Cluster Field Description instances int The total number of PVC Groups detected in the cluster. It may differ from the number of existing instance pods. readyInstances int The total number of ready instances in the cluster. It is equal to the number of ready instance pods. instancesStatus map[PodStatus][]string InstancesStatus indicates in which status the instances are instancesReportedState map[PodName]InstanceReportedState The reported state of the instances during the last reconciliation loop managedRolesStatus ManagedRoles ManagedRolesStatus reports the state of the managed roles in the cluster tablespacesStatus []TablespaceState TablespacesStatus reports the state of the declarative tablespaces in the cluster timelineID int The timeline of the Postgres cluster topology Topology Instances topology. latestGeneratedNode int ID of the latest generated node (used to avoid node name clashing) currentPrimary string Current primary instance targetPrimary string Target primary instance, this is different from the previous one during a switchover or a failover lastPromotionToken string LastPromotionToken is the last verified promotion token that was used to promote a replica cluster pvcCount int32 How many PVCs have been created by this cluster jobCount int32 How many Jobs have been created by this cluster danglingPVC []string List of all the PVCs created by this cluster and still available which are not attached to a Pod resizingPVC []string List of all the PVCs that have ResizingPVC condition. initializingPVC []string List of all the PVCs that are being initialized by this cluster healthyPVC []string List of all the PVCs not dangling nor initializing unusablePVC []string List of all the PVCs that are unusable because another PVC is missing writeService string Current write pod readService string Current list of read pods phase string Current phase of the cluster phaseReason string Reason for the current phase secretsResourceVersion SecretsResourceVersion The list of resource versions of the secrets managed by the operator. Every change here is done in the interest of the instance manager, which will refresh the secret data configMapResourceVersion ConfigMapResourceVersion The list of resource versions of the configmaps, managed by the operator. Every change here is done in the interest of the instance manager, which will refresh the configmap data certificates CertificatesStatus The configuration for the CA and related certificates, initialized with defaults. firstRecoverabilityPoint string The first recoverability point, stored as a date in RFC3339 format. This field is calculated from the content of FirstRecoverabilityPointByMethod firstRecoverabilityPointByMethod map[BackupMethod]meta/v1.Time The first recoverability point, stored as a date in RFC3339 format, per backup method type lastSuccessfulBackup string Last successful backup, stored as a date in RFC3339 format This field is calculated from the content of LastSuccessfulBackupByMethod lastSuccessfulBackupByMethod map[BackupMethod]meta/v1.Time Last successful backup, stored as a date in RFC3339 format, per backup method type lastFailedBackup string Stored as a date in RFC3339 format cloudNativePGCommitHash string The commit hash number of which this operator running currentPrimaryTimestamp string The timestamp when the last actual promotion to primary has occurred currentPrimaryFailingSinceTimestamp string The timestamp when the primary was detected to be unhealthy This field is reported when .spec.failoverDelay is populated or during online upgrades targetPrimaryTimestamp string The timestamp when the last request for a new primary has occurred poolerIntegrations PoolerIntegrations The integration needed by poolers referencing the cluster cloudNativePGOperatorHash string The hash of the binary of the operator availableArchitectures []AvailableArchitecture AvailableArchitectures reports the available architectures of a cluster conditions []meta/v1.Condition Conditions for cluster object instanceNames []string List of instance names in the cluster onlineUpdateEnabled bool OnlineUpdateEnabled shows if the online upgrade is enabled inside the cluster azurePVCUpdateEnabled bool AzurePVCUpdateEnabled shows if the PVC online upgrade is enabled for this cluster image string Image contains the image name used by the pods pluginStatus []PluginStatus PluginStatus is the status of the loaded plugins switchReplicaClusterStatus SwitchReplicaClusterStatus SwitchReplicaClusterStatus is the status of the switch to replica cluster demotionToken string DemotionToken is a JSON token containing the information from pg_controldata such as Database system identifier, Latest checkpoint's TimeLineID, Latest checkpoint's REDO location, Latest checkpoint's REDO WAL file, and Time of latest checkpoint systemID string SystemID is the latest detected PostgreSQL SystemID ConfigMapResourceVersion Appears in: ClusterStatus ConfigMapResourceVersion is the resource versions of the secrets managed by the operator Field Description metrics map[string]string A map with the versions of all the config maps used to pass metrics. Map keys are the config map names, map values are the versions DataDurabilityLevel (Alias of string ) Appears in: SynchronousReplicaConfiguration DataDurabilityLevel specifies how strictly to enforce synchronous replication when cluster instances are unavailable. Options are required or preferred . DataSource Appears in: BootstrapRecovery DataSource contains the configuration required to bootstrap a PostgreSQL cluster from an existing storage Field Description storage [Required] core/v1.TypedLocalObjectReference Configuration of the storage of the instances walStorage core/v1.TypedLocalObjectReference Configuration of the storage for PostgreSQL WAL (Write-Ahead Log) tablespaceStorage map[string]core/v1.TypedLocalObjectReference Configuration of the storage for PostgreSQL tablespaces DatabaseReclaimPolicy (Alias of string ) Appears in: DatabaseSpec DatabaseReclaimPolicy describes a policy for end-of-life maintenance of databases. DatabaseRoleRef Appears in: TablespaceConfiguration DatabaseRoleRef is a reference an a role available inside PostgreSQL Field Description name string No description provided. DatabaseSpec Appears in: Database DatabaseSpec is the specification of a Postgresql Database, built around the CREATE DATABASE , ALTER DATABASE , and DROP DATABASE SQL commands of PostgreSQL. Field Description cluster [Required] core/v1.LocalObjectReference The name of the PostgreSQL cluster hosting the database. ensure EnsureOption Ensure the PostgreSQL database is present or absent - defaults to \"present\". name [Required] string The name of the database to create inside PostgreSQL. This setting cannot be changed. owner [Required] string Maps to the OWNER parameter of CREATE DATABASE . Maps to the OWNER TO command of ALTER DATABASE . The role name of the user who owns the database inside PostgreSQL. template string Maps to the TEMPLATE parameter of CREATE DATABASE . This setting cannot be changed. The name of the template from which to create this database. encoding string Maps to the ENCODING parameter of CREATE DATABASE . This setting cannot be changed. Character set encoding to use in the database. locale string Maps to the LOCALE parameter of CREATE DATABASE . This setting cannot be changed. Sets the default collation order and character classification in the new database. localeProvider string Maps to the LOCALE_PROVIDER parameter of CREATE DATABASE . This setting cannot be changed. This option sets the locale provider for databases created in the new cluster. Available from PostgreSQL 16. localeCollate string Maps to the LC_COLLATE parameter of CREATE DATABASE . This setting cannot be changed. localeCType string Maps to the LC_CTYPE parameter of CREATE DATABASE . This setting cannot be changed. icuLocale string Maps to the ICU_LOCALE parameter of CREATE DATABASE . This setting cannot be changed. Specifies the ICU locale when the ICU provider is used. This option requires localeProvider to be set to icu . Available from PostgreSQL 15. icuRules string Maps to the ICU_RULES parameter of CREATE DATABASE . This setting cannot be changed. Specifies additional collation rules to customize the behavior of the default collation. This option requires localeProvider to be set to icu . Available from PostgreSQL 16. builtinLocale string Maps to the BUILTIN_LOCALE parameter of CREATE DATABASE . This setting cannot be changed. Specifies the locale name when the builtin provider is used. This option requires localeProvider to be set to builtin . Available from PostgreSQL 17. collationVersion string Maps to the COLLATION_VERSION parameter of CREATE DATABASE . This setting cannot be changed. isTemplate bool Maps to the IS_TEMPLATE parameter of CREATE DATABASE and ALTER DATABASE . If true, this database is considered a template and can be cloned by any user with CREATEDB privileges. allowConnections bool Maps to the ALLOW_CONNECTIONS parameter of CREATE DATABASE and ALTER DATABASE . If false then no one can connect to this database. connectionLimit int Maps to the CONNECTION LIMIT clause of CREATE DATABASE and ALTER DATABASE . How many concurrent connections can be made to this database. -1 (the default) means no limit. tablespace string Maps to the TABLESPACE parameter of CREATE DATABASE . Maps to the SET TABLESPACE command of ALTER DATABASE . The name of the tablespace (in PostgreSQL) that will be associated with the new database. This tablespace will be the default tablespace used for objects created in this database. databaseReclaimPolicy DatabaseReclaimPolicy The policy for end-of-life maintenance of this database. DatabaseStatus Appears in: Database DatabaseStatus defines the observed state of Database Field Description observedGeneration int64 A sequence number representing the latest desired state that was synchronized applied bool Applied is true if the database was reconciled correctly message string Message is the reconciliation output message EmbeddedObjectMetadata Appears in: ClusterSpec EmbeddedObjectMetadata contains metadata to be inherited by all resources related to a Cluster Field Description labels map[string]string No description provided. annotations map[string]string No description provided. EnsureOption (Alias of string ) Appears in: DatabaseSpec RoleConfiguration EnsureOption represents whether we should enforce the presence or absence of a Role in a PostgreSQL instance EphemeralVolumesSizeLimitConfiguration Appears in: ClusterSpec EphemeralVolumesSizeLimitConfiguration contains the configuration of the ephemeral storage Field Description shm k8s.io/apimachinery/pkg/api/resource.Quantity Shm is the size limit of the shared memory volume temporaryData k8s.io/apimachinery/pkg/api/resource.Quantity TemporaryData is the size limit of the temporary data volume ExternalCluster Appears in: ClusterSpec ExternalCluster represents the connection parameters to an external cluster which is used in the other sections of the configuration Field Description name [Required] string The server name, required connectionParameters map[string]string The list of connection parameters, such as dbname, host, username, etc sslCert core/v1.SecretKeySelector The reference to an SSL certificate to be used to connect to this instance sslKey core/v1.SecretKeySelector The reference to an SSL private key to be used to connect to this instance sslRootCert core/v1.SecretKeySelector The reference to an SSL CA public key to be used to connect to this instance password core/v1.SecretKeySelector The reference to the password to be used to connect to the server. If a password is provided, CloudNativePG creates a PostgreSQL passfile at /controller/external/NAME/pass (where \"NAME\" is the cluster's name). This passfile is automatically referenced in the connection string when establishing a connection to the remote PostgreSQL server from the current PostgreSQL Cluster . This ensures secure and efficient password management for external clusters. barmanObjectStore github.com/cloudnative-pg/barman-cloud/pkg/api.BarmanObjectStoreConfiguration The configuration for the barman-cloud tool suite plugin [Required] PluginConfiguration The configuration of the plugin that is taking care of WAL archiving and backups for this external cluster ImageCatalogRef Appears in: ClusterSpec ImageCatalogRef defines the reference to a major version in an ImageCatalog Field Description TypedLocalObjectReference core/v1.TypedLocalObjectReference (Members of TypedLocalObjectReference are embedded into this type.) No description provided. major [Required] int The major version of PostgreSQL we want to use from the ImageCatalog ImageCatalogSpec Appears in: ClusterImageCatalog ImageCatalog ImageCatalogSpec defines the desired ImageCatalog Field Description images [Required] []CatalogImage List of CatalogImages available in the catalog Import Appears in: BootstrapInitDB Import contains the configuration to init a database from a logic snapshot of an externalCluster Field Description source [Required] ImportSource The source of the import type [Required] SnapshotType The import type. Can be microservice or monolith . databases [Required] []string The databases to import roles []string The roles to import postImportApplicationSQL []string List of SQL queries to be executed as a superuser in the application database right after is imported - to be used with extreme care (by default empty). Only available in microservice type. schemaOnly bool When set to true, only the pre-data and post-data sections of pg_restore are invoked, avoiding data import. Default: false . pgDumpExtraOptions []string List of custom options to pass to the pg_dump command. IMPORTANT: Use these options with caution and at your own risk, as the operator does not validate their content. Be aware that certain options may conflict with the operator's intended functionality or design. pgRestoreExtraOptions []string List of custom options to pass to the pg_restore command. IMPORTANT: Use these options with caution and at your own risk, as the operator does not validate their content. Be aware that certain options may conflict with the operator's intended functionality or design. ImportSource Appears in: Import ImportSource describes the source for the logical snapshot Field Description externalCluster [Required] string The name of the externalCluster used for import InstanceID Appears in: BackupStatus InstanceID contains the information to identify an instance Field Description podName string The pod name ContainerID string The container ID InstanceReportedState Appears in: ClusterStatus InstanceReportedState describes the last reported state of an instance during a reconciliation loop Field Description isPrimary [Required] bool indicates if an instance is the primary one timeLineID int indicates on which TimelineId the instance is LDAPBindAsAuth Appears in: LDAPConfig LDAPBindAsAuth provides the required fields to use the bind authentication for LDAP Field Description prefix string Prefix for the bind authentication option suffix string Suffix for the bind authentication option LDAPBindSearchAuth Appears in: LDAPConfig LDAPBindSearchAuth provides the required fields to use the bind+search LDAP authentication process Field Description baseDN string Root DN to begin the user search bindDN string DN of the user to bind to the directory bindPassword core/v1.SecretKeySelector Secret with the password for the user to bind to the directory searchAttribute string Attribute to match against the username searchFilter string Search filter to use when doing the search+bind authentication LDAPConfig Appears in: PostgresConfiguration LDAPConfig contains the parameters needed for LDAP authentication Field Description server string LDAP hostname or IP address port int LDAP server port scheme LDAPScheme LDAP schema to be used, possible options are ldap and ldaps bindAsAuth LDAPBindAsAuth Bind as authentication configuration bindSearchAuth LDAPBindSearchAuth Bind+Search authentication configuration tls bool Set to 'true' to enable LDAP over TLS. 'false' is default LDAPScheme (Alias of string ) Appears in: LDAPConfig LDAPScheme defines the possible schemes for LDAP ManagedConfiguration Appears in: ClusterSpec ManagedConfiguration represents the portions of PostgreSQL that are managed by the instance manager Field Description roles []RoleConfiguration Database roles managed by the Cluster services ManagedServices Services roles managed by the Cluster ManagedRoles Appears in: ClusterStatus ManagedRoles tracks the status of a cluster's managed roles Field Description byStatus map[RoleStatus][]string ByStatus gives the list of roles in each state cannotReconcile map[string][]string CannotReconcile lists roles that cannot be reconciled in PostgreSQL, with an explanation of the cause passwordStatus map[string]PasswordState PasswordStatus gives the last transaction id and password secret version for each managed role ManagedService Appears in: ManagedServices ManagedService represents a specific service managed by the cluster. It includes the type of service and its associated template specification. Field Description selectorType [Required] ServiceSelectorType SelectorType specifies the type of selectors that the service will have. Valid values are \"rw\", \"r\", and \"ro\", representing read-write, read, and read-only services. updateStrategy ServiceUpdateStrategy UpdateStrategy describes how the service differences should be reconciled serviceTemplate [Required] ServiceTemplateSpec ServiceTemplate is the template specification for the service. ManagedServices Appears in: ManagedConfiguration ManagedServices represents the services managed by the cluster. Field Description disabledDefaultServices []ServiceSelectorType DisabledDefaultServices is a list of service types that are disabled by default. Valid values are \"r\", and \"ro\", representing read, and read-only services. additional []ManagedService Additional is a list of additional managed services specified by the user. Metadata Appears in: PodTemplateSpec ServiceAccountTemplate ServiceTemplateSpec Metadata is a structure similar to the metav1.ObjectMeta, but still parseable by controller-gen to create a suitable CRD for the user. The comment of PodTemplateSpec has an explanation of why we are not using the core data types. Field Description name string The name of the resource. Only supported for certain types labels map[string]string Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels annotations map[string]string Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations MonitoringConfiguration Appears in: ClusterSpec MonitoringConfiguration is the type containing all the monitoring configuration for a certain cluster Field Description disableDefaultQueries bool Whether the default queries should be injected. Set it to true if you don't want to inject default queries into the cluster. Default: false. customQueriesConfigMap []github.com/cloudnative-pg/machinery/pkg/api.ConfigMapKeySelector The list of config maps containing the custom queries customQueriesSecret []github.com/cloudnative-pg/machinery/pkg/api.SecretKeySelector The list of secrets containing the custom queries enablePodMonitor bool Enable or disable the PodMonitor tls ClusterMonitoringTLSConfiguration Configure TLS communication for the metrics endpoint. Changing tls.enabled option will force a rollout of all instances. podMonitorMetricRelabelings []github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1.RelabelConfig The list of metric relabelings for the PodMonitor . Applied to samples before ingestion. podMonitorRelabelings []github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1.RelabelConfig The list of relabelings for the PodMonitor . Applied to samples before scraping. NodeMaintenanceWindow Appears in: ClusterSpec NodeMaintenanceWindow contains information that the operator will use while upgrading the underlying node. This option is only useful when the chosen storage prevents the Pods from being freely moved across nodes. Field Description reusePVC bool Reuse the existing PVC (wait for the node to come up again) or not (recreate it elsewhere - when instances >1) inProgress bool Is there a node maintenance activity in progress? OnlineConfiguration Appears in: BackupSpec ScheduledBackupSpec VolumeSnapshotConfiguration OnlineConfiguration contains the configuration parameters for the online volume snapshot Field Description waitForArchive bool If false, the function will return immediately after the backup is completed, without waiting for WAL to be archived. This behavior is only useful with backup software that independently monitors WAL archiving. Otherwise, WAL required to make the backup consistent might be missing and make the backup useless. By default, or when this parameter is true, pg_backup_stop will wait for WAL to be archived when archiving is enabled. On a standby, this means that it will wait only when archive_mode = always. If write activity on the primary is low, it may be useful to run pg_switch_wal on the primary in order to trigger an immediate segment switch. immediateCheckpoint bool Control whether the I/O workload for the backup initial checkpoint will be limited, according to the checkpoint_completion_target setting on the PostgreSQL server. If set to true, an immediate checkpoint will be used, meaning PostgreSQL will complete the checkpoint as soon as possible. false by default. PasswordState Appears in: ManagedRoles PasswordState represents the state of the password of a managed RoleConfiguration Field Description transactionID int64 the last transaction ID to affect the role definition in PostgreSQL resourceVersion string the resource version of the password secret PgBouncerIntegrationStatus Appears in: PoolerIntegrations PgBouncerIntegrationStatus encapsulates the needed integration for the pgbouncer poolers referencing the cluster Field Description secrets []string No description provided. PgBouncerPoolMode (Alias of string ) Appears in: PgBouncerSpec PgBouncerPoolMode is the mode of PgBouncer PgBouncerSecrets Appears in: PoolerSecrets PgBouncerSecrets contains the versions of the secrets used by pgbouncer Field Description authQuery SecretVersion The auth query secret version PgBouncerSpec Appears in: PoolerSpec PgBouncerSpec defines how to configure PgBouncer Field Description poolMode PgBouncerPoolMode The pool mode. Default: session . authQuerySecret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The credentials of the user that need to be used for the authentication query. In case it is specified, also an AuthQuery (e.g. \"SELECT usename, passwd FROM pg_catalog.pg_shadow WHERE usename=$1\") has to be specified and no automatic CNPG Cluster integration will be triggered. authQuery string The query that will be used to download the hash of the password of a certain user. Default: \"SELECT usename, passwd FROM public.user_search($1)\". In case it is specified, also an AuthQuerySecret has to be specified and no automatic CNPG Cluster integration will be triggered. parameters map[string]string Additional parameters to be passed to PgBouncer - please check the CNPG documentation for a list of options you can configure pg_hba []string PostgreSQL Host Based Authentication rules (lines to be appended to the pg_hba.conf file) paused bool When set to true , PgBouncer will disconnect from the PostgreSQL server, first waiting for all queries to complete, and pause all new client connections until this value is set to false (default). Internally, the operator calls PgBouncer's PAUSE and RESUME commands. PluginConfiguration Appears in: ClusterSpec ExternalCluster PluginConfiguration specifies a plugin that need to be loaded for this cluster to be reconciled Field Description name [Required] string Name is the plugin name enabled bool Enabled is true if this plugin will be used isWALArchiver bool Only one plugin can be declared as WALArchiver. Cannot be active if \".spec.backup.barmanObjectStore\" configuration is present. parameters map[string]string Parameters is the configuration of the plugin PluginStatus Appears in: ClusterStatus PluginStatus is the status of a loaded plugin Field Description name [Required] string Name is the name of the plugin version [Required] string Version is the version of the plugin loaded by the latest reconciliation loop capabilities []string Capabilities are the list of capabilities of the plugin operatorCapabilities []string OperatorCapabilities are the list of capabilities of the plugin regarding the reconciler walCapabilities []string WALCapabilities are the list of capabilities of the plugin regarding the WAL management backupCapabilities []string BackupCapabilities are the list of capabilities of the plugin regarding the Backup management restoreJobHookCapabilities []string RestoreJobHookCapabilities are the list of capabilities of the plugin regarding the RestoreJobHook management status string Status contain the status reported by the plugin through the SetStatusInCluster interface PodTemplateSpec Appears in: PoolerSpec PodTemplateSpec is a structure allowing the user to set a template for Pod generation. Unfortunately we can't use the corev1.PodTemplateSpec type because the generated CRD won't have the field for the metadata section. References: https://github.com/kubernetes-sigs/controller-tools/issues/385 https://github.com/kubernetes-sigs/controller-tools/issues/448 https://github.com/prometheus-operator/prometheus-operator/issues/3041 Field Description metadata Metadata Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata spec core/v1.PodSpec Specification of the desired behavior of the pod. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status PodTopologyLabels (Alias of map[string]string ) Appears in: Topology PodTopologyLabels represent the topology of a Pod. map[labelName]labelValue PoolerIntegrations Appears in: ClusterStatus PoolerIntegrations encapsulates the needed integration for the poolers referencing the cluster Field Description pgBouncerIntegration PgBouncerIntegrationStatus No description provided. PoolerMonitoringConfiguration Appears in: PoolerSpec PoolerMonitoringConfiguration is the type containing all the monitoring configuration for a certain Pooler. Mirrors the Cluster's MonitoringConfiguration but without the custom queries part for now. Field Description enablePodMonitor bool Enable or disable the PodMonitor podMonitorMetricRelabelings []github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1.RelabelConfig The list of metric relabelings for the PodMonitor . Applied to samples before ingestion. podMonitorRelabelings []github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1.RelabelConfig The list of relabelings for the PodMonitor . Applied to samples before scraping. PoolerSecrets Appears in: PoolerStatus PoolerSecrets contains the versions of all the secrets used Field Description serverTLS SecretVersion The server TLS secret version serverCA SecretVersion The server CA secret version clientCA SecretVersion The client CA secret version pgBouncerSecrets PgBouncerSecrets The version of the secrets used by PgBouncer PoolerSpec Appears in: Pooler PoolerSpec defines the desired state of Pooler Field Description cluster [Required] github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference This is the cluster reference on which the Pooler will work. Pooler name should never match with any cluster name within the same namespace. type PoolerType Type of service to forward traffic to. Default: rw . instances int32 The number of replicas we want. Default: 1. template PodTemplateSpec The template of the Pod to be created pgbouncer [Required] PgBouncerSpec The PgBouncer configuration deploymentStrategy apps/v1.DeploymentStrategy The deployment strategy to use for pgbouncer to replace existing pods with new ones monitoring PoolerMonitoringConfiguration The configuration of the monitoring infrastructure of this pooler. serviceTemplate ServiceTemplateSpec Template for the Service to be created PoolerStatus Appears in: Pooler PoolerStatus defines the observed state of Pooler Field Description secrets PoolerSecrets The resource version of the config object instances int32 The number of pods trying to be scheduled PoolerType (Alias of string ) Appears in: PoolerSpec PoolerType is the type of the connection pool, meaning the service we are targeting. Allowed values are rw and ro . PostgresConfiguration Appears in: ClusterSpec PostgresConfiguration defines the PostgreSQL configuration Field Description parameters map[string]string PostgreSQL configuration options (postgresql.conf) synchronous SynchronousReplicaConfiguration Configuration of the PostgreSQL synchronous replication feature pg_hba []string PostgreSQL Host Based Authentication rules (lines to be appended to the pg_hba.conf file) pg_ident []string PostgreSQL User Name Maps rules (lines to be appended to the pg_ident.conf file) syncReplicaElectionConstraint SyncReplicaElectionConstraints Requirements to be met by sync replicas. This will affect how the \"synchronous_standby_names\" parameter will be set up. shared_preload_libraries []string Lists of shared preload libraries to add to the default ones ldap LDAPConfig Options to specify LDAP configuration promotionTimeout int32 Specifies the maximum number of seconds to wait when promoting an instance to primary. Default value is 40000000, greater than one year in seconds, big enough to simulate an infinite timeout enableAlterSystem bool If this parameter is true, the user will be able to invoke ALTER SYSTEM on this CloudNativePG Cluster. This should only be used for debugging and troubleshooting. Defaults to false. PrimaryUpdateMethod (Alias of string ) Appears in: ClusterSpec PrimaryUpdateMethod contains the method to use when upgrading the primary server of the cluster as part of rolling updates PrimaryUpdateStrategy (Alias of string ) Appears in: ClusterSpec PrimaryUpdateStrategy contains the strategy to follow when upgrading the primary server of the cluster as part of rolling updates Probe Appears in: ProbesConfiguration Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic. Field Description initialDelaySeconds int32 Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes timeoutSeconds int32 Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes periodSeconds int32 How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. successThreshold int32 Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1. failureThreshold int32 Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. terminationGracePeriodSeconds int64 Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is a beta field and requires enabling ProbeTerminationGracePeriod feature gate. Minimum value is 1. spec.terminationGracePeriodSeconds is used if unset. ProbesConfiguration Appears in: ClusterSpec ProbesConfiguration represent the configuration for the probes to be injected in the PostgreSQL Pods Field Description startup [Required] Probe The startup probe configuration liveness [Required] Probe The liveness probe configuration readiness [Required] Probe The readiness probe configuration PublicationReclaimPolicy (Alias of string ) Appears in: PublicationSpec PublicationReclaimPolicy defines a policy for end-of-life maintenance of Publications. PublicationSpec Appears in: Publication PublicationSpec defines the desired state of Publication Field Description cluster [Required] core/v1.LocalObjectReference The name of the PostgreSQL cluster that identifies the \"publisher\" name [Required] string The name of the publication inside PostgreSQL dbname [Required] string The name of the database where the publication will be installed in the \"publisher\" cluster parameters map[string]string Publication parameters part of the WITH clause as expected by PostgreSQL CREATE PUBLICATION command target [Required] PublicationTarget Target of the publication as expected by PostgreSQL CREATE PUBLICATION command publicationReclaimPolicy PublicationReclaimPolicy The policy for end-of-life maintenance of this publication PublicationStatus Appears in: Publication PublicationStatus defines the observed state of Publication Field Description observedGeneration int64 A sequence number representing the latest desired state that was synchronized applied bool Applied is true if the publication was reconciled correctly message string Message is the reconciliation output message PublicationTarget Appears in: PublicationSpec PublicationTarget is what this publication should publish Field Description allTables bool Marks the publication as one that replicates changes for all tables in the database, including tables created in the future. Corresponding to FOR ALL TABLES in PostgreSQL. objects []PublicationTargetObject Just the following schema objects PublicationTargetObject Appears in: PublicationTarget PublicationTargetObject is an object to publish Field Description tablesInSchema string Marks the publication as one that replicates changes for all tables in the specified list of schemas, including tables created in the future. Corresponding to FOR TABLES IN SCHEMA in PostgreSQL. table PublicationTargetTable Specifies a list of tables to add to the publication. Corresponding to FOR TABLE in PostgreSQL. PublicationTargetTable Appears in: PublicationTargetObject PublicationTargetTable is a table to publish Field Description only bool Whether to limit to the table only or include all its descendants name [Required] string The table name schema string The schema name columns []string The columns to publish RecoveryTarget Appears in: BootstrapRecovery RecoveryTarget allows to configure the moment where the recovery process will stop. All the target options except TargetTLI are mutually exclusive. Field Description backupID string The ID of the backup from which to start the recovery process. If empty (default) the operator will automatically detect the backup based on targetTime or targetLSN if specified. Otherwise use the latest available backup in chronological order. targetTLI string The target timeline (\"latest\" or a positive integer) targetXID string The target transaction ID targetName string The target name (to be previously created with pg_create_restore_point ) targetLSN string The target LSN (Log Sequence Number) targetTime string The target time as a timestamp in the RFC3339 standard targetImmediate bool End recovery as soon as a consistent state is reached exclusive bool Set the target to be exclusive. If omitted, defaults to false, so that in Postgres, recovery_target_inclusive will be true ReplicaClusterConfiguration Appears in: ClusterSpec ReplicaClusterConfiguration encapsulates the configuration of a replica cluster Field Description self string Self defines the name of this cluster. It is used to determine if this is a primary or a replica cluster, comparing it with primary primary string Primary defines which Cluster is defined to be the primary in the distributed PostgreSQL cluster, based on the topology specified in externalClusters source [Required] string The name of the external cluster which is the replication origin enabled bool If replica mode is enabled, this cluster will be a replica of an existing cluster. Replica cluster can be created from a recovery object store or via streaming through pg_basebackup. Refer to the Replica clusters page of the documentation for more information. promotionToken string A demotion token generated by an external cluster used to check if the promotion requirements are met. minApplyDelay meta/v1.Duration When replica mode is enabled, this parameter allows you to replay transactions only when the system time is at least the configured time past the commit time. This provides an opportunity to correct data loss errors. Note that when this parameter is set, a promotion token cannot be used. ReplicationSlotsConfiguration Appears in: ClusterSpec ReplicationSlotsConfiguration encapsulates the configuration of replication slots Field Description highAvailability ReplicationSlotsHAConfiguration Replication slots for high availability configuration updateInterval int Standby will update the status of the local replication slots every updateInterval seconds (default 30). synchronizeReplicas SynchronizeReplicasConfiguration Configures the synchronization of the user defined physical replication slots ReplicationSlotsHAConfiguration Appears in: ReplicationSlotsConfiguration ReplicationSlotsHAConfiguration encapsulates the configuration of the replication slots that are automatically managed by the operator to control the streaming replication connections with the standby instances for high availability (HA) purposes. Replication slots are a PostgreSQL feature that makes sure that PostgreSQL automatically keeps WAL files in the primary when a streaming client (in this specific case a replica that is part of the HA cluster) gets disconnected. Field Description enabled bool If enabled (default), the operator will automatically manage replication slots on the primary instance and use them in streaming replication connections with all the standby instances that are part of the HA cluster. If disabled, the operator will not take advantage of replication slots in streaming connections with the replicas. This feature also controls replication slots in replica cluster, from the designated primary to its cascading replicas. slotPrefix string Prefix for replication slots managed by the operator for HA. It may only contain lower case letters, numbers, and the underscore character. This can only be set at creation time. By default set to _cnpg_ . RoleConfiguration Appears in: ManagedConfiguration RoleConfiguration is the representation, in Kubernetes, of a PostgreSQL role with the additional field Ensure specifying whether to ensure the presence or absence of the role in the database The defaults of the CREATE ROLE command are applied Reference: https://www.postgresql.org/docs/current/sql-createrole.html Field Description name [Required] string Name of the role comment string Description of the role ensure EnsureOption Ensure the role is present or absent - defaults to \"present\" passwordSecret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference Secret containing the password of the role (if present) If null, the password will be ignored unless DisablePassword is set connectionLimit int64 If the role can log in, this specifies how many concurrent connections the role can make. -1 (the default) means no limit. validUntil meta/v1.Time Date and time after which the role's password is no longer valid. When omitted, the password will never expire (default). inRoles []string List of one or more existing roles to which this role will be immediately added as a new member. Default empty. inherit bool Whether a role \"inherits\" the privileges of roles it is a member of. Defaults is true . disablePassword bool DisablePassword indicates that a role's password should be set to NULL in Postgres superuser bool Whether the role is a superuser who can override all access restrictions within the database - superuser status is dangerous and should be used only when really needed. You must yourself be a superuser to create a new superuser. Defaults is false . createdb bool When set to true , the role being defined will be allowed to create new databases. Specifying false (default) will deny a role the ability to create databases. createrole bool Whether the role will be permitted to create, alter, drop, comment on, change the security label for, and grant or revoke membership in other roles. Default is false . login bool Whether the role is allowed to log in. A role having the login attribute can be thought of as a user. Roles without this attribute are useful for managing database privileges, but are not users in the usual sense of the word. Default is false . replication bool Whether a role is a replication role. A role must have this attribute (or be a superuser) in order to be able to connect to the server in replication mode (physical or logical replication) and in order to be able to create or drop replication slots. A role having the replication attribute is a very highly privileged role, and should only be used on roles actually used for replication. Default is false . bypassrls bool Whether a role bypasses every row-level security (RLS) policy. Default is false . SQLRefs Appears in: BootstrapInitDB SQLRefs holds references to ConfigMaps or Secrets containing SQL files. The references are processed in a specific order: first, all Secrets are processed, followed by all ConfigMaps. Within each group, the processing order follows the sequence specified in their respective arrays. Field Description secretRefs []github.com/cloudnative-pg/machinery/pkg/api.SecretKeySelector SecretRefs holds a list of references to Secrets configMapRefs []github.com/cloudnative-pg/machinery/pkg/api.ConfigMapKeySelector ConfigMapRefs holds a list of references to ConfigMaps ScheduledBackupSpec Appears in: ScheduledBackup ScheduledBackupSpec defines the desired state of ScheduledBackup Field Description suspend bool If this backup is suspended or not immediate bool If the first backup has to be immediately start after creation or not schedule [Required] string The schedule does not follow the same format used in Kubernetes CronJobs as it includes an additional seconds specifier, see https://pkg.go.dev/github.com/robfig/cron#hdr-CRON_Expression_Format cluster [Required] github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The cluster to backup backupOwnerReference string Indicates which ownerReference should be put inside the created backup resources. none: no owner reference for created backup objects (same behavior as before the field was introduced) self: sets the Scheduled backup object as owner of the backup cluster: set the cluster as owner of the backup target BackupTarget The policy to decide which instance should perform this backup. If empty, it defaults to cluster.spec.backup.target . Available options are empty string, primary and prefer-standby . primary to have backups run always on primary instances, prefer-standby to have backups run preferably on the most updated standby, if available. method BackupMethod The backup method to be used, possible options are barmanObjectStore , volumeSnapshot or plugin . Defaults to: barmanObjectStore . pluginConfiguration BackupPluginConfiguration Configuration parameters passed to the plugin managing this backup online bool Whether the default type of backup with volume snapshots is online/hot ( true , default) or offline/cold ( false ) Overrides the default setting specified in the cluster field '.spec.backup.volumeSnapshot.online' onlineConfiguration OnlineConfiguration Configuration parameters to control the online/hot backup with volume snapshots Overrides the default settings specified in the cluster '.backup.volumeSnapshot.onlineConfiguration' stanza ScheduledBackupStatus Appears in: ScheduledBackup ScheduledBackupStatus defines the observed state of ScheduledBackup Field Description lastCheckTime meta/v1.Time The latest time the schedule lastScheduleTime meta/v1.Time Information when was the last time that backup was successfully scheduled. nextScheduleTime meta/v1.Time Next time we will run a backup SecretVersion Appears in: PgBouncerSecrets PoolerSecrets SecretVersion contains a secret name and its ResourceVersion Field Description name string The name of the secret version string The ResourceVersion of the secret SecretsResourceVersion Appears in: ClusterStatus SecretsResourceVersion is the resource versions of the secrets managed by the operator Field Description superuserSecretVersion string The resource version of the \"postgres\" user secret replicationSecretVersion string The resource version of the \"streaming_replica\" user secret applicationSecretVersion string The resource version of the \"app\" user secret managedRoleSecretVersion map[string]string The resource versions of the managed roles secrets caSecretVersion string Unused. Retained for compatibility with old versions. clientCaSecretVersion string The resource version of the PostgreSQL client-side CA secret version serverCaSecretVersion string The resource version of the PostgreSQL server-side CA secret version serverSecretVersion string The resource version of the PostgreSQL server-side secret version barmanEndpointCA string The resource version of the Barman Endpoint CA if provided externalClusterSecretVersion map[string]string The resource versions of the external cluster secrets metrics map[string]string A map with the versions of all the secrets used to pass metrics. Map keys are the secret names, map values are the versions ServiceAccountTemplate Appears in: ClusterSpec ServiceAccountTemplate contains the template needed to generate the service accounts Field Description metadata [Required] Metadata Metadata are the metadata to be used for the generated service account ServiceSelectorType (Alias of string ) Appears in: ManagedService ManagedServices ServiceSelectorType describes a valid value for generating the service selectors. It indicates which type of service the selector applies to, such as read-write, read, or read-only ServiceTemplateSpec Appears in: ManagedService PoolerSpec ServiceTemplateSpec is a structure allowing the user to set a template for Service generation. Field Description metadata Metadata Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata spec core/v1.ServiceSpec Specification of the desired behavior of the service. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status ServiceUpdateStrategy (Alias of string ) Appears in: ManagedService ServiceUpdateStrategy describes how the changes to the managed service should be handled SnapshotOwnerReference (Alias of string ) Appears in: VolumeSnapshotConfiguration SnapshotOwnerReference defines the reference type for the owner of the snapshot. This specifies which owner the processed resources should relate to. SnapshotType (Alias of string ) Appears in: Import SnapshotType is a type of allowed import StorageConfiguration Appears in: ClusterSpec TablespaceConfiguration StorageConfiguration is the configuration used to create and reconcile PVCs, usable for WAL volumes, PGDATA volumes, or tablespaces Field Description storageClass string StorageClass to use for PVCs. Applied after evaluating the PVC template, if available. If not specified, the generated PVCs will use the default storage class size string Size of the storage. Required if not already specified in the PVC template. Changes to this field are automatically reapplied to the created PVCs. Size cannot be decreased. resizeInUseVolumes bool Resize existent PVCs, defaults to true pvcTemplate core/v1.PersistentVolumeClaimSpec Template to be used to generate the Persistent Volume Claim SubscriptionReclaimPolicy (Alias of string ) Appears in: SubscriptionSpec SubscriptionReclaimPolicy describes a policy for end-of-life maintenance of Subscriptions. SubscriptionSpec Appears in: Subscription SubscriptionSpec defines the desired state of Subscription Field Description cluster [Required] core/v1.LocalObjectReference The name of the PostgreSQL cluster that identifies the \"subscriber\" name [Required] string The name of the subscription inside PostgreSQL dbname [Required] string The name of the database where the publication will be installed in the \"subscriber\" cluster parameters map[string]string Subscription parameters included in the WITH clause of the PostgreSQL CREATE SUBSCRIPTION command. Most parameters cannot be changed after the subscription is created and will be ignored if modified later, except for a limited set documented at: https://www.postgresql.org/docs/current/sql-altersubscription.html#SQL-ALTERSUBSCRIPTION-PARAMS-SET publicationName [Required] string The name of the publication inside the PostgreSQL database in the \"publisher\" publicationDBName string The name of the database containing the publication on the external cluster. Defaults to the one in the external cluster definition. externalClusterName [Required] string The name of the external cluster with the publication (\"publisher\") subscriptionReclaimPolicy SubscriptionReclaimPolicy The policy for end-of-life maintenance of this subscription SubscriptionStatus Appears in: Subscription SubscriptionStatus defines the observed state of Subscription Field Description observedGeneration int64 A sequence number representing the latest desired state that was synchronized applied bool Applied is true if the subscription was reconciled correctly message string Message is the reconciliation output message SwitchReplicaClusterStatus Appears in: ClusterStatus SwitchReplicaClusterStatus contains all the statuses regarding the switch of a cluster to a replica cluster Field Description inProgress bool InProgress indicates if there is an ongoing procedure of switching a cluster to a replica cluster. SyncReplicaElectionConstraints Appears in: PostgresConfiguration SyncReplicaElectionConstraints contains the constraints for sync replicas election. For anti-affinity parameters two instances are considered in the same location if all the labels values match. In future synchronous replica election restriction by name will be supported. Field Description nodeLabelsAntiAffinity []string A list of node labels values to extract and compare to evaluate if the pods reside in the same topology or not enabled [Required] bool This flag enables the constraints for sync replicas SynchronizeReplicasConfiguration Appears in: ReplicationSlotsConfiguration SynchronizeReplicasConfiguration contains the configuration for the synchronization of user defined physical replication slots Field Description enabled [Required] bool When set to true, every replication slot that is on the primary is synchronized on each standby excludePatterns []string List of regular expression patterns to match the names of replication slots to be excluded (by default empty) SynchronousReplicaConfiguration Appears in: PostgresConfiguration SynchronousReplicaConfiguration contains the configuration of the PostgreSQL synchronous replication feature. Important: at this moment, also .spec.minSyncReplicas and .spec.maxSyncReplicas need to be considered. Field Description method [Required] SynchronousReplicaConfigurationMethod Method to select synchronous replication standbys from the listed servers, accepting 'any' (quorum-based synchronous replication) or 'first' (priority-based synchronous replication) as values. number [Required] int Specifies the number of synchronous standby servers that transactions must wait for responses from. maxStandbyNamesFromCluster int Specifies the maximum number of local cluster pods that can be automatically included in the synchronous_standby_names option in PostgreSQL. standbyNamesPre []string A user-defined list of application names to be added to synchronous_standby_names before local cluster pods (the order is only useful for priority-based synchronous replication). standbyNamesPost []string A user-defined list of application names to be added to synchronous_standby_names after local cluster pods (the order is only useful for priority-based synchronous replication). dataDurability DataDurabilityLevel If set to \"required\", data durability is strictly enforced. Write operations with synchronous commit settings ( on , remote_write , or remote_apply ) will block if there are insufficient healthy replicas, ensuring data persistence. If set to \"preferred\", data durability is maintained when healthy replicas are available, but the required number of instances will adjust dynamically if replicas become unavailable. This setting relaxes strict durability enforcement to allow for operational continuity. This setting is only applicable if both standbyNamesPre and standbyNamesPost are unset (empty). SynchronousReplicaConfigurationMethod (Alias of string ) Appears in: SynchronousReplicaConfiguration SynchronousReplicaConfigurationMethod configures whether to use quorum based replication or a priority list TablespaceConfiguration Appears in: ClusterSpec TablespaceConfiguration is the configuration of a tablespace, and includes the storage specification for the tablespace Field Description name [Required] string The name of the tablespace storage [Required] StorageConfiguration The storage configuration for the tablespace owner DatabaseRoleRef Owner is the PostgreSQL user owning the tablespace temporary bool When set to true, the tablespace will be added as a temp_tablespaces entry in PostgreSQL, and will be available to automatically house temp database objects, or other temporary files. Please refer to PostgreSQL documentation for more information on the temp_tablespaces GUC. TablespaceState Appears in: ClusterStatus TablespaceState represents the state of a tablespace in a cluster Field Description name [Required] string Name is the name of the tablespace owner string Owner is the PostgreSQL user owning the tablespace state [Required] TablespaceStatus State is the latest reconciliation state error string Error is the reconciliation error, if any TablespaceStatus (Alias of string ) Appears in: TablespaceState TablespaceStatus represents the status of a tablespace in the cluster Topology Appears in: ClusterStatus Topology contains the cluster topology Field Description instances map[PodName]PodTopologyLabels Instances contains the pod topology of the instances nodesUsed int32 NodesUsed represents the count of distinct nodes accommodating the instances. A value of '1' suggests that all instances are hosted on a single node, implying the absence of High Availability (HA). Ideally, this value should be the same as the number of instances in the Postgres HA cluster, implying shared nothing architecture on the compute side. successfullyExtracted bool SuccessfullyExtracted indicates if the topology data was extract. It is useful to enact fallback behaviors in synchronous replica election in case of failures VolumeSnapshotConfiguration Appears in: BackupConfiguration VolumeSnapshotConfiguration represents the configuration for the execution of snapshot backups. Field Description labels map[string]string Labels are key-value pairs that will be added to .metadata.labels snapshot resources. annotations map[string]string Annotations key-value pairs that will be added to .metadata.annotations snapshot resources. className string ClassName specifies the Snapshot Class to be used for PG_DATA PersistentVolumeClaim. It is the default class for the other types if no specific class is present walClassName string WalClassName specifies the Snapshot Class to be used for the PG_WAL PersistentVolumeClaim. tablespaceClassName map[string]string TablespaceClassName specifies the Snapshot Class to be used for the tablespaces. defaults to the PGDATA Snapshot Class, if set snapshotOwnerReference SnapshotOwnerReference SnapshotOwnerReference indicates the type of owner reference the snapshot should have online bool Whether the default type of backup with volume snapshots is online/hot ( true , default) or offline/cold ( false ) onlineConfiguration OnlineConfiguration Configuration parameters to control the online/hot backup with volume snapshots","title":"API Reference"},{"location":"cloudnative-pg.v1/#api-reference","text":"Package v1 contains API Schema definitions for the postgresql v1 API group","title":"API Reference"},{"location":"cloudnative-pg.v1/#resource-types","text":"Backup Cluster ClusterImageCatalog Database ImageCatalog Pooler Publication ScheduledBackup Subscription","title":"Resource Types"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Backup","text":"A Backup resource is a request for a PostgreSQL backup by the user. Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string Backup metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] BackupSpec Specification of the desired behavior of the backup. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status status BackupStatus Most recently observed status of the backup. This data may not be up to date. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"Backup"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Cluster","text":"Cluster is the Schema for the PostgreSQL API Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string Cluster metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] ClusterSpec Specification of the desired behavior of the cluster. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status status ClusterStatus Most recently observed status of the cluster. This data may not be up to date. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"Cluster"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ClusterImageCatalog","text":"ClusterImageCatalog is the Schema for the clusterimagecatalogs API Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string ClusterImageCatalog metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] ImageCatalogSpec Specification of the desired behavior of the ClusterImageCatalog. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"ClusterImageCatalog"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Database","text":"Database is the Schema for the databases API Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string Database metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] DatabaseSpec Specification of the desired Database. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status status DatabaseStatus Most recently observed status of the Database. This data may not be up to date. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"Database"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ImageCatalog","text":"ImageCatalog is the Schema for the imagecatalogs API Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string ImageCatalog metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] ImageCatalogSpec Specification of the desired behavior of the ImageCatalog. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"ImageCatalog"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Pooler","text":"Pooler is the Schema for the poolers API Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string Pooler metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] PoolerSpec Specification of the desired behavior of the Pooler. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status status PoolerStatus Most recently observed status of the Pooler. This data may not be up to date. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"Pooler"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Publication","text":"Publication is the Schema for the publications API Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string Publication metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] PublicationSpec No description provided. status [Required] PublicationStatus No description provided.","title":"Publication"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ScheduledBackup","text":"ScheduledBackup is the Schema for the scheduledbackups API Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string ScheduledBackup metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] ScheduledBackupSpec Specification of the desired behavior of the ScheduledBackup. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status status ScheduledBackupStatus Most recently observed status of the ScheduledBackup. This data may not be up to date. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"ScheduledBackup"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Subscription","text":"Subscription is the Schema for the subscriptions API Field Description apiVersion [Required] string postgresql.cnpg.io/v1 kind [Required] string Subscription metadata [Required] meta/v1.ObjectMeta No description provided. Refer to the Kubernetes API documentation for the fields of the metadata field. spec [Required] SubscriptionSpec No description provided. status [Required] SubscriptionStatus No description provided.","title":"Subscription"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-AffinityConfiguration","text":"Appears in: ClusterSpec AffinityConfiguration contains the info we need to create the affinity rules for Pods Field Description enablePodAntiAffinity bool Activates anti-affinity for the pods. The operator will define pods anti-affinity unless this field is explicitly set to false topologyKey string TopologyKey to use for anti-affinity configuration. See k8s documentation for more info on that nodeSelector map[string]string NodeSelector is map of key-value pairs used to define the nodes on which the pods can run. More info: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ nodeAffinity core/v1.NodeAffinity NodeAffinity describes node affinity scheduling rules for the pod. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity tolerations []core/v1.Toleration Tolerations is a list of Tolerations that should be set for all the pods, in order to allow them to run on tainted nodes. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ podAntiAffinityType string PodAntiAffinityType allows the user to decide whether pod anti-affinity between cluster instance has to be considered a strong requirement during scheduling or not. Allowed values are: \"preferred\" (default if empty) or \"required\". Setting it to \"required\", could lead to instances remaining pending until new kubernetes nodes are added if all the existing nodes don't match the required pod anti-affinity rule. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity additionalPodAntiAffinity core/v1.PodAntiAffinity AdditionalPodAntiAffinity allows to specify pod anti-affinity terms to be added to the ones generated by the operator if EnablePodAntiAffinity is set to true (default) or to be used exclusively if set to false. additionalPodAffinity core/v1.PodAffinity AdditionalPodAffinity allows to specify pod affinity terms to be passed to all the cluster's pods.","title":"AffinityConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-AvailableArchitecture","text":"Appears in: ClusterStatus AvailableArchitecture represents the state of a cluster's architecture Field Description goArch [Required] string GoArch is the name of the executable architecture hash [Required] string Hash is the hash of the executable","title":"AvailableArchitecture"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupConfiguration","text":"Appears in: ClusterSpec BackupConfiguration defines how the backup of the cluster are taken. The supported backup methods are BarmanObjectStore and VolumeSnapshot. For details and examples refer to the Backup and Recovery section of the documentation Field Description volumeSnapshot VolumeSnapshotConfiguration VolumeSnapshot provides the configuration for the execution of volume snapshot backups. barmanObjectStore github.com/cloudnative-pg/barman-cloud/pkg/api.BarmanObjectStoreConfiguration The configuration for the barman-cloud tool suite retentionPolicy string RetentionPolicy is the retention policy to be used for backups and WALs (i.e. '60d'). The retention policy is expressed in the form of XXu where XX is a positive integer and u is in [dwm] - days, weeks, months. It's currently only applicable when using the BarmanObjectStore method. target BackupTarget The policy to decide which instance should perform backups. Available options are empty string, which will default to prefer-standby policy, primary to have backups run always on primary instances, prefer-standby to have backups run preferably on the most updated standby, if available.","title":"BackupConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupMethod","text":"(Alias of string ) Appears in: BackupSpec BackupStatus ScheduledBackupSpec BackupMethod defines the way of executing the physical base backups of the selected PostgreSQL instance","title":"BackupMethod"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupPhase","text":"(Alias of string ) Appears in: BackupStatus BackupPhase is the phase of the backup","title":"BackupPhase"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupPluginConfiguration","text":"Appears in: BackupSpec ScheduledBackupSpec BackupPluginConfiguration contains the backup configuration used by the backup plugin Field Description name [Required] string Name is the name of the plugin managing this backup parameters map[string]string Parameters are the configuration parameters passed to the backup plugin for this backup","title":"BackupPluginConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupSnapshotElementStatus","text":"Appears in: BackupSnapshotStatus BackupSnapshotElementStatus is a volume snapshot that is part of a volume snapshot method backup Field Description name [Required] string Name is the snapshot resource name type [Required] string Type is tho role of the snapshot in the cluster, such as PG_DATA, PG_WAL and PG_TABLESPACE tablespaceName string TablespaceName is the name of the snapshotted tablespace. Only set when type is PG_TABLESPACE","title":"BackupSnapshotElementStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupSnapshotStatus","text":"Appears in: BackupStatus BackupSnapshotStatus the fields exclusive to the volumeSnapshot method backup Field Description elements []BackupSnapshotElementStatus The elements list, populated with the gathered volume snapshots","title":"BackupSnapshotStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupSource","text":"Appears in: BootstrapRecovery BackupSource contains the backup we need to restore from, plus some information that could be needed to correctly restore it. Field Description LocalObjectReference github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference (Members of LocalObjectReference are embedded into this type.) No description provided. endpointCA github.com/cloudnative-pg/machinery/pkg/api.SecretKeySelector EndpointCA store the CA bundle of the barman endpoint. Useful when using self-signed certificates to avoid errors with certificate issuer and barman-cloud-wal-archive.","title":"BackupSource"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupSpec","text":"Appears in: Backup BackupSpec defines the desired state of Backup Field Description cluster [Required] github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The cluster to backup target BackupTarget The policy to decide which instance should perform this backup. If empty, it defaults to cluster.spec.backup.target . Available options are empty string, primary and prefer-standby . primary to have backups run always on primary instances, prefer-standby to have backups run preferably on the most updated standby, if available. method BackupMethod The backup method to be used, possible options are barmanObjectStore , volumeSnapshot or plugin . Defaults to: barmanObjectStore . pluginConfiguration BackupPluginConfiguration Configuration parameters passed to the plugin managing this backup online bool Whether the default type of backup with volume snapshots is online/hot ( true , default) or offline/cold ( false ) Overrides the default setting specified in the cluster field '.spec.backup.volumeSnapshot.online' onlineConfiguration OnlineConfiguration Configuration parameters to control the online/hot backup with volume snapshots Overrides the default settings specified in the cluster '.backup.volumeSnapshot.onlineConfiguration' stanza","title":"BackupSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupStatus","text":"Appears in: Backup BackupStatus defines the observed state of Backup Field Description BarmanCredentials github.com/cloudnative-pg/barman-cloud/pkg/api.BarmanCredentials (Members of BarmanCredentials are embedded into this type.) The potential credentials for each cloud provider endpointCA github.com/cloudnative-pg/machinery/pkg/api.SecretKeySelector EndpointCA store the CA bundle of the barman endpoint. Useful when using self-signed certificates to avoid errors with certificate issuer and barman-cloud-wal-archive. endpointURL string Endpoint to be used to upload data to the cloud, overriding the automatic endpoint discovery destinationPath string The path where to store the backup (i.e. s3://bucket/path/to/folder) this path, with different destination folders, will be used for WALs and for data. This may not be populated in case of errors. serverName string The server name on S3, the cluster name is used if this parameter is omitted encryption string Encryption method required to S3 API backupId string The ID of the Barman backup backupName string The Name of the Barman backup phase BackupPhase The last backup status startedAt meta/v1.Time When the backup was started stoppedAt meta/v1.Time When the backup was terminated beginWal string The starting WAL endWal string The ending WAL beginLSN string The starting xlog endLSN string The ending xlog error string The detected error commandOutput string Unused. Retained for compatibility with old versions. commandError string The backup command output in case of error backupLabelFile []byte Backup label file content as returned by Postgres in case of online (hot) backups tablespaceMapFile []byte Tablespace map file content as returned by Postgres in case of online (hot) backups instanceID InstanceID Information to identify the instance where the backup has been taken from snapshotBackupStatus BackupSnapshotStatus Status of the volumeSnapshot backup method BackupMethod The backup method being used online bool Whether the backup was online/hot ( true ) or offline/cold ( false ) pluginMetadata map[string]string A map containing the plugin metadata","title":"BackupStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BackupTarget","text":"(Alias of string ) Appears in: BackupConfiguration BackupSpec ScheduledBackupSpec BackupTarget describes the preferred targets for a backup","title":"BackupTarget"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BootstrapConfiguration","text":"Appears in: ClusterSpec BootstrapConfiguration contains information about how to create the PostgreSQL cluster. Only a single bootstrap method can be defined among the supported ones. initdb will be used as the bootstrap method if left unspecified. Refer to the Bootstrap page of the documentation for more information. Field Description initdb BootstrapInitDB Bootstrap the cluster via initdb recovery BootstrapRecovery Bootstrap the cluster from a backup pg_basebackup BootstrapPgBaseBackup Bootstrap the cluster taking a physical backup of another compatible PostgreSQL instance","title":"BootstrapConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BootstrapInitDB","text":"Appears in: BootstrapConfiguration BootstrapInitDB is the configuration of the bootstrap process when initdb is used Refer to the Bootstrap page of the documentation for more information. Field Description database string Name of the database used by the application. Default: app . owner string Name of the owner of the database in the instance to be used by applications. Defaults to the value of the database key. secret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference Name of the secret containing the initial credentials for the owner of the user database. If empty a new secret will be created from scratch options []string The list of options that must be passed to initdb when creating the cluster. Deprecated: This could lead to inconsistent configurations, please use the explicit provided parameters instead. If defined, explicit values will be ignored. dataChecksums bool Whether the -k option should be passed to initdb, enabling checksums on data pages (default: false ) encoding string The value to be passed as option --encoding for initdb (default: UTF8 ) localeCollate string The value to be passed as option --lc-collate for initdb (default: C ) localeCType string The value to be passed as option --lc-ctype for initdb (default: C ) locale string Sets the default collation order and character classification in the new database. localeProvider string This option sets the locale provider for databases created in the new cluster. Available from PostgreSQL 16. icuLocale string Specifies the ICU locale when the ICU provider is used. This option requires localeProvider to be set to icu . Available from PostgreSQL 15. icuRules string Specifies additional collation rules to customize the behavior of the default collation. This option requires localeProvider to be set to icu . Available from PostgreSQL 16. builtinLocale string Specifies the locale name when the builtin provider is used. This option requires localeProvider to be set to builtin . Available from PostgreSQL 17. walSegmentSize int The value in megabytes (1 to 1024) to be passed to the --wal-segsize option for initdb (default: empty, resulting in PostgreSQL default: 16MB) postInitSQL []string List of SQL queries to be executed as a superuser in the postgres database right after the cluster has been created - to be used with extreme care (by default empty) postInitApplicationSQL []string List of SQL queries to be executed as a superuser in the application database right after the cluster has been created - to be used with extreme care (by default empty) postInitTemplateSQL []string List of SQL queries to be executed as a superuser in the template1 database right after the cluster has been created - to be used with extreme care (by default empty) import Import Bootstraps the new cluster by importing data from an existing PostgreSQL instance using logical backup ( pg_dump and pg_restore ) postInitApplicationSQLRefs SQLRefs List of references to ConfigMaps or Secrets containing SQL files to be executed as a superuser in the application database right after the cluster has been created. The references are processed in a specific order: first, all Secrets are processed, followed by all ConfigMaps. Within each group, the processing order follows the sequence specified in their respective arrays. (by default empty) postInitTemplateSQLRefs SQLRefs List of references to ConfigMaps or Secrets containing SQL files to be executed as a superuser in the template1 database right after the cluster has been created. The references are processed in a specific order: first, all Secrets are processed, followed by all ConfigMaps. Within each group, the processing order follows the sequence specified in their respective arrays. (by default empty) postInitSQLRefs SQLRefs List of references to ConfigMaps or Secrets containing SQL files to be executed as a superuser in the postgres database right after the cluster has been created. The references are processed in a specific order: first, all Secrets are processed, followed by all ConfigMaps. Within each group, the processing order follows the sequence specified in their respective arrays. (by default empty)","title":"BootstrapInitDB"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BootstrapPgBaseBackup","text":"Appears in: BootstrapConfiguration BootstrapPgBaseBackup contains the configuration required to take a physical backup of an existing PostgreSQL cluster Field Description source [Required] string The name of the server of which we need to take a physical backup database string Name of the database used by the application. Default: app . owner string Name of the owner of the database in the instance to be used by applications. Defaults to the value of the database key. secret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference Name of the secret containing the initial credentials for the owner of the user database. If empty a new secret will be created from scratch","title":"BootstrapPgBaseBackup"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-BootstrapRecovery","text":"Appears in: BootstrapConfiguration BootstrapRecovery contains the configuration required to restore from an existing cluster using 3 methodologies: external cluster, volume snapshots or backup objects. Full recovery and Point-In-Time Recovery are supported. The method can be also be used to create clusters in continuous recovery (replica clusters), also supporting cascading replication when instances > Once the cluster exits recovery, the password for the superuser will be changed through the provided secret. Refer to the Bootstrap page of the documentation for more information. Field Description backup BackupSource The backup object containing the physical base backup from which to initiate the recovery procedure. Mutually exclusive with source and volumeSnapshots . source string The external cluster whose backup we will restore. This is also used as the name of the folder under which the backup is stored, so it must be set to the name of the source cluster Mutually exclusive with backup . volumeSnapshots DataSource The static PVC data source(s) from which to initiate the recovery procedure. Currently supporting VolumeSnapshot and PersistentVolumeClaim resources that map an existing PVC group, compatible with CloudNativePG, and taken with a cold backup copy on a fenced Postgres instance (limitation which will be removed in the future when online backup will be implemented). Mutually exclusive with backup . recoveryTarget RecoveryTarget By default, the recovery process applies all the available WAL files in the archive (full recovery). However, you can also end the recovery as soon as a consistent state is reached or recover to a point-in-time (PITR) by specifying a RecoveryTarget object, as expected by PostgreSQL (i.e., timestamp, transaction Id, LSN, ...). More info: https://www.postgresql.org/docs/current/runtime-config-wal.html#RUNTIME-CONFIG-WAL-RECOVERY-TARGET database string Name of the database used by the application. Default: app . owner string Name of the owner of the database in the instance to be used by applications. Defaults to the value of the database key. secret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference Name of the secret containing the initial credentials for the owner of the user database. If empty a new secret will be created from scratch","title":"BootstrapRecovery"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-CatalogImage","text":"Appears in: ImageCatalogSpec CatalogImage defines the image and major version Field Description image [Required] string The image reference major [Required] int The PostgreSQL major version of the image. Must be unique within the catalog.","title":"CatalogImage"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-CertificatesConfiguration","text":"Appears in: CertificatesStatus ClusterSpec CertificatesConfiguration contains the needed configurations to handle server certificates. Field Description serverCASecret string The secret containing the Server CA certificate. If not defined, a new secret will be created with a self-signed CA and will be used to generate the TLS certificate ServerTLSSecret. Contains: ca.crt : CA that should be used to validate the server certificate, used as sslrootcert in client connection strings. ca.key : key used to generate Server SSL certs, if ServerTLSSecret is provided, this can be omitted. serverTLSSecret string The secret of type kubernetes.io/tls containing the server TLS certificate and key that will be set as ssl_cert_file and ssl_key_file so that clients can connect to postgres securely. If not defined, ServerCASecret must provide also ca.key and a new secret will be created using the provided CA. replicationTLSSecret string The secret of type kubernetes.io/tls containing the client certificate to authenticate as the streaming_replica user. If not defined, ClientCASecret must provide also ca.key , and a new secret will be created using the provided CA. clientCASecret string The secret containing the Client CA certificate. If not defined, a new secret will be created with a self-signed CA and will be used to generate all the client certificates. Contains: ca.crt : CA that should be used to validate the client certificates, used as ssl_ca_file of all the instances. ca.key : key used to generate client certificates, if ReplicationTLSSecret is provided, this can be omitted. serverAltDNSNames []string The list of the server alternative DNS names to be added to the generated server TLS certificates, when required.","title":"CertificatesConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-CertificatesStatus","text":"Appears in: ClusterStatus CertificatesStatus contains configuration certificates and related expiration dates. Field Description CertificatesConfiguration CertificatesConfiguration (Members of CertificatesConfiguration are embedded into this type.) Needed configurations to handle server certificates, initialized with default values, if needed. expirations map[string]string Expiration dates for all certificates.","title":"CertificatesStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ClusterMonitoringTLSConfiguration","text":"Appears in: MonitoringConfiguration ClusterMonitoringTLSConfiguration is the type containing the TLS configuration for the cluster's monitoring Field Description enabled bool Enable TLS for the monitoring endpoint. Changing this option will force a rollout of all instances.","title":"ClusterMonitoringTLSConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ClusterSpec","text":"Appears in: Cluster ClusterSpec defines the desired state of Cluster Field Description description string Description of this PostgreSQL cluster inheritedMetadata EmbeddedObjectMetadata Metadata that will be inherited by all objects related to the Cluster imageName string Name of the container image, supporting both tags ( : ) and digests for deterministic and repeatable deployments ( :@sha256: ) imageCatalogRef ImageCatalogRef Defines the major PostgreSQL version we want to use within an ImageCatalog imagePullPolicy core/v1.PullPolicy Image pull policy. One of Always , Never or IfNotPresent . If not defined, it defaults to IfNotPresent . Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images schedulerName string If specified, the pod will be dispatched by specified Kubernetes scheduler. If not specified, the pod will be dispatched by the default scheduler. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/ postgresUID int64 The UID of the postgres user inside the image, defaults to 26 postgresGID int64 The GID of the postgres user inside the image, defaults to 26 instances [Required] int Number of instances required in the cluster minSyncReplicas int Minimum number of instances required in synchronous replication with the primary. Undefined or 0 allow writes to complete when no standby is available. maxSyncReplicas int The target value for the synchronous replication quorum, that can be decreased if the number of ready standbys is lower than this. Undefined or 0 disable synchronous replication. postgresql PostgresConfiguration Configuration of the PostgreSQL server replicationSlots ReplicationSlotsConfiguration Replication slots management configuration bootstrap BootstrapConfiguration Instructions to bootstrap this cluster replica ReplicaClusterConfiguration Replica cluster configuration superuserSecret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The secret containing the superuser password. If not defined a new secret will be created with a randomly generated password enableSuperuserAccess bool When this option is enabled, the operator will use the SuperuserSecret to update the postgres user password (if the secret is not present, the operator will automatically create one). When this option is disabled, the operator will ignore the SuperuserSecret content, delete it when automatically created, and then blank the password of the postgres user by setting it to NULL . Disabled by default. certificates CertificatesConfiguration The configuration for the CA and related certificates imagePullSecrets []github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The list of pull secrets to be used to pull the images storage StorageConfiguration Configuration of the storage of the instances serviceAccountTemplate ServiceAccountTemplate Configure the generation of the service account walStorage StorageConfiguration Configuration of the storage for PostgreSQL WAL (Write-Ahead Log) ephemeralVolumeSource core/v1.EphemeralVolumeSource EphemeralVolumeSource allows the user to configure the source of ephemeral volumes. startDelay int32 The time in seconds that is allowed for a PostgreSQL instance to successfully start up (default 3600). The startup probe failure threshold is derived from this value using the formula: ceiling(startDelay / 10). stopDelay int32 The time in seconds that is allowed for a PostgreSQL instance to gracefully shutdown (default 1800) smartShutdownTimeout int32 The time in seconds that controls the window of time reserved for the smart shutdown of Postgres to complete. Make sure you reserve enough time for the operator to request a fast shutdown of Postgres (that is: stopDelay - smartShutdownTimeout ). switchoverDelay int32 The time in seconds that is allowed for a primary PostgreSQL instance to gracefully shutdown during a switchover. Default value is 3600 seconds (1 hour). failoverDelay int32 The amount of time (in seconds) to wait before triggering a failover after the primary PostgreSQL instance in the cluster was detected to be unhealthy livenessProbeTimeout int32 LivenessProbeTimeout is the time (in seconds) that is allowed for a PostgreSQL instance to successfully respond to the liveness probe (default 30). The Liveness probe failure threshold is derived from this value using the formula: ceiling(livenessProbe / 10). affinity AffinityConfiguration Affinity/Anti-affinity rules for Pods topologySpreadConstraints []core/v1.TopologySpreadConstraint TopologySpreadConstraints specifies how to spread matching pods among the given topology. More info: https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/ resources core/v1.ResourceRequirements Resources requirements of every generated Pod. Please refer to https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ for more information. ephemeralVolumesSizeLimit EphemeralVolumesSizeLimitConfiguration EphemeralVolumesSizeLimit allows the user to set the limits for the ephemeral volumes priorityClassName string Name of the priority class which will be used in every generated Pod, if the PriorityClass specified does not exist, the pod will not be able to schedule. Please refer to https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass for more information primaryUpdateStrategy PrimaryUpdateStrategy Deployment strategy to follow to upgrade the primary server during a rolling update procedure, after all replicas have been successfully updated: it can be automated ( unsupervised - default) or manual ( supervised ) primaryUpdateMethod PrimaryUpdateMethod Method to follow to upgrade the primary server during a rolling update procedure, after all replicas have been successfully updated: it can be with a switchover ( switchover ) or in-place ( restart - default) backup BackupConfiguration The configuration to be used for backups nodeMaintenanceWindow NodeMaintenanceWindow Define a maintenance window for the Kubernetes nodes monitoring MonitoringConfiguration The configuration of the monitoring infrastructure of this cluster externalClusters []ExternalCluster The list of external clusters which are used in the configuration logLevel string The instances' log level, one of the following values: error, warning, info (default), debug, trace projectedVolumeTemplate core/v1.ProjectedVolumeSource Template to be used to define projected volumes, projected volumes will be mounted under /projected base folder env []core/v1.EnvVar Env follows the Env format to pass environment variables to the pods created in the cluster envFrom []core/v1.EnvFromSource EnvFrom follows the EnvFrom format to pass environment variables sources to the pods to be used by Env managed ManagedConfiguration The configuration that is used by the portions of PostgreSQL that are managed by the instance manager seccompProfile core/v1.SeccompProfile The SeccompProfile applied to every Pod and Container. Defaults to: RuntimeDefault tablespaces []TablespaceConfiguration The tablespaces configuration enablePDB bool Manage the PodDisruptionBudget resources within the cluster. When configured as true (default setting), the pod disruption budgets will safeguard the primary node from being terminated. Conversely, setting it to false will result in the absence of any PodDisruptionBudget resource, permitting the shutdown of all nodes hosting the PostgreSQL cluster. This latter configuration is advisable for any PostgreSQL cluster employed for development/staging purposes. plugins []PluginConfiguration The plugins configuration, containing any plugin to be loaded with the corresponding configuration probes ProbesConfiguration The configuration of the probes to be injected in the PostgreSQL Pods.","title":"ClusterSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ClusterStatus","text":"Appears in: Cluster ClusterStatus defines the observed state of Cluster Field Description instances int The total number of PVC Groups detected in the cluster. It may differ from the number of existing instance pods. readyInstances int The total number of ready instances in the cluster. It is equal to the number of ready instance pods. instancesStatus map[PodStatus][]string InstancesStatus indicates in which status the instances are instancesReportedState map[PodName]InstanceReportedState The reported state of the instances during the last reconciliation loop managedRolesStatus ManagedRoles ManagedRolesStatus reports the state of the managed roles in the cluster tablespacesStatus []TablespaceState TablespacesStatus reports the state of the declarative tablespaces in the cluster timelineID int The timeline of the Postgres cluster topology Topology Instances topology. latestGeneratedNode int ID of the latest generated node (used to avoid node name clashing) currentPrimary string Current primary instance targetPrimary string Target primary instance, this is different from the previous one during a switchover or a failover lastPromotionToken string LastPromotionToken is the last verified promotion token that was used to promote a replica cluster pvcCount int32 How many PVCs have been created by this cluster jobCount int32 How many Jobs have been created by this cluster danglingPVC []string List of all the PVCs created by this cluster and still available which are not attached to a Pod resizingPVC []string List of all the PVCs that have ResizingPVC condition. initializingPVC []string List of all the PVCs that are being initialized by this cluster healthyPVC []string List of all the PVCs not dangling nor initializing unusablePVC []string List of all the PVCs that are unusable because another PVC is missing writeService string Current write pod readService string Current list of read pods phase string Current phase of the cluster phaseReason string Reason for the current phase secretsResourceVersion SecretsResourceVersion The list of resource versions of the secrets managed by the operator. Every change here is done in the interest of the instance manager, which will refresh the secret data configMapResourceVersion ConfigMapResourceVersion The list of resource versions of the configmaps, managed by the operator. Every change here is done in the interest of the instance manager, which will refresh the configmap data certificates CertificatesStatus The configuration for the CA and related certificates, initialized with defaults. firstRecoverabilityPoint string The first recoverability point, stored as a date in RFC3339 format. This field is calculated from the content of FirstRecoverabilityPointByMethod firstRecoverabilityPointByMethod map[BackupMethod]meta/v1.Time The first recoverability point, stored as a date in RFC3339 format, per backup method type lastSuccessfulBackup string Last successful backup, stored as a date in RFC3339 format This field is calculated from the content of LastSuccessfulBackupByMethod lastSuccessfulBackupByMethod map[BackupMethod]meta/v1.Time Last successful backup, stored as a date in RFC3339 format, per backup method type lastFailedBackup string Stored as a date in RFC3339 format cloudNativePGCommitHash string The commit hash number of which this operator running currentPrimaryTimestamp string The timestamp when the last actual promotion to primary has occurred currentPrimaryFailingSinceTimestamp string The timestamp when the primary was detected to be unhealthy This field is reported when .spec.failoverDelay is populated or during online upgrades targetPrimaryTimestamp string The timestamp when the last request for a new primary has occurred poolerIntegrations PoolerIntegrations The integration needed by poolers referencing the cluster cloudNativePGOperatorHash string The hash of the binary of the operator availableArchitectures []AvailableArchitecture AvailableArchitectures reports the available architectures of a cluster conditions []meta/v1.Condition Conditions for cluster object instanceNames []string List of instance names in the cluster onlineUpdateEnabled bool OnlineUpdateEnabled shows if the online upgrade is enabled inside the cluster azurePVCUpdateEnabled bool AzurePVCUpdateEnabled shows if the PVC online upgrade is enabled for this cluster image string Image contains the image name used by the pods pluginStatus []PluginStatus PluginStatus is the status of the loaded plugins switchReplicaClusterStatus SwitchReplicaClusterStatus SwitchReplicaClusterStatus is the status of the switch to replica cluster demotionToken string DemotionToken is a JSON token containing the information from pg_controldata such as Database system identifier, Latest checkpoint's TimeLineID, Latest checkpoint's REDO location, Latest checkpoint's REDO WAL file, and Time of latest checkpoint systemID string SystemID is the latest detected PostgreSQL SystemID","title":"ClusterStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ConfigMapResourceVersion","text":"Appears in: ClusterStatus ConfigMapResourceVersion is the resource versions of the secrets managed by the operator Field Description metrics map[string]string A map with the versions of all the config maps used to pass metrics. Map keys are the config map names, map values are the versions","title":"ConfigMapResourceVersion"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-DataDurabilityLevel","text":"(Alias of string ) Appears in: SynchronousReplicaConfiguration DataDurabilityLevel specifies how strictly to enforce synchronous replication when cluster instances are unavailable. Options are required or preferred .","title":"DataDurabilityLevel"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-DataSource","text":"Appears in: BootstrapRecovery DataSource contains the configuration required to bootstrap a PostgreSQL cluster from an existing storage Field Description storage [Required] core/v1.TypedLocalObjectReference Configuration of the storage of the instances walStorage core/v1.TypedLocalObjectReference Configuration of the storage for PostgreSQL WAL (Write-Ahead Log) tablespaceStorage map[string]core/v1.TypedLocalObjectReference Configuration of the storage for PostgreSQL tablespaces","title":"DataSource"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-DatabaseReclaimPolicy","text":"(Alias of string ) Appears in: DatabaseSpec DatabaseReclaimPolicy describes a policy for end-of-life maintenance of databases.","title":"DatabaseReclaimPolicy"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-DatabaseRoleRef","text":"Appears in: TablespaceConfiguration DatabaseRoleRef is a reference an a role available inside PostgreSQL Field Description name string No description provided.","title":"DatabaseRoleRef"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-DatabaseSpec","text":"Appears in: Database DatabaseSpec is the specification of a Postgresql Database, built around the CREATE DATABASE , ALTER DATABASE , and DROP DATABASE SQL commands of PostgreSQL. Field Description cluster [Required] core/v1.LocalObjectReference The name of the PostgreSQL cluster hosting the database. ensure EnsureOption Ensure the PostgreSQL database is present or absent - defaults to \"present\". name [Required] string The name of the database to create inside PostgreSQL. This setting cannot be changed. owner [Required] string Maps to the OWNER parameter of CREATE DATABASE . Maps to the OWNER TO command of ALTER DATABASE . The role name of the user who owns the database inside PostgreSQL. template string Maps to the TEMPLATE parameter of CREATE DATABASE . This setting cannot be changed. The name of the template from which to create this database. encoding string Maps to the ENCODING parameter of CREATE DATABASE . This setting cannot be changed. Character set encoding to use in the database. locale string Maps to the LOCALE parameter of CREATE DATABASE . This setting cannot be changed. Sets the default collation order and character classification in the new database. localeProvider string Maps to the LOCALE_PROVIDER parameter of CREATE DATABASE . This setting cannot be changed. This option sets the locale provider for databases created in the new cluster. Available from PostgreSQL 16. localeCollate string Maps to the LC_COLLATE parameter of CREATE DATABASE . This setting cannot be changed. localeCType string Maps to the LC_CTYPE parameter of CREATE DATABASE . This setting cannot be changed. icuLocale string Maps to the ICU_LOCALE parameter of CREATE DATABASE . This setting cannot be changed. Specifies the ICU locale when the ICU provider is used. This option requires localeProvider to be set to icu . Available from PostgreSQL 15. icuRules string Maps to the ICU_RULES parameter of CREATE DATABASE . This setting cannot be changed. Specifies additional collation rules to customize the behavior of the default collation. This option requires localeProvider to be set to icu . Available from PostgreSQL 16. builtinLocale string Maps to the BUILTIN_LOCALE parameter of CREATE DATABASE . This setting cannot be changed. Specifies the locale name when the builtin provider is used. This option requires localeProvider to be set to builtin . Available from PostgreSQL 17. collationVersion string Maps to the COLLATION_VERSION parameter of CREATE DATABASE . This setting cannot be changed. isTemplate bool Maps to the IS_TEMPLATE parameter of CREATE DATABASE and ALTER DATABASE . If true, this database is considered a template and can be cloned by any user with CREATEDB privileges. allowConnections bool Maps to the ALLOW_CONNECTIONS parameter of CREATE DATABASE and ALTER DATABASE . If false then no one can connect to this database. connectionLimit int Maps to the CONNECTION LIMIT clause of CREATE DATABASE and ALTER DATABASE . How many concurrent connections can be made to this database. -1 (the default) means no limit. tablespace string Maps to the TABLESPACE parameter of CREATE DATABASE . Maps to the SET TABLESPACE command of ALTER DATABASE . The name of the tablespace (in PostgreSQL) that will be associated with the new database. This tablespace will be the default tablespace used for objects created in this database. databaseReclaimPolicy DatabaseReclaimPolicy The policy for end-of-life maintenance of this database.","title":"DatabaseSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-DatabaseStatus","text":"Appears in: Database DatabaseStatus defines the observed state of Database Field Description observedGeneration int64 A sequence number representing the latest desired state that was synchronized applied bool Applied is true if the database was reconciled correctly message string Message is the reconciliation output message","title":"DatabaseStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-EmbeddedObjectMetadata","text":"Appears in: ClusterSpec EmbeddedObjectMetadata contains metadata to be inherited by all resources related to a Cluster Field Description labels map[string]string No description provided. annotations map[string]string No description provided.","title":"EmbeddedObjectMetadata"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-EnsureOption","text":"(Alias of string ) Appears in: DatabaseSpec RoleConfiguration EnsureOption represents whether we should enforce the presence or absence of a Role in a PostgreSQL instance","title":"EnsureOption"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-EphemeralVolumesSizeLimitConfiguration","text":"Appears in: ClusterSpec EphemeralVolumesSizeLimitConfiguration contains the configuration of the ephemeral storage Field Description shm k8s.io/apimachinery/pkg/api/resource.Quantity Shm is the size limit of the shared memory volume temporaryData k8s.io/apimachinery/pkg/api/resource.Quantity TemporaryData is the size limit of the temporary data volume","title":"EphemeralVolumesSizeLimitConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ExternalCluster","text":"Appears in: ClusterSpec ExternalCluster represents the connection parameters to an external cluster which is used in the other sections of the configuration Field Description name [Required] string The server name, required connectionParameters map[string]string The list of connection parameters, such as dbname, host, username, etc sslCert core/v1.SecretKeySelector The reference to an SSL certificate to be used to connect to this instance sslKey core/v1.SecretKeySelector The reference to an SSL private key to be used to connect to this instance sslRootCert core/v1.SecretKeySelector The reference to an SSL CA public key to be used to connect to this instance password core/v1.SecretKeySelector The reference to the password to be used to connect to the server. If a password is provided, CloudNativePG creates a PostgreSQL passfile at /controller/external/NAME/pass (where \"NAME\" is the cluster's name). This passfile is automatically referenced in the connection string when establishing a connection to the remote PostgreSQL server from the current PostgreSQL Cluster . This ensures secure and efficient password management for external clusters. barmanObjectStore github.com/cloudnative-pg/barman-cloud/pkg/api.BarmanObjectStoreConfiguration The configuration for the barman-cloud tool suite plugin [Required] PluginConfiguration The configuration of the plugin that is taking care of WAL archiving and backups for this external cluster","title":"ExternalCluster"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ImageCatalogRef","text":"Appears in: ClusterSpec ImageCatalogRef defines the reference to a major version in an ImageCatalog Field Description TypedLocalObjectReference core/v1.TypedLocalObjectReference (Members of TypedLocalObjectReference are embedded into this type.) No description provided. major [Required] int The major version of PostgreSQL we want to use from the ImageCatalog","title":"ImageCatalogRef"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ImageCatalogSpec","text":"Appears in: ClusterImageCatalog ImageCatalog ImageCatalogSpec defines the desired ImageCatalog Field Description images [Required] []CatalogImage List of CatalogImages available in the catalog","title":"ImageCatalogSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Import","text":"Appears in: BootstrapInitDB Import contains the configuration to init a database from a logic snapshot of an externalCluster Field Description source [Required] ImportSource The source of the import type [Required] SnapshotType The import type. Can be microservice or monolith . databases [Required] []string The databases to import roles []string The roles to import postImportApplicationSQL []string List of SQL queries to be executed as a superuser in the application database right after is imported - to be used with extreme care (by default empty). Only available in microservice type. schemaOnly bool When set to true, only the pre-data and post-data sections of pg_restore are invoked, avoiding data import. Default: false . pgDumpExtraOptions []string List of custom options to pass to the pg_dump command. IMPORTANT: Use these options with caution and at your own risk, as the operator does not validate their content. Be aware that certain options may conflict with the operator's intended functionality or design. pgRestoreExtraOptions []string List of custom options to pass to the pg_restore command. IMPORTANT: Use these options with caution and at your own risk, as the operator does not validate their content. Be aware that certain options may conflict with the operator's intended functionality or design.","title":"Import"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ImportSource","text":"Appears in: Import ImportSource describes the source for the logical snapshot Field Description externalCluster [Required] string The name of the externalCluster used for import","title":"ImportSource"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-InstanceID","text":"Appears in: BackupStatus InstanceID contains the information to identify an instance Field Description podName string The pod name ContainerID string The container ID","title":"InstanceID"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-InstanceReportedState","text":"Appears in: ClusterStatus InstanceReportedState describes the last reported state of an instance during a reconciliation loop Field Description isPrimary [Required] bool indicates if an instance is the primary one timeLineID int indicates on which TimelineId the instance is","title":"InstanceReportedState"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-LDAPBindAsAuth","text":"Appears in: LDAPConfig LDAPBindAsAuth provides the required fields to use the bind authentication for LDAP Field Description prefix string Prefix for the bind authentication option suffix string Suffix for the bind authentication option","title":"LDAPBindAsAuth"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-LDAPBindSearchAuth","text":"Appears in: LDAPConfig LDAPBindSearchAuth provides the required fields to use the bind+search LDAP authentication process Field Description baseDN string Root DN to begin the user search bindDN string DN of the user to bind to the directory bindPassword core/v1.SecretKeySelector Secret with the password for the user to bind to the directory searchAttribute string Attribute to match against the username searchFilter string Search filter to use when doing the search+bind authentication","title":"LDAPBindSearchAuth"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-LDAPConfig","text":"Appears in: PostgresConfiguration LDAPConfig contains the parameters needed for LDAP authentication Field Description server string LDAP hostname or IP address port int LDAP server port scheme LDAPScheme LDAP schema to be used, possible options are ldap and ldaps bindAsAuth LDAPBindAsAuth Bind as authentication configuration bindSearchAuth LDAPBindSearchAuth Bind+Search authentication configuration tls bool Set to 'true' to enable LDAP over TLS. 'false' is default","title":"LDAPConfig"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-LDAPScheme","text":"(Alias of string ) Appears in: LDAPConfig LDAPScheme defines the possible schemes for LDAP","title":"LDAPScheme"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ManagedConfiguration","text":"Appears in: ClusterSpec ManagedConfiguration represents the portions of PostgreSQL that are managed by the instance manager Field Description roles []RoleConfiguration Database roles managed by the Cluster services ManagedServices Services roles managed by the Cluster","title":"ManagedConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ManagedRoles","text":"Appears in: ClusterStatus ManagedRoles tracks the status of a cluster's managed roles Field Description byStatus map[RoleStatus][]string ByStatus gives the list of roles in each state cannotReconcile map[string][]string CannotReconcile lists roles that cannot be reconciled in PostgreSQL, with an explanation of the cause passwordStatus map[string]PasswordState PasswordStatus gives the last transaction id and password secret version for each managed role","title":"ManagedRoles"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ManagedService","text":"Appears in: ManagedServices ManagedService represents a specific service managed by the cluster. It includes the type of service and its associated template specification. Field Description selectorType [Required] ServiceSelectorType SelectorType specifies the type of selectors that the service will have. Valid values are \"rw\", \"r\", and \"ro\", representing read-write, read, and read-only services. updateStrategy ServiceUpdateStrategy UpdateStrategy describes how the service differences should be reconciled serviceTemplate [Required] ServiceTemplateSpec ServiceTemplate is the template specification for the service.","title":"ManagedService"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ManagedServices","text":"Appears in: ManagedConfiguration ManagedServices represents the services managed by the cluster. Field Description disabledDefaultServices []ServiceSelectorType DisabledDefaultServices is a list of service types that are disabled by default. Valid values are \"r\", and \"ro\", representing read, and read-only services. additional []ManagedService Additional is a list of additional managed services specified by the user.","title":"ManagedServices"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Metadata","text":"Appears in: PodTemplateSpec ServiceAccountTemplate ServiceTemplateSpec Metadata is a structure similar to the metav1.ObjectMeta, but still parseable by controller-gen to create a suitable CRD for the user. The comment of PodTemplateSpec has an explanation of why we are not using the core data types. Field Description name string The name of the resource. Only supported for certain types labels map[string]string Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels annotations map[string]string Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations","title":"Metadata"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-MonitoringConfiguration","text":"Appears in: ClusterSpec MonitoringConfiguration is the type containing all the monitoring configuration for a certain cluster Field Description disableDefaultQueries bool Whether the default queries should be injected. Set it to true if you don't want to inject default queries into the cluster. Default: false. customQueriesConfigMap []github.com/cloudnative-pg/machinery/pkg/api.ConfigMapKeySelector The list of config maps containing the custom queries customQueriesSecret []github.com/cloudnative-pg/machinery/pkg/api.SecretKeySelector The list of secrets containing the custom queries enablePodMonitor bool Enable or disable the PodMonitor tls ClusterMonitoringTLSConfiguration Configure TLS communication for the metrics endpoint. Changing tls.enabled option will force a rollout of all instances. podMonitorMetricRelabelings []github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1.RelabelConfig The list of metric relabelings for the PodMonitor . Applied to samples before ingestion. podMonitorRelabelings []github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1.RelabelConfig The list of relabelings for the PodMonitor . Applied to samples before scraping.","title":"MonitoringConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-NodeMaintenanceWindow","text":"Appears in: ClusterSpec NodeMaintenanceWindow contains information that the operator will use while upgrading the underlying node. This option is only useful when the chosen storage prevents the Pods from being freely moved across nodes. Field Description reusePVC bool Reuse the existing PVC (wait for the node to come up again) or not (recreate it elsewhere - when instances >1) inProgress bool Is there a node maintenance activity in progress?","title":"NodeMaintenanceWindow"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-OnlineConfiguration","text":"Appears in: BackupSpec ScheduledBackupSpec VolumeSnapshotConfiguration OnlineConfiguration contains the configuration parameters for the online volume snapshot Field Description waitForArchive bool If false, the function will return immediately after the backup is completed, without waiting for WAL to be archived. This behavior is only useful with backup software that independently monitors WAL archiving. Otherwise, WAL required to make the backup consistent might be missing and make the backup useless. By default, or when this parameter is true, pg_backup_stop will wait for WAL to be archived when archiving is enabled. On a standby, this means that it will wait only when archive_mode = always. If write activity on the primary is low, it may be useful to run pg_switch_wal on the primary in order to trigger an immediate segment switch. immediateCheckpoint bool Control whether the I/O workload for the backup initial checkpoint will be limited, according to the checkpoint_completion_target setting on the PostgreSQL server. If set to true, an immediate checkpoint will be used, meaning PostgreSQL will complete the checkpoint as soon as possible. false by default.","title":"OnlineConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PasswordState","text":"Appears in: ManagedRoles PasswordState represents the state of the password of a managed RoleConfiguration Field Description transactionID int64 the last transaction ID to affect the role definition in PostgreSQL resourceVersion string the resource version of the password secret","title":"PasswordState"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PgBouncerIntegrationStatus","text":"Appears in: PoolerIntegrations PgBouncerIntegrationStatus encapsulates the needed integration for the pgbouncer poolers referencing the cluster Field Description secrets []string No description provided.","title":"PgBouncerIntegrationStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PgBouncerPoolMode","text":"(Alias of string ) Appears in: PgBouncerSpec PgBouncerPoolMode is the mode of PgBouncer","title":"PgBouncerPoolMode"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PgBouncerSecrets","text":"Appears in: PoolerSecrets PgBouncerSecrets contains the versions of the secrets used by pgbouncer Field Description authQuery SecretVersion The auth query secret version","title":"PgBouncerSecrets"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PgBouncerSpec","text":"Appears in: PoolerSpec PgBouncerSpec defines how to configure PgBouncer Field Description poolMode PgBouncerPoolMode The pool mode. Default: session . authQuerySecret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The credentials of the user that need to be used for the authentication query. In case it is specified, also an AuthQuery (e.g. \"SELECT usename, passwd FROM pg_catalog.pg_shadow WHERE usename=$1\") has to be specified and no automatic CNPG Cluster integration will be triggered. authQuery string The query that will be used to download the hash of the password of a certain user. Default: \"SELECT usename, passwd FROM public.user_search($1)\". In case it is specified, also an AuthQuerySecret has to be specified and no automatic CNPG Cluster integration will be triggered. parameters map[string]string Additional parameters to be passed to PgBouncer - please check the CNPG documentation for a list of options you can configure pg_hba []string PostgreSQL Host Based Authentication rules (lines to be appended to the pg_hba.conf file) paused bool When set to true , PgBouncer will disconnect from the PostgreSQL server, first waiting for all queries to complete, and pause all new client connections until this value is set to false (default). Internally, the operator calls PgBouncer's PAUSE and RESUME commands.","title":"PgBouncerSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PluginConfiguration","text":"Appears in: ClusterSpec ExternalCluster PluginConfiguration specifies a plugin that need to be loaded for this cluster to be reconciled Field Description name [Required] string Name is the plugin name enabled bool Enabled is true if this plugin will be used isWALArchiver bool Only one plugin can be declared as WALArchiver. Cannot be active if \".spec.backup.barmanObjectStore\" configuration is present. parameters map[string]string Parameters is the configuration of the plugin","title":"PluginConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PluginStatus","text":"Appears in: ClusterStatus PluginStatus is the status of a loaded plugin Field Description name [Required] string Name is the name of the plugin version [Required] string Version is the version of the plugin loaded by the latest reconciliation loop capabilities []string Capabilities are the list of capabilities of the plugin operatorCapabilities []string OperatorCapabilities are the list of capabilities of the plugin regarding the reconciler walCapabilities []string WALCapabilities are the list of capabilities of the plugin regarding the WAL management backupCapabilities []string BackupCapabilities are the list of capabilities of the plugin regarding the Backup management restoreJobHookCapabilities []string RestoreJobHookCapabilities are the list of capabilities of the plugin regarding the RestoreJobHook management status string Status contain the status reported by the plugin through the SetStatusInCluster interface","title":"PluginStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PodTemplateSpec","text":"Appears in: PoolerSpec PodTemplateSpec is a structure allowing the user to set a template for Pod generation. Unfortunately we can't use the corev1.PodTemplateSpec type because the generated CRD won't have the field for the metadata section. References: https://github.com/kubernetes-sigs/controller-tools/issues/385 https://github.com/kubernetes-sigs/controller-tools/issues/448 https://github.com/prometheus-operator/prometheus-operator/issues/3041 Field Description metadata Metadata Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata spec core/v1.PodSpec Specification of the desired behavior of the pod. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"PodTemplateSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PodTopologyLabels","text":"(Alias of map[string]string ) Appears in: Topology PodTopologyLabels represent the topology of a Pod. map[labelName]labelValue","title":"PodTopologyLabels"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PoolerIntegrations","text":"Appears in: ClusterStatus PoolerIntegrations encapsulates the needed integration for the poolers referencing the cluster Field Description pgBouncerIntegration PgBouncerIntegrationStatus No description provided.","title":"PoolerIntegrations"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PoolerMonitoringConfiguration","text":"Appears in: PoolerSpec PoolerMonitoringConfiguration is the type containing all the monitoring configuration for a certain Pooler. Mirrors the Cluster's MonitoringConfiguration but without the custom queries part for now. Field Description enablePodMonitor bool Enable or disable the PodMonitor podMonitorMetricRelabelings []github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1.RelabelConfig The list of metric relabelings for the PodMonitor . Applied to samples before ingestion. podMonitorRelabelings []github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1.RelabelConfig The list of relabelings for the PodMonitor . Applied to samples before scraping.","title":"PoolerMonitoringConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PoolerSecrets","text":"Appears in: PoolerStatus PoolerSecrets contains the versions of all the secrets used Field Description serverTLS SecretVersion The server TLS secret version serverCA SecretVersion The server CA secret version clientCA SecretVersion The client CA secret version pgBouncerSecrets PgBouncerSecrets The version of the secrets used by PgBouncer","title":"PoolerSecrets"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PoolerSpec","text":"Appears in: Pooler PoolerSpec defines the desired state of Pooler Field Description cluster [Required] github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference This is the cluster reference on which the Pooler will work. Pooler name should never match with any cluster name within the same namespace. type PoolerType Type of service to forward traffic to. Default: rw . instances int32 The number of replicas we want. Default: 1. template PodTemplateSpec The template of the Pod to be created pgbouncer [Required] PgBouncerSpec The PgBouncer configuration deploymentStrategy apps/v1.DeploymentStrategy The deployment strategy to use for pgbouncer to replace existing pods with new ones monitoring PoolerMonitoringConfiguration The configuration of the monitoring infrastructure of this pooler. serviceTemplate ServiceTemplateSpec Template for the Service to be created","title":"PoolerSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PoolerStatus","text":"Appears in: Pooler PoolerStatus defines the observed state of Pooler Field Description secrets PoolerSecrets The resource version of the config object instances int32 The number of pods trying to be scheduled","title":"PoolerStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PoolerType","text":"(Alias of string ) Appears in: PoolerSpec PoolerType is the type of the connection pool, meaning the service we are targeting. Allowed values are rw and ro .","title":"PoolerType"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PostgresConfiguration","text":"Appears in: ClusterSpec PostgresConfiguration defines the PostgreSQL configuration Field Description parameters map[string]string PostgreSQL configuration options (postgresql.conf) synchronous SynchronousReplicaConfiguration Configuration of the PostgreSQL synchronous replication feature pg_hba []string PostgreSQL Host Based Authentication rules (lines to be appended to the pg_hba.conf file) pg_ident []string PostgreSQL User Name Maps rules (lines to be appended to the pg_ident.conf file) syncReplicaElectionConstraint SyncReplicaElectionConstraints Requirements to be met by sync replicas. This will affect how the \"synchronous_standby_names\" parameter will be set up. shared_preload_libraries []string Lists of shared preload libraries to add to the default ones ldap LDAPConfig Options to specify LDAP configuration promotionTimeout int32 Specifies the maximum number of seconds to wait when promoting an instance to primary. Default value is 40000000, greater than one year in seconds, big enough to simulate an infinite timeout enableAlterSystem bool If this parameter is true, the user will be able to invoke ALTER SYSTEM on this CloudNativePG Cluster. This should only be used for debugging and troubleshooting. Defaults to false.","title":"PostgresConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PrimaryUpdateMethod","text":"(Alias of string ) Appears in: ClusterSpec PrimaryUpdateMethod contains the method to use when upgrading the primary server of the cluster as part of rolling updates","title":"PrimaryUpdateMethod"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PrimaryUpdateStrategy","text":"(Alias of string ) Appears in: ClusterSpec PrimaryUpdateStrategy contains the strategy to follow when upgrading the primary server of the cluster as part of rolling updates","title":"PrimaryUpdateStrategy"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Probe","text":"Appears in: ProbesConfiguration Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic. Field Description initialDelaySeconds int32 Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes timeoutSeconds int32 Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes periodSeconds int32 How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. successThreshold int32 Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1. failureThreshold int32 Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. terminationGracePeriodSeconds int64 Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is a beta field and requires enabling ProbeTerminationGracePeriod feature gate. Minimum value is 1. spec.terminationGracePeriodSeconds is used if unset.","title":"Probe"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ProbesConfiguration","text":"Appears in: ClusterSpec ProbesConfiguration represent the configuration for the probes to be injected in the PostgreSQL Pods Field Description startup [Required] Probe The startup probe configuration liveness [Required] Probe The liveness probe configuration readiness [Required] Probe The readiness probe configuration","title":"ProbesConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PublicationReclaimPolicy","text":"(Alias of string ) Appears in: PublicationSpec PublicationReclaimPolicy defines a policy for end-of-life maintenance of Publications.","title":"PublicationReclaimPolicy"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PublicationSpec","text":"Appears in: Publication PublicationSpec defines the desired state of Publication Field Description cluster [Required] core/v1.LocalObjectReference The name of the PostgreSQL cluster that identifies the \"publisher\" name [Required] string The name of the publication inside PostgreSQL dbname [Required] string The name of the database where the publication will be installed in the \"publisher\" cluster parameters map[string]string Publication parameters part of the WITH clause as expected by PostgreSQL CREATE PUBLICATION command target [Required] PublicationTarget Target of the publication as expected by PostgreSQL CREATE PUBLICATION command publicationReclaimPolicy PublicationReclaimPolicy The policy for end-of-life maintenance of this publication","title":"PublicationSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PublicationStatus","text":"Appears in: Publication PublicationStatus defines the observed state of Publication Field Description observedGeneration int64 A sequence number representing the latest desired state that was synchronized applied bool Applied is true if the publication was reconciled correctly message string Message is the reconciliation output message","title":"PublicationStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PublicationTarget","text":"Appears in: PublicationSpec PublicationTarget is what this publication should publish Field Description allTables bool Marks the publication as one that replicates changes for all tables in the database, including tables created in the future. Corresponding to FOR ALL TABLES in PostgreSQL. objects []PublicationTargetObject Just the following schema objects","title":"PublicationTarget"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PublicationTargetObject","text":"Appears in: PublicationTarget PublicationTargetObject is an object to publish Field Description tablesInSchema string Marks the publication as one that replicates changes for all tables in the specified list of schemas, including tables created in the future. Corresponding to FOR TABLES IN SCHEMA in PostgreSQL. table PublicationTargetTable Specifies a list of tables to add to the publication. Corresponding to FOR TABLE in PostgreSQL.","title":"PublicationTargetObject"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-PublicationTargetTable","text":"Appears in: PublicationTargetObject PublicationTargetTable is a table to publish Field Description only bool Whether to limit to the table only or include all its descendants name [Required] string The table name schema string The schema name columns []string The columns to publish","title":"PublicationTargetTable"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-RecoveryTarget","text":"Appears in: BootstrapRecovery RecoveryTarget allows to configure the moment where the recovery process will stop. All the target options except TargetTLI are mutually exclusive. Field Description backupID string The ID of the backup from which to start the recovery process. If empty (default) the operator will automatically detect the backup based on targetTime or targetLSN if specified. Otherwise use the latest available backup in chronological order. targetTLI string The target timeline (\"latest\" or a positive integer) targetXID string The target transaction ID targetName string The target name (to be previously created with pg_create_restore_point ) targetLSN string The target LSN (Log Sequence Number) targetTime string The target time as a timestamp in the RFC3339 standard targetImmediate bool End recovery as soon as a consistent state is reached exclusive bool Set the target to be exclusive. If omitted, defaults to false, so that in Postgres, recovery_target_inclusive will be true","title":"RecoveryTarget"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ReplicaClusterConfiguration","text":"Appears in: ClusterSpec ReplicaClusterConfiguration encapsulates the configuration of a replica cluster Field Description self string Self defines the name of this cluster. It is used to determine if this is a primary or a replica cluster, comparing it with primary primary string Primary defines which Cluster is defined to be the primary in the distributed PostgreSQL cluster, based on the topology specified in externalClusters source [Required] string The name of the external cluster which is the replication origin enabled bool If replica mode is enabled, this cluster will be a replica of an existing cluster. Replica cluster can be created from a recovery object store or via streaming through pg_basebackup. Refer to the Replica clusters page of the documentation for more information. promotionToken string A demotion token generated by an external cluster used to check if the promotion requirements are met. minApplyDelay meta/v1.Duration When replica mode is enabled, this parameter allows you to replay transactions only when the system time is at least the configured time past the commit time. This provides an opportunity to correct data loss errors. Note that when this parameter is set, a promotion token cannot be used.","title":"ReplicaClusterConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ReplicationSlotsConfiguration","text":"Appears in: ClusterSpec ReplicationSlotsConfiguration encapsulates the configuration of replication slots Field Description highAvailability ReplicationSlotsHAConfiguration Replication slots for high availability configuration updateInterval int Standby will update the status of the local replication slots every updateInterval seconds (default 30). synchronizeReplicas SynchronizeReplicasConfiguration Configures the synchronization of the user defined physical replication slots","title":"ReplicationSlotsConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ReplicationSlotsHAConfiguration","text":"Appears in: ReplicationSlotsConfiguration ReplicationSlotsHAConfiguration encapsulates the configuration of the replication slots that are automatically managed by the operator to control the streaming replication connections with the standby instances for high availability (HA) purposes. Replication slots are a PostgreSQL feature that makes sure that PostgreSQL automatically keeps WAL files in the primary when a streaming client (in this specific case a replica that is part of the HA cluster) gets disconnected. Field Description enabled bool If enabled (default), the operator will automatically manage replication slots on the primary instance and use them in streaming replication connections with all the standby instances that are part of the HA cluster. If disabled, the operator will not take advantage of replication slots in streaming connections with the replicas. This feature also controls replication slots in replica cluster, from the designated primary to its cascading replicas. slotPrefix string Prefix for replication slots managed by the operator for HA. It may only contain lower case letters, numbers, and the underscore character. This can only be set at creation time. By default set to _cnpg_ .","title":"ReplicationSlotsHAConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-RoleConfiguration","text":"Appears in: ManagedConfiguration RoleConfiguration is the representation, in Kubernetes, of a PostgreSQL role with the additional field Ensure specifying whether to ensure the presence or absence of the role in the database The defaults of the CREATE ROLE command are applied Reference: https://www.postgresql.org/docs/current/sql-createrole.html Field Description name [Required] string Name of the role comment string Description of the role ensure EnsureOption Ensure the role is present or absent - defaults to \"present\" passwordSecret github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference Secret containing the password of the role (if present) If null, the password will be ignored unless DisablePassword is set connectionLimit int64 If the role can log in, this specifies how many concurrent connections the role can make. -1 (the default) means no limit. validUntil meta/v1.Time Date and time after which the role's password is no longer valid. When omitted, the password will never expire (default). inRoles []string List of one or more existing roles to which this role will be immediately added as a new member. Default empty. inherit bool Whether a role \"inherits\" the privileges of roles it is a member of. Defaults is true . disablePassword bool DisablePassword indicates that a role's password should be set to NULL in Postgres superuser bool Whether the role is a superuser who can override all access restrictions within the database - superuser status is dangerous and should be used only when really needed. You must yourself be a superuser to create a new superuser. Defaults is false . createdb bool When set to true , the role being defined will be allowed to create new databases. Specifying false (default) will deny a role the ability to create databases. createrole bool Whether the role will be permitted to create, alter, drop, comment on, change the security label for, and grant or revoke membership in other roles. Default is false . login bool Whether the role is allowed to log in. A role having the login attribute can be thought of as a user. Roles without this attribute are useful for managing database privileges, but are not users in the usual sense of the word. Default is false . replication bool Whether a role is a replication role. A role must have this attribute (or be a superuser) in order to be able to connect to the server in replication mode (physical or logical replication) and in order to be able to create or drop replication slots. A role having the replication attribute is a very highly privileged role, and should only be used on roles actually used for replication. Default is false . bypassrls bool Whether a role bypasses every row-level security (RLS) policy. Default is false .","title":"RoleConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SQLRefs","text":"Appears in: BootstrapInitDB SQLRefs holds references to ConfigMaps or Secrets containing SQL files. The references are processed in a specific order: first, all Secrets are processed, followed by all ConfigMaps. Within each group, the processing order follows the sequence specified in their respective arrays. Field Description secretRefs []github.com/cloudnative-pg/machinery/pkg/api.SecretKeySelector SecretRefs holds a list of references to Secrets configMapRefs []github.com/cloudnative-pg/machinery/pkg/api.ConfigMapKeySelector ConfigMapRefs holds a list of references to ConfigMaps","title":"SQLRefs"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ScheduledBackupSpec","text":"Appears in: ScheduledBackup ScheduledBackupSpec defines the desired state of ScheduledBackup Field Description suspend bool If this backup is suspended or not immediate bool If the first backup has to be immediately start after creation or not schedule [Required] string The schedule does not follow the same format used in Kubernetes CronJobs as it includes an additional seconds specifier, see https://pkg.go.dev/github.com/robfig/cron#hdr-CRON_Expression_Format cluster [Required] github.com/cloudnative-pg/machinery/pkg/api.LocalObjectReference The cluster to backup backupOwnerReference string Indicates which ownerReference should be put inside the created backup resources. none: no owner reference for created backup objects (same behavior as before the field was introduced) self: sets the Scheduled backup object as owner of the backup cluster: set the cluster as owner of the backup target BackupTarget The policy to decide which instance should perform this backup. If empty, it defaults to cluster.spec.backup.target . Available options are empty string, primary and prefer-standby . primary to have backups run always on primary instances, prefer-standby to have backups run preferably on the most updated standby, if available. method BackupMethod The backup method to be used, possible options are barmanObjectStore , volumeSnapshot or plugin . Defaults to: barmanObjectStore . pluginConfiguration BackupPluginConfiguration Configuration parameters passed to the plugin managing this backup online bool Whether the default type of backup with volume snapshots is online/hot ( true , default) or offline/cold ( false ) Overrides the default setting specified in the cluster field '.spec.backup.volumeSnapshot.online' onlineConfiguration OnlineConfiguration Configuration parameters to control the online/hot backup with volume snapshots Overrides the default settings specified in the cluster '.backup.volumeSnapshot.onlineConfiguration' stanza","title":"ScheduledBackupSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ScheduledBackupStatus","text":"Appears in: ScheduledBackup ScheduledBackupStatus defines the observed state of ScheduledBackup Field Description lastCheckTime meta/v1.Time The latest time the schedule lastScheduleTime meta/v1.Time Information when was the last time that backup was successfully scheduled. nextScheduleTime meta/v1.Time Next time we will run a backup","title":"ScheduledBackupStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SecretVersion","text":"Appears in: PgBouncerSecrets PoolerSecrets SecretVersion contains a secret name and its ResourceVersion Field Description name string The name of the secret version string The ResourceVersion of the secret","title":"SecretVersion"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SecretsResourceVersion","text":"Appears in: ClusterStatus SecretsResourceVersion is the resource versions of the secrets managed by the operator Field Description superuserSecretVersion string The resource version of the \"postgres\" user secret replicationSecretVersion string The resource version of the \"streaming_replica\" user secret applicationSecretVersion string The resource version of the \"app\" user secret managedRoleSecretVersion map[string]string The resource versions of the managed roles secrets caSecretVersion string Unused. Retained for compatibility with old versions. clientCaSecretVersion string The resource version of the PostgreSQL client-side CA secret version serverCaSecretVersion string The resource version of the PostgreSQL server-side CA secret version serverSecretVersion string The resource version of the PostgreSQL server-side secret version barmanEndpointCA string The resource version of the Barman Endpoint CA if provided externalClusterSecretVersion map[string]string The resource versions of the external cluster secrets metrics map[string]string A map with the versions of all the secrets used to pass metrics. Map keys are the secret names, map values are the versions","title":"SecretsResourceVersion"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ServiceAccountTemplate","text":"Appears in: ClusterSpec ServiceAccountTemplate contains the template needed to generate the service accounts Field Description metadata [Required] Metadata Metadata are the metadata to be used for the generated service account","title":"ServiceAccountTemplate"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ServiceSelectorType","text":"(Alias of string ) Appears in: ManagedService ManagedServices ServiceSelectorType describes a valid value for generating the service selectors. It indicates which type of service the selector applies to, such as read-write, read, or read-only","title":"ServiceSelectorType"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ServiceTemplateSpec","text":"Appears in: ManagedService PoolerSpec ServiceTemplateSpec is a structure allowing the user to set a template for Service generation. Field Description metadata Metadata Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata spec core/v1.ServiceSpec Specification of the desired behavior of the service. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status","title":"ServiceTemplateSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-ServiceUpdateStrategy","text":"(Alias of string ) Appears in: ManagedService ServiceUpdateStrategy describes how the changes to the managed service should be handled","title":"ServiceUpdateStrategy"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SnapshotOwnerReference","text":"(Alias of string ) Appears in: VolumeSnapshotConfiguration SnapshotOwnerReference defines the reference type for the owner of the snapshot. This specifies which owner the processed resources should relate to.","title":"SnapshotOwnerReference"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SnapshotType","text":"(Alias of string ) Appears in: Import SnapshotType is a type of allowed import","title":"SnapshotType"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-StorageConfiguration","text":"Appears in: ClusterSpec TablespaceConfiguration StorageConfiguration is the configuration used to create and reconcile PVCs, usable for WAL volumes, PGDATA volumes, or tablespaces Field Description storageClass string StorageClass to use for PVCs. Applied after evaluating the PVC template, if available. If not specified, the generated PVCs will use the default storage class size string Size of the storage. Required if not already specified in the PVC template. Changes to this field are automatically reapplied to the created PVCs. Size cannot be decreased. resizeInUseVolumes bool Resize existent PVCs, defaults to true pvcTemplate core/v1.PersistentVolumeClaimSpec Template to be used to generate the Persistent Volume Claim","title":"StorageConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SubscriptionReclaimPolicy","text":"(Alias of string ) Appears in: SubscriptionSpec SubscriptionReclaimPolicy describes a policy for end-of-life maintenance of Subscriptions.","title":"SubscriptionReclaimPolicy"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SubscriptionSpec","text":"Appears in: Subscription SubscriptionSpec defines the desired state of Subscription Field Description cluster [Required] core/v1.LocalObjectReference The name of the PostgreSQL cluster that identifies the \"subscriber\" name [Required] string The name of the subscription inside PostgreSQL dbname [Required] string The name of the database where the publication will be installed in the \"subscriber\" cluster parameters map[string]string Subscription parameters included in the WITH clause of the PostgreSQL CREATE SUBSCRIPTION command. Most parameters cannot be changed after the subscription is created and will be ignored if modified later, except for a limited set documented at: https://www.postgresql.org/docs/current/sql-altersubscription.html#SQL-ALTERSUBSCRIPTION-PARAMS-SET publicationName [Required] string The name of the publication inside the PostgreSQL database in the \"publisher\" publicationDBName string The name of the database containing the publication on the external cluster. Defaults to the one in the external cluster definition. externalClusterName [Required] string The name of the external cluster with the publication (\"publisher\") subscriptionReclaimPolicy SubscriptionReclaimPolicy The policy for end-of-life maintenance of this subscription","title":"SubscriptionSpec"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SubscriptionStatus","text":"Appears in: Subscription SubscriptionStatus defines the observed state of Subscription Field Description observedGeneration int64 A sequence number representing the latest desired state that was synchronized applied bool Applied is true if the subscription was reconciled correctly message string Message is the reconciliation output message","title":"SubscriptionStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SwitchReplicaClusterStatus","text":"Appears in: ClusterStatus SwitchReplicaClusterStatus contains all the statuses regarding the switch of a cluster to a replica cluster Field Description inProgress bool InProgress indicates if there is an ongoing procedure of switching a cluster to a replica cluster.","title":"SwitchReplicaClusterStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SyncReplicaElectionConstraints","text":"Appears in: PostgresConfiguration SyncReplicaElectionConstraints contains the constraints for sync replicas election. For anti-affinity parameters two instances are considered in the same location if all the labels values match. In future synchronous replica election restriction by name will be supported. Field Description nodeLabelsAntiAffinity []string A list of node labels values to extract and compare to evaluate if the pods reside in the same topology or not enabled [Required] bool This flag enables the constraints for sync replicas","title":"SyncReplicaElectionConstraints"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SynchronizeReplicasConfiguration","text":"Appears in: ReplicationSlotsConfiguration SynchronizeReplicasConfiguration contains the configuration for the synchronization of user defined physical replication slots Field Description enabled [Required] bool When set to true, every replication slot that is on the primary is synchronized on each standby excludePatterns []string List of regular expression patterns to match the names of replication slots to be excluded (by default empty)","title":"SynchronizeReplicasConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SynchronousReplicaConfiguration","text":"Appears in: PostgresConfiguration SynchronousReplicaConfiguration contains the configuration of the PostgreSQL synchronous replication feature. Important: at this moment, also .spec.minSyncReplicas and .spec.maxSyncReplicas need to be considered. Field Description method [Required] SynchronousReplicaConfigurationMethod Method to select synchronous replication standbys from the listed servers, accepting 'any' (quorum-based synchronous replication) or 'first' (priority-based synchronous replication) as values. number [Required] int Specifies the number of synchronous standby servers that transactions must wait for responses from. maxStandbyNamesFromCluster int Specifies the maximum number of local cluster pods that can be automatically included in the synchronous_standby_names option in PostgreSQL. standbyNamesPre []string A user-defined list of application names to be added to synchronous_standby_names before local cluster pods (the order is only useful for priority-based synchronous replication). standbyNamesPost []string A user-defined list of application names to be added to synchronous_standby_names after local cluster pods (the order is only useful for priority-based synchronous replication). dataDurability DataDurabilityLevel If set to \"required\", data durability is strictly enforced. Write operations with synchronous commit settings ( on , remote_write , or remote_apply ) will block if there are insufficient healthy replicas, ensuring data persistence. If set to \"preferred\", data durability is maintained when healthy replicas are available, but the required number of instances will adjust dynamically if replicas become unavailable. This setting relaxes strict durability enforcement to allow for operational continuity. This setting is only applicable if both standbyNamesPre and standbyNamesPost are unset (empty).","title":"SynchronousReplicaConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-SynchronousReplicaConfigurationMethod","text":"(Alias of string ) Appears in: SynchronousReplicaConfiguration SynchronousReplicaConfigurationMethod configures whether to use quorum based replication or a priority list","title":"SynchronousReplicaConfigurationMethod"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-TablespaceConfiguration","text":"Appears in: ClusterSpec TablespaceConfiguration is the configuration of a tablespace, and includes the storage specification for the tablespace Field Description name [Required] string The name of the tablespace storage [Required] StorageConfiguration The storage configuration for the tablespace owner DatabaseRoleRef Owner is the PostgreSQL user owning the tablespace temporary bool When set to true, the tablespace will be added as a temp_tablespaces entry in PostgreSQL, and will be available to automatically house temp database objects, or other temporary files. Please refer to PostgreSQL documentation for more information on the temp_tablespaces GUC.","title":"TablespaceConfiguration"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-TablespaceState","text":"Appears in: ClusterStatus TablespaceState represents the state of a tablespace in a cluster Field Description name [Required] string Name is the name of the tablespace owner string Owner is the PostgreSQL user owning the tablespace state [Required] TablespaceStatus State is the latest reconciliation state error string Error is the reconciliation error, if any","title":"TablespaceState"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-TablespaceStatus","text":"(Alias of string ) Appears in: TablespaceState TablespaceStatus represents the status of a tablespace in the cluster","title":"TablespaceStatus"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-Topology","text":"Appears in: ClusterStatus Topology contains the cluster topology Field Description instances map[PodName]PodTopologyLabels Instances contains the pod topology of the instances nodesUsed int32 NodesUsed represents the count of distinct nodes accommodating the instances. A value of '1' suggests that all instances are hosted on a single node, implying the absence of High Availability (HA). Ideally, this value should be the same as the number of instances in the Postgres HA cluster, implying shared nothing architecture on the compute side. successfullyExtracted bool SuccessfullyExtracted indicates if the topology data was extract. It is useful to enact fallback behaviors in synchronous replica election in case of failures","title":"Topology"},{"location":"cloudnative-pg.v1/#postgresql-cnpg-io-v1-VolumeSnapshotConfiguration","text":"Appears in: BackupConfiguration VolumeSnapshotConfiguration represents the configuration for the execution of snapshot backups. Field Description labels map[string]string Labels are key-value pairs that will be added to .metadata.labels snapshot resources. annotations map[string]string Annotations key-value pairs that will be added to .metadata.annotations snapshot resources. className string ClassName specifies the Snapshot Class to be used for PG_DATA PersistentVolumeClaim. It is the default class for the other types if no specific class is present walClassName string WalClassName specifies the Snapshot Class to be used for the PG_WAL PersistentVolumeClaim. tablespaceClassName map[string]string TablespaceClassName specifies the Snapshot Class to be used for the tablespaces. defaults to the PGDATA Snapshot Class, if set snapshotOwnerReference SnapshotOwnerReference SnapshotOwnerReference indicates the type of owner reference the snapshot should have online bool Whether the default type of backup with volume snapshots is online/hot ( true , default) or offline/cold ( false ) onlineConfiguration OnlineConfiguration Configuration parameters to control the online/hot backup with volume snapshots","title":"VolumeSnapshotConfiguration"},{"location":"cluster_conf/","text":"Instance pod configuration Projected volumes CloudNativePG supports mounting custom files inside the Postgres pods through .spec.projectedVolumeTemplate . This ability is useful for several Postgres features and extensions that require additional data files. In CloudNativePG, the .spec.projectedVolumeTemplate field is a projected volume template in Kubernetes that allows you to mount arbitrary data under the /projected folder in Postgres pods. This simple example shows how to mount an existing TLS secret (named sample-secret ) as files into Postgres pods. The values for the secret keys tls.crt and tls.key in sample-secret are mounted as files into the paths /projected/certificate/tls.crt and /projected/certificate/tls.key in the Postgres pod. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-projected-volumes spec: instances: 3 projectedVolumeTemplate: sources: - secret: name: sample-secret items: - key: tls.crt path: certificate/tls.crt - key: tls.key path: certificate/tls.key storage: size: 1Gi You can find a complete example that uses a projected volume template to mount the secret and ConfigMap in the cluster-example-projected-volume.yaml deployment manifest. Ephemeral volumes CloudNativePG relies on ephemeral volumes for part of the internal activities. Ephemeral volumes exist for the sole duration of a pod's life, without persisting across pod restarts. Volume Claim Template for Temporary Storage The operator uses by default an emptyDir volume, which can be customized by using the .spec.ephemeralVolumesSizeLimit field . This can be overridden by specifying a volume claim template in the .spec.ephemeralVolumeSource field. In the following example, a 1Gi ephemeral volume is set. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-ephemeral-volume-source spec: instances: 3 ephemeralVolumeSource: volumeClaimTemplate: spec: accessModes: [\"ReadWriteOnce\"] # example storageClassName, replace with one existing in your Kubernetes cluster storageClassName: \"scratch-storage-class\" resources: requests: storage: 1Gi Both .spec.emphemeralVolumeSource and .spec.ephemeralVolumesSizeLimit.temporaryData cannot be specified simultaneously. Volume for shared memory This volume is used as shared memory space for Postgres and as an ephemeral type but stored in memory. You can configure an upper bound on the size using the .spec.ephemeralVolumesSizeLimit.shm field in the cluster spec. Use this field only in case of PostgreSQL running with posix shared memory dynamic allocation . Environment variables You can customize some system behavior using environment variables. One example is the LDAPCONF variable, which can point to a custom LDAP configuration file. Another example is the TZ environment variable, which represents the timezone used by the PostgreSQL container. CloudNativePG allows you to set custom environment variables using the env and the envFrom stanza of the cluster specification. This example defines a PostgreSQL cluster using the Australia/Sydney timezone as the default cluster-level timezone: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 3 env: - name: TZ value: Australia/Sydney storage: size: 1Gi The envFrom stanza can refer to ConfigMaps or secrets to use their content as environment variables: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 3 envFrom: - configMapRef: name: config-map-name - secretRef: name: secret-name storage: size: 1Gi The operator doesn't allow setting the following environment variables: POD_NAME NAMESPACE Any environment variable whose name starts with PG . Any change in the env or in the envFrom section triggers a rolling update of the PostgreSQL pods. If the env or the envFrom section refers to a secret or a ConfigMap, the operator doesn't detect any changes in them and doesn't trigger a rollout. The kubelet uses the same behavior with pods, and you must trigger the pod rollout manually.","title":"Instance pod configuration"},{"location":"cluster_conf/#instance-pod-configuration","text":"","title":"Instance pod configuration"},{"location":"cluster_conf/#projected-volumes","text":"CloudNativePG supports mounting custom files inside the Postgres pods through .spec.projectedVolumeTemplate . This ability is useful for several Postgres features and extensions that require additional data files. In CloudNativePG, the .spec.projectedVolumeTemplate field is a projected volume template in Kubernetes that allows you to mount arbitrary data under the /projected folder in Postgres pods. This simple example shows how to mount an existing TLS secret (named sample-secret ) as files into Postgres pods. The values for the secret keys tls.crt and tls.key in sample-secret are mounted as files into the paths /projected/certificate/tls.crt and /projected/certificate/tls.key in the Postgres pod. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-projected-volumes spec: instances: 3 projectedVolumeTemplate: sources: - secret: name: sample-secret items: - key: tls.crt path: certificate/tls.crt - key: tls.key path: certificate/tls.key storage: size: 1Gi You can find a complete example that uses a projected volume template to mount the secret and ConfigMap in the cluster-example-projected-volume.yaml deployment manifest.","title":"Projected volumes"},{"location":"cluster_conf/#ephemeral-volumes","text":"CloudNativePG relies on ephemeral volumes for part of the internal activities. Ephemeral volumes exist for the sole duration of a pod's life, without persisting across pod restarts.","title":"Ephemeral volumes"},{"location":"cluster_conf/#volume-claim-template-for-temporary-storage","text":"The operator uses by default an emptyDir volume, which can be customized by using the .spec.ephemeralVolumesSizeLimit field . This can be overridden by specifying a volume claim template in the .spec.ephemeralVolumeSource field. In the following example, a 1Gi ephemeral volume is set. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-ephemeral-volume-source spec: instances: 3 ephemeralVolumeSource: volumeClaimTemplate: spec: accessModes: [\"ReadWriteOnce\"] # example storageClassName, replace with one existing in your Kubernetes cluster storageClassName: \"scratch-storage-class\" resources: requests: storage: 1Gi Both .spec.emphemeralVolumeSource and .spec.ephemeralVolumesSizeLimit.temporaryData cannot be specified simultaneously.","title":"Volume Claim Template for Temporary Storage"},{"location":"cluster_conf/#volume-for-shared-memory","text":"This volume is used as shared memory space for Postgres and as an ephemeral type but stored in memory. You can configure an upper bound on the size using the .spec.ephemeralVolumesSizeLimit.shm field in the cluster spec. Use this field only in case of PostgreSQL running with posix shared memory dynamic allocation .","title":"Volume for shared memory"},{"location":"cluster_conf/#environment-variables","text":"You can customize some system behavior using environment variables. One example is the LDAPCONF variable, which can point to a custom LDAP configuration file. Another example is the TZ environment variable, which represents the timezone used by the PostgreSQL container. CloudNativePG allows you to set custom environment variables using the env and the envFrom stanza of the cluster specification. This example defines a PostgreSQL cluster using the Australia/Sydney timezone as the default cluster-level timezone: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 3 env: - name: TZ value: Australia/Sydney storage: size: 1Gi The envFrom stanza can refer to ConfigMaps or secrets to use their content as environment variables: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 3 envFrom: - configMapRef: name: config-map-name - secretRef: name: secret-name storage: size: 1Gi The operator doesn't allow setting the following environment variables: POD_NAME NAMESPACE Any environment variable whose name starts with PG . Any change in the env or in the envFrom section triggers a rolling update of the PostgreSQL pods. If the env or the envFrom section refers to a secret or a ConfigMap, the operator doesn't detect any changes in them and doesn't trigger a rollout. The kubelet uses the same behavior with pods, and you must trigger the pod rollout manually.","title":"Environment variables"},{"location":"connection_pooling/","text":"Connection pooling CloudNativePG provides native support for connection pooling with PgBouncer , one of the most popular open source connection poolers for PostgreSQL, through the Pooler custom resource definition (CRD). In brief, a pooler in CloudNativePG is a deployment of PgBouncer pods that sits between your applications and a PostgreSQL service, for example, the rw service. It creates a separate, scalable, configurable, and highly available database access layer. Architecture The following diagram highlights how introducing a database access layer based on PgBouncer changes the architecture of CloudNativePG. Instead of directly connecting to the PostgreSQL primary service, applications can connect to the equivalent service for PgBouncer. This ability enables reuse of existing connections for faster performance and better resource management on the PostgreSQL side. Quick start This example helps to show how CloudNativePG implements a PgBouncer pooler: apiVersion: postgresql.cnpg.io/v1 kind: Pooler metadata: name: pooler-example-rw spec: cluster: name: cluster-example instances: 3 type: rw pgbouncer: poolMode: session parameters: max_client_conn: \"1000\" default_pool_size: \"10\" Important The pooler name can't be the same as any cluster name in the same namespace. This example creates a Pooler resource called pooler-example-rw that's strictly associated with the Postgres Cluster resource called cluster-example . It points to the primary, identified by the read/write service ( rw , therefore cluster-example-rw ). The Pooler resource must live in the same namespace as the Postgres cluster. It consists of a Kubernetes deployment of 3 pods running the latest stable image of PgBouncer , configured with the session pooling mode and accepting up to 1000 connections each. The default pool size is 10 user/database pairs toward PostgreSQL. Important The Pooler resource sets only the * fallback database in PgBouncer. This setting means that that all parameters in the connection strings passed from the client are relayed to the PostgreSQL server. For details, see \"Section [databases]\" in the PgBouncer documentation . CloudNativePG also creates a secret with the same name as the pooler containing the configuration files used with PgBouncer. API reference For details, see PgBouncerSpec in the API reference. Pooler resource lifecycle Pooler resources aren't cluster-managed resources. You create poolers manually when they're needed. You can also deploy multiple poolers per PostgreSQL cluster. What's important is that the life cycles of the Cluster and the Pooler resources are currently independent. Deleting the cluster doesn't imply the deletion of the pooler, and vice versa. Important Once you know how a pooler works, you have full freedom in terms of possible architectures. You can have clusters without poolers, clusters with a single pooler, or clusters with several poolers, that is, one per application. Important When the operator is upgraded, the pooler pods will undergo a rolling upgrade. This is necessary to ensure that the instance manager within the pooler pods is also upgraded. Security Any PgBouncer pooler is transparently integrated with CloudNativePG support for in-transit encryption by way of TLS connections, both on the client (application) and server (PostgreSQL) side of the pool. Specifically, PgBouncer reuses the certificates of the PostgreSQL server. It also uses TLS client certificate authentication to connect to the PostgreSQL server to run the auth_query for clients' password authentication (see Authentication ). Containers run as the pgbouncer system user, and access to the pgbouncer database is allowed only by way of local connections, through peer authentication. Certificates By default, a PgBouncer pooler uses the same certificates that are used by the cluster. However, if you provide those certificates, the pooler accepts secrets with the following formats: Basic Auth TLS Opaque In the Opaque case, it looks for the following specific keys that need to be used: tls.crt tls.key So you can treat this secret as a TLS secret, and start from there. Authentication Password-based authentication is the only supported method for clients of PgBouncer in CloudNativePG. Internally, the implementation relies on PgBouncer's auth_user and auth_query options. Specifically, the operator: Creates a standard user called cnpg_pooler_pgbouncer in the PostgreSQL server Creates the lookup function in the postgres database and grants execution privileges to the cnpg_pooler_pgbouncer user (PoLA) Issues a TLS certificate for this user Sets cnpg_pooler_pgbouncer as the auth_user Configures PgBouncer to use the TLS certificate to authenticate cnpg_pooler_pgbouncer against the PostgreSQL server Removes all the above when it detects that a cluster doesn't have any pooler associated to it Important If you specify your own secrets, the operator doesn't automatically integrate the pooler. To manually integrate the pooler, if you specified your own secrets, you must run the following queries from inside your cluster. First, you must create the role: CREATE ROLE cnpg_pooler_pgbouncer WITH LOGIN; Then, for each application database, grant the permission for cnpg_pooler_pgbouncer to connect to it: GRANT CONNECT ON DATABASE { database name here } TO cnpg_pooler_pgbouncer; Finally, as a superuser connect in each application database, and then create the authentication function inside each of the application databases: CREATE OR REPLACE FUNCTION public.user_search(uname TEXT) RETURNS TABLE (usename name, passwd text) LANGUAGE sql SECURITY DEFINER AS 'SELECT usename, passwd FROM pg_catalog.pg_shadow WHERE usename=$1;'; REVOKE ALL ON FUNCTION public.user_search(text) FROM public; GRANT EXECUTE ON FUNCTION public.user_search(text) TO cnpg_pooler_pgbouncer; Important Given that user_search is a SECURITY DEFINER function, you need to create it through a role with SUPERUSER privileges, such as the postgres user. Pod templates You can take advantage of pod templates specification in the template section of a Pooler resource. For details, see PoolerSpec in the API reference. Using templates, you can configure pods as you like, including fine control over affinity and anti-affinity rules for pods and nodes. By default, containers use images from ghcr.io/cloudnative-pg/pgbouncer . This example shows Pooler specifying `PodAntiAffinity``: apiVersion: postgresql.cnpg.io/v1 kind: Pooler metadata: name: pooler-example-rw spec: cluster: name: cluster-example instances: 3 type: rw template: metadata: labels: app: pooler spec: containers: [] affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - pooler topologyKey: \"kubernetes.io/hostname\" Note Explicitly set .spec.template.spec.containers to [] when not modified, as it's a required field for a PodSpec . If .spec.template.spec.containers isn't set, the Kubernetes api-server returns the following error when trying to apply the manifest: error validating \"pooler.yaml\": error validating data: ValidationError(Pooler.spec.template.spec): missing required field \"containers\" This example sets resources and changes the used image: apiVersion: postgresql.cnpg.io/v1 kind: Pooler metadata: name: pooler-example-rw spec: cluster: name: cluster-example instances: 3 type: rw template: metadata: labels: app: pooler spec: containers: - name: pgbouncer image: my-pgbouncer:latest resources: requests: cpu: \"0.1\" memory: 100Mi limits: cpu: \"0.5\" memory: 500Mi Service Template Sometimes, your pooler will require some different labels, annotations, or even change the type of the service, you can achieve that by using the serviceTemplate field: apiVersion: postgresql.cnpg.io/v1 kind: Pooler metadata: name: pooler-example-rw spec: cluster: name: cluster-example instances: 3 type: rw serviceTemplate: metadata: labels: app: pooler spec: type: LoadBalancer pgbouncer: poolMode: session parameters: max_client_conn: \"1000\" default_pool_size: \"10\" The operator by default adds a ServicePort with the following data: ports: - name: pgbouncer port: 5432 protocol: TCP targetPort: pgbouncer Warning Specifying a ServicePort with the name pgbouncer or the port 5432 will prevent the default ServicePort from being added. This because ServicePort entries with the same name or port are not allowed on Kubernetes and result in errors. High availability (HA) Because of Kubernetes' deployments, you can configure your pooler to run on a single instance or over multiple pods. The exposed service makes sure that your clients are randomly distributed over the available pods running PgBouncer, which then manages and reuses connections toward the underlying server (if using the rw service) or servers (if using the ro service with multiple replicas). Warning If your infrastructure spans multiple availability zones with high latency across them, be aware of network hops. Consider, for example, the case of your application running in zone 2, connecting to PgBouncer running in zone 3, and pointing to the PostgreSQL primary in zone 1. PgBouncer configuration options The operator manages most of the configuration options for PgBouncer , allowing you to modify only a subset of them. Warning You are responsible for correctly setting the value of each option, as the operator doesn't validate them. These are the PgBouncer options you can customize, with links to the PgBouncer documentation for each parameter. Unless stated otherwise, the default values are the ones directly set by PgBouncer. application_name_add_host autodb_idle_timeout cancel_wait_timeout client_idle_timeout client_login_timeout default_pool_size disable_pqexec dns_max_ttl dns_nxdomain_ttl idle_transaction_timeout ignore_startup_parameters : to be appended to extra_float_digits,options - required by CloudNativePG listen_backlog log_connections log_disconnections log_pooler_errors log_stats : by default disabled ( 0 ), given that statistics are already collected by the Prometheus export as described in the \"Monitoring\" section below max_client_conn max_db_connections max_packet_size max_prepared_statements max_user_connections min_pool_size pkt_buf query_timeout query_wait_timeout reserve_pool_size reserve_pool_timeout sbuf_loopcnt server_check_delay server_check_query server_connect_timeout server_fast_close server_idle_timeout server_lifetime server_login_retry server_reset_query server_reset_query_always server_round_robin server_tls_ciphers server_tls_protocols stats_period suspend_timeout tcp_defer_accept tcp_keepalive tcp_keepcnt tcp_keepidle tcp_keepintvl tcp_user_timeout tcp_socket_buffer track_extra_parameters verbose Customizations of the PgBouncer configuration are written declaratively in the .spec.pgbouncer.parameters map. The operator reacts to the changes in the pooler specification, and every PgBouncer instance reloads the updated configuration without disrupting the service. Warning Every PgBouncer pod has the same configuration, aligned with the parameters in the specification. A mistake in these parameters might disrupt the operability of the whole pooler. The operator doesn't validate the value of any option. Monitoring The PgBouncer implementation of the Pooler comes with a default Prometheus exporter. It makes available several metrics having the cnpg_pgbouncer_ prefix by running: SHOW LISTS (prefix: cnpg_pgbouncer_lists ) SHOW POOLS (prefix: cnpg_pgbouncer_pools ) SHOW STATS (prefix: cnpg_pgbouncer_stats ) Like the CloudNativePG instance, the exporter runs on port 9127 of each pod running PgBouncer and also provides metrics related to the Go runtime (with the prefix go_* ). Info You can inspect the exported metrics on a pod running PgBouncer. For instructions, see How to inspect the exported metrics . Make sure that you use the correct IP and the 9127 port. This example shows the output for cnpg_pgbouncer metrics: # HELP cnpg_pgbouncer_collection_duration_seconds Collection time duration in seconds # TYPE cnpg_pgbouncer_collection_duration_seconds gauge cnpg_pgbouncer_collection_duration_seconds{collector=\"Collect.up\"} 0.002338805 # HELP cnpg_pgbouncer_collection_errors_total Total errors occurred accessing PostgreSQL for metrics. # TYPE cnpg_pgbouncer_collection_errors_total counter cnpg_pgbouncer_collection_errors_total{collector=\"sql: Scan error on column index 16, name \\\"load_balance_hosts\\\": converting NULL to int is unsupported\"} 5 # HELP cnpg_pgbouncer_collections_total Total number of times PostgreSQL was accessed for metrics. # TYPE cnpg_pgbouncer_collections_total counter cnpg_pgbouncer_collections_total 5 # HELP cnpg_pgbouncer_last_collection_error 1 if the last collection ended with error, 0 otherwise. # TYPE cnpg_pgbouncer_last_collection_error gauge cnpg_pgbouncer_last_collection_error 0 # HELP cnpg_pgbouncer_lists_databases Count of databases. # TYPE cnpg_pgbouncer_lists_databases gauge cnpg_pgbouncer_lists_databases 1 # HELP cnpg_pgbouncer_lists_dns_names Count of DNS names in the cache. # TYPE cnpg_pgbouncer_lists_dns_names gauge cnpg_pgbouncer_lists_dns_names 0 # HELP cnpg_pgbouncer_lists_dns_pending Not used. # TYPE cnpg_pgbouncer_lists_dns_pending gauge cnpg_pgbouncer_lists_dns_pending 0 # HELP cnpg_pgbouncer_lists_dns_queries Count of in-flight DNS queries. # TYPE cnpg_pgbouncer_lists_dns_queries gauge cnpg_pgbouncer_lists_dns_queries 0 # HELP cnpg_pgbouncer_lists_dns_zones Count of DNS zones in the cache. # TYPE cnpg_pgbouncer_lists_dns_zones gauge cnpg_pgbouncer_lists_dns_zones 0 # HELP cnpg_pgbouncer_lists_free_clients Count of free clients. # TYPE cnpg_pgbouncer_lists_free_clients gauge cnpg_pgbouncer_lists_free_clients 49 # HELP cnpg_pgbouncer_lists_free_servers Count of free servers. # TYPE cnpg_pgbouncer_lists_free_servers gauge cnpg_pgbouncer_lists_free_servers 0 # HELP cnpg_pgbouncer_lists_login_clients Count of clients in login state. # TYPE cnpg_pgbouncer_lists_login_clients gauge cnpg_pgbouncer_lists_login_clients 0 # HELP cnpg_pgbouncer_lists_pools Count of pools. # TYPE cnpg_pgbouncer_lists_pools gauge cnpg_pgbouncer_lists_pools 1 # HELP cnpg_pgbouncer_lists_used_clients Count of used clients. # TYPE cnpg_pgbouncer_lists_used_clients gauge cnpg_pgbouncer_lists_used_clients 1 # HELP cnpg_pgbouncer_lists_used_servers Count of used servers. # TYPE cnpg_pgbouncer_lists_used_servers gauge cnpg_pgbouncer_lists_used_servers 0 # HELP cnpg_pgbouncer_lists_users Count of users. # TYPE cnpg_pgbouncer_lists_users gauge cnpg_pgbouncer_lists_users 2 # HELP cnpg_pgbouncer_pools_cl_active Client connections that are linked to server connection and can process queries. # TYPE cnpg_pgbouncer_pools_cl_active gauge cnpg_pgbouncer_pools_cl_active{database=\"pgbouncer\",user=\"pgbouncer\"} 1 # HELP cnpg_pgbouncer_pools_cl_active_cancel_req Client connections that have forwarded query cancellations to the server and are waiting for the server response. # TYPE cnpg_pgbouncer_pools_cl_active_cancel_req gauge cnpg_pgbouncer_pools_cl_active_cancel_req{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_cl_cancel_req Client connections that have not forwarded query cancellations to the server yet. # TYPE cnpg_pgbouncer_pools_cl_cancel_req gauge cnpg_pgbouncer_pools_cl_cancel_req{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_cl_waiting Client connections that have sent queries but have not yet got a server connection. # TYPE cnpg_pgbouncer_pools_cl_waiting gauge cnpg_pgbouncer_pools_cl_waiting{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_cl_waiting_cancel_req Client connections that have not forwarded query cancellations to the server yet. # TYPE cnpg_pgbouncer_pools_cl_waiting_cancel_req gauge cnpg_pgbouncer_pools_cl_waiting_cancel_req{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_load_balance_hosts Number of hosts not load balancing between hosts # TYPE cnpg_pgbouncer_pools_load_balance_hosts gauge cnpg_pgbouncer_pools_load_balance_hosts{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_maxwait How long the first (oldest) client in the queue has waited, in seconds. If this starts increasing, then the current pool of servers does not handle requests quickly enough. The reason may be either an overloaded server or just too small of a pool_size setting. # TYPE cnpg_pgbouncer_pools_maxwait gauge cnpg_pgbouncer_pools_maxwait{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_maxwait_us Microsecond part of the maximum waiting time. # TYPE cnpg_pgbouncer_pools_maxwait_us gauge cnpg_pgbouncer_pools_maxwait_us{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_pool_mode The pooling mode in use. 1 for session, 2 for transaction, 3 for statement, -1 if unknown # TYPE cnpg_pgbouncer_pools_pool_mode gauge cnpg_pgbouncer_pools_pool_mode{database=\"pgbouncer\",user=\"pgbouncer\"} 3 # HELP cnpg_pgbouncer_pools_sv_active Server connections that are linked to a client. # TYPE cnpg_pgbouncer_pools_sv_active gauge cnpg_pgbouncer_pools_sv_active{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_active_cancel Server connections that are currently forwarding a cancel request # TYPE cnpg_pgbouncer_pools_sv_active_cancel gauge cnpg_pgbouncer_pools_sv_active_cancel{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_idle Server connections that are unused and immediately usable for client queries. # TYPE cnpg_pgbouncer_pools_sv_idle gauge cnpg_pgbouncer_pools_sv_idle{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_login Server connections currently in the process of logging in. # TYPE cnpg_pgbouncer_pools_sv_login gauge cnpg_pgbouncer_pools_sv_login{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_tested Server connections that are currently running either server_reset_query or server_check_query. # TYPE cnpg_pgbouncer_pools_sv_tested gauge cnpg_pgbouncer_pools_sv_tested{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_used Server connections that have been idle for more than server_check_delay, so they need server_check_query to run on them before they can be used again. # TYPE cnpg_pgbouncer_pools_sv_used gauge cnpg_pgbouncer_pools_sv_used{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_wait_cancels Servers that normally could become idle, but are waiting to do so until all in-flight cancel requests have completed that were sent to cancel a query on this server. # TYPE cnpg_pgbouncer_pools_sv_wait_cancels gauge cnpg_pgbouncer_pools_sv_wait_cancels{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_bind_count Average number of prepared statements readied for execution by clients and forwarded to PostgreSQL by pgbouncer. # TYPE cnpg_pgbouncer_stats_avg_bind_count gauge cnpg_pgbouncer_stats_avg_bind_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_client_parse_count Average number of prepared statements created by clients. # TYPE cnpg_pgbouncer_stats_avg_client_parse_count gauge cnpg_pgbouncer_stats_avg_client_parse_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_query_count Average queries per second in last stat period. # TYPE cnpg_pgbouncer_stats_avg_query_count gauge cnpg_pgbouncer_stats_avg_query_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_query_time Average query duration, in microseconds. # TYPE cnpg_pgbouncer_stats_avg_query_time gauge cnpg_pgbouncer_stats_avg_query_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_recv Average received (from clients) bytes per second. # TYPE cnpg_pgbouncer_stats_avg_recv gauge cnpg_pgbouncer_stats_avg_recv{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_sent Average sent (to clients) bytes per second. # TYPE cnpg_pgbouncer_stats_avg_sent gauge cnpg_pgbouncer_stats_avg_sent{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_server_parse_count Average number of prepared statements created by pgbouncer on a server. # TYPE cnpg_pgbouncer_stats_avg_server_parse_count gauge cnpg_pgbouncer_stats_avg_server_parse_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_wait_time Time spent by clients waiting for a server, in microseconds (average per second). # TYPE cnpg_pgbouncer_stats_avg_wait_time gauge cnpg_pgbouncer_stats_avg_wait_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_xact_count Average transactions per second in last stat period. # TYPE cnpg_pgbouncer_stats_avg_xact_count gauge cnpg_pgbouncer_stats_avg_xact_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_xact_time Average transaction duration, in microseconds. # TYPE cnpg_pgbouncer_stats_avg_xact_time gauge cnpg_pgbouncer_stats_avg_xact_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_bind_count Total number of prepared statements readied for execution by clients and forwarded to PostgreSQL by pgbouncer # TYPE cnpg_pgbouncer_stats_total_bind_count gauge cnpg_pgbouncer_stats_total_bind_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_client_parse_count Total number of prepared statements created by clients. # TYPE cnpg_pgbouncer_stats_total_client_parse_count gauge cnpg_pgbouncer_stats_total_client_parse_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_query_count Total number of SQL queries pooled by pgbouncer. # TYPE cnpg_pgbouncer_stats_total_query_count gauge cnpg_pgbouncer_stats_total_query_count{database=\"pgbouncer\"} 15 # HELP cnpg_pgbouncer_stats_total_query_time Total number of microseconds spent by pgbouncer when actively connected to PostgreSQL, executing queries. # TYPE cnpg_pgbouncer_stats_total_query_time gauge cnpg_pgbouncer_stats_total_query_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_received Total volume in bytes of network traffic received by pgbouncer. # TYPE cnpg_pgbouncer_stats_total_received gauge cnpg_pgbouncer_stats_total_received{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_sent Total volume in bytes of network traffic sent by pgbouncer. # TYPE cnpg_pgbouncer_stats_total_sent gauge cnpg_pgbouncer_stats_total_sent{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_server_parse_count Total number of prepared statements created by pgbouncer on a server. # TYPE cnpg_pgbouncer_stats_total_server_parse_count gauge cnpg_pgbouncer_stats_total_server_parse_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_wait_time Time spent by clients waiting for a server, in microseconds. # TYPE cnpg_pgbouncer_stats_total_wait_time gauge cnpg_pgbouncer_stats_total_wait_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_xact_count Total number of SQL transactions pooled by pgbouncer. # TYPE cnpg_pgbouncer_stats_total_xact_count gauge cnpg_pgbouncer_stats_total_xact_count{database=\"pgbouncer\"} 15 # HELP cnpg_pgbouncer_stats_total_xact_time Total number of microseconds spent by pgbouncer when connected to PostgreSQL in a transaction, either idle in transaction or executing queries. # TYPE cnpg_pgbouncer_stats_total_xact_time gauge cnpg_pgbouncer_stats_total_xact_time{database=\"pgbouncer\"} 0 Info For a better understanding of the metrics please refer to the PgBouncer documentation. As for clusters, a specific pooler can be monitored using the Prometheus operator's resource PodMonitor . A PodMonitor correctly pointing to a pooler can be created by the operator by setting .spec.monitoring.enablePodMonitor to true in the Pooler resource. The default is false . Important Any change to PodMonitor created automatically is overridden by the operator at the next reconciliation cycle. If you need to customize it, you can do so as shown in the following example. To deploy a PodMonitor for a specific pooler manually, you can define it as follows and change it as needed: apiVersion: monitoring.coreos.com/v1 kind: PodMonitor metadata: name: spec: selector: matchLabels: cnpg.io/poolerName: podMetricsEndpoints: - port: metrics Logging Logs are directly sent to standard output, in JSON format, like in the following example: { \"level\": \"info\", \"ts\": SECONDS.MICROSECONDS, \"msg\": \"record\", \"pipe\": \"stderr\", \"record\": { \"timestamp\": \"YYYY-MM-DD HH:MM:SS.MS UTC\", \"pid\": \"\", \"level\": \"LOG\", \"msg\": \"kernel file descriptor limit: 1048576 (hard: 1048576); max_client_conn: 100, max expected fd use: 112\" } } Pausing connections The Pooler specification allows you to take advantage of PgBouncer's PAUSE and RESUME commands, using only declarative configuration. You can ado this using the paused option, which by default is set to false . When set to true , the operator internally invokes the PAUSE command in PgBouncer, which: Closes all active connections toward the PostgreSQL server, after waiting for the queries to complete Pauses any new connection coming from the client When the paused option is reset to false , the operator invokes the RESUME command in PgBouncer, reopening the taps toward the PostgreSQL service defined in the Pooler resource. PAUSE For more information, see PAUSE in the PgBouncer documentation . Important In future versions, the switchover operation will be fully integrated with the PgBouncer pooler and take advantage of the PAUSE / RESUME features to reduce the perceived downtime by client applications. Currently, you can achieve the same results by setting the paused attribute to true , issuing the switchover command through the cnpg plugin , and then restoring the paused attribute to false . Limitations Single PostgreSQL cluster The current implementation of the pooler is designed to work as part of a specific CloudNativePG cluster (a service). It isn't currently possible to create a pooler that spans multiple clusters. Controlled configurability CloudNativePG transparently manages several configuration options that are used for the PgBouncer layer to communicate with PostgreSQL. Such options aren't configurable from outside and include TLS certificates, authentication settings, the databases section, and the users section. Also, considering the specific use case for the single PostgreSQL cluster, the adopted criteria is to explicitly list the options that can be configured by users. Note The adopted solution likely addresses the majority of use cases. It leaves room for the future implementation of a separate operator for PgBouncer to complete the gamma with more advanced and customized scenarios.","title":"Connection pooling"},{"location":"connection_pooling/#connection-pooling","text":"CloudNativePG provides native support for connection pooling with PgBouncer , one of the most popular open source connection poolers for PostgreSQL, through the Pooler custom resource definition (CRD). In brief, a pooler in CloudNativePG is a deployment of PgBouncer pods that sits between your applications and a PostgreSQL service, for example, the rw service. It creates a separate, scalable, configurable, and highly available database access layer.","title":"Connection pooling"},{"location":"connection_pooling/#architecture","text":"The following diagram highlights how introducing a database access layer based on PgBouncer changes the architecture of CloudNativePG. Instead of directly connecting to the PostgreSQL primary service, applications can connect to the equivalent service for PgBouncer. This ability enables reuse of existing connections for faster performance and better resource management on the PostgreSQL side.","title":"Architecture"},{"location":"connection_pooling/#quick-start","text":"This example helps to show how CloudNativePG implements a PgBouncer pooler: apiVersion: postgresql.cnpg.io/v1 kind: Pooler metadata: name: pooler-example-rw spec: cluster: name: cluster-example instances: 3 type: rw pgbouncer: poolMode: session parameters: max_client_conn: \"1000\" default_pool_size: \"10\" Important The pooler name can't be the same as any cluster name in the same namespace. This example creates a Pooler resource called pooler-example-rw that's strictly associated with the Postgres Cluster resource called cluster-example . It points to the primary, identified by the read/write service ( rw , therefore cluster-example-rw ). The Pooler resource must live in the same namespace as the Postgres cluster. It consists of a Kubernetes deployment of 3 pods running the latest stable image of PgBouncer , configured with the session pooling mode and accepting up to 1000 connections each. The default pool size is 10 user/database pairs toward PostgreSQL. Important The Pooler resource sets only the * fallback database in PgBouncer. This setting means that that all parameters in the connection strings passed from the client are relayed to the PostgreSQL server. For details, see \"Section [databases]\" in the PgBouncer documentation . CloudNativePG also creates a secret with the same name as the pooler containing the configuration files used with PgBouncer. API reference For details, see PgBouncerSpec in the API reference.","title":"Quick start"},{"location":"connection_pooling/#pooler-resource-lifecycle","text":"Pooler resources aren't cluster-managed resources. You create poolers manually when they're needed. You can also deploy multiple poolers per PostgreSQL cluster. What's important is that the life cycles of the Cluster and the Pooler resources are currently independent. Deleting the cluster doesn't imply the deletion of the pooler, and vice versa. Important Once you know how a pooler works, you have full freedom in terms of possible architectures. You can have clusters without poolers, clusters with a single pooler, or clusters with several poolers, that is, one per application. Important When the operator is upgraded, the pooler pods will undergo a rolling upgrade. This is necessary to ensure that the instance manager within the pooler pods is also upgraded.","title":"Pooler resource lifecycle"},{"location":"connection_pooling/#security","text":"Any PgBouncer pooler is transparently integrated with CloudNativePG support for in-transit encryption by way of TLS connections, both on the client (application) and server (PostgreSQL) side of the pool. Specifically, PgBouncer reuses the certificates of the PostgreSQL server. It also uses TLS client certificate authentication to connect to the PostgreSQL server to run the auth_query for clients' password authentication (see Authentication ). Containers run as the pgbouncer system user, and access to the pgbouncer database is allowed only by way of local connections, through peer authentication.","title":"Security"},{"location":"connection_pooling/#certificates","text":"By default, a PgBouncer pooler uses the same certificates that are used by the cluster. However, if you provide those certificates, the pooler accepts secrets with the following formats: Basic Auth TLS Opaque In the Opaque case, it looks for the following specific keys that need to be used: tls.crt tls.key So you can treat this secret as a TLS secret, and start from there.","title":"Certificates"},{"location":"connection_pooling/#authentication","text":"Password-based authentication is the only supported method for clients of PgBouncer in CloudNativePG. Internally, the implementation relies on PgBouncer's auth_user and auth_query options. Specifically, the operator: Creates a standard user called cnpg_pooler_pgbouncer in the PostgreSQL server Creates the lookup function in the postgres database and grants execution privileges to the cnpg_pooler_pgbouncer user (PoLA) Issues a TLS certificate for this user Sets cnpg_pooler_pgbouncer as the auth_user Configures PgBouncer to use the TLS certificate to authenticate cnpg_pooler_pgbouncer against the PostgreSQL server Removes all the above when it detects that a cluster doesn't have any pooler associated to it Important If you specify your own secrets, the operator doesn't automatically integrate the pooler. To manually integrate the pooler, if you specified your own secrets, you must run the following queries from inside your cluster. First, you must create the role: CREATE ROLE cnpg_pooler_pgbouncer WITH LOGIN; Then, for each application database, grant the permission for cnpg_pooler_pgbouncer to connect to it: GRANT CONNECT ON DATABASE { database name here } TO cnpg_pooler_pgbouncer; Finally, as a superuser connect in each application database, and then create the authentication function inside each of the application databases: CREATE OR REPLACE FUNCTION public.user_search(uname TEXT) RETURNS TABLE (usename name, passwd text) LANGUAGE sql SECURITY DEFINER AS 'SELECT usename, passwd FROM pg_catalog.pg_shadow WHERE usename=$1;'; REVOKE ALL ON FUNCTION public.user_search(text) FROM public; GRANT EXECUTE ON FUNCTION public.user_search(text) TO cnpg_pooler_pgbouncer; Important Given that user_search is a SECURITY DEFINER function, you need to create it through a role with SUPERUSER privileges, such as the postgres user.","title":"Authentication"},{"location":"connection_pooling/#pod-templates","text":"You can take advantage of pod templates specification in the template section of a Pooler resource. For details, see PoolerSpec in the API reference. Using templates, you can configure pods as you like, including fine control over affinity and anti-affinity rules for pods and nodes. By default, containers use images from ghcr.io/cloudnative-pg/pgbouncer . This example shows Pooler specifying `PodAntiAffinity``: apiVersion: postgresql.cnpg.io/v1 kind: Pooler metadata: name: pooler-example-rw spec: cluster: name: cluster-example instances: 3 type: rw template: metadata: labels: app: pooler spec: containers: [] affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - pooler topologyKey: \"kubernetes.io/hostname\" Note Explicitly set .spec.template.spec.containers to [] when not modified, as it's a required field for a PodSpec . If .spec.template.spec.containers isn't set, the Kubernetes api-server returns the following error when trying to apply the manifest: error validating \"pooler.yaml\": error validating data: ValidationError(Pooler.spec.template.spec): missing required field \"containers\" This example sets resources and changes the used image: apiVersion: postgresql.cnpg.io/v1 kind: Pooler metadata: name: pooler-example-rw spec: cluster: name: cluster-example instances: 3 type: rw template: metadata: labels: app: pooler spec: containers: - name: pgbouncer image: my-pgbouncer:latest resources: requests: cpu: \"0.1\" memory: 100Mi limits: cpu: \"0.5\" memory: 500Mi","title":"Pod templates"},{"location":"connection_pooling/#service-template","text":"Sometimes, your pooler will require some different labels, annotations, or even change the type of the service, you can achieve that by using the serviceTemplate field: apiVersion: postgresql.cnpg.io/v1 kind: Pooler metadata: name: pooler-example-rw spec: cluster: name: cluster-example instances: 3 type: rw serviceTemplate: metadata: labels: app: pooler spec: type: LoadBalancer pgbouncer: poolMode: session parameters: max_client_conn: \"1000\" default_pool_size: \"10\" The operator by default adds a ServicePort with the following data: ports: - name: pgbouncer port: 5432 protocol: TCP targetPort: pgbouncer Warning Specifying a ServicePort with the name pgbouncer or the port 5432 will prevent the default ServicePort from being added. This because ServicePort entries with the same name or port are not allowed on Kubernetes and result in errors.","title":"Service Template"},{"location":"connection_pooling/#high-availability-ha","text":"Because of Kubernetes' deployments, you can configure your pooler to run on a single instance or over multiple pods. The exposed service makes sure that your clients are randomly distributed over the available pods running PgBouncer, which then manages and reuses connections toward the underlying server (if using the rw service) or servers (if using the ro service with multiple replicas). Warning If your infrastructure spans multiple availability zones with high latency across them, be aware of network hops. Consider, for example, the case of your application running in zone 2, connecting to PgBouncer running in zone 3, and pointing to the PostgreSQL primary in zone 1.","title":"High availability (HA)"},{"location":"connection_pooling/#pgbouncer-configuration-options","text":"The operator manages most of the configuration options for PgBouncer , allowing you to modify only a subset of them. Warning You are responsible for correctly setting the value of each option, as the operator doesn't validate them. These are the PgBouncer options you can customize, with links to the PgBouncer documentation for each parameter. Unless stated otherwise, the default values are the ones directly set by PgBouncer. application_name_add_host autodb_idle_timeout cancel_wait_timeout client_idle_timeout client_login_timeout default_pool_size disable_pqexec dns_max_ttl dns_nxdomain_ttl idle_transaction_timeout ignore_startup_parameters : to be appended to extra_float_digits,options - required by CloudNativePG listen_backlog log_connections log_disconnections log_pooler_errors log_stats : by default disabled ( 0 ), given that statistics are already collected by the Prometheus export as described in the \"Monitoring\" section below max_client_conn max_db_connections max_packet_size max_prepared_statements max_user_connections min_pool_size pkt_buf query_timeout query_wait_timeout reserve_pool_size reserve_pool_timeout sbuf_loopcnt server_check_delay server_check_query server_connect_timeout server_fast_close server_idle_timeout server_lifetime server_login_retry server_reset_query server_reset_query_always server_round_robin server_tls_ciphers server_tls_protocols stats_period suspend_timeout tcp_defer_accept tcp_keepalive tcp_keepcnt tcp_keepidle tcp_keepintvl tcp_user_timeout tcp_socket_buffer track_extra_parameters verbose Customizations of the PgBouncer configuration are written declaratively in the .spec.pgbouncer.parameters map. The operator reacts to the changes in the pooler specification, and every PgBouncer instance reloads the updated configuration without disrupting the service. Warning Every PgBouncer pod has the same configuration, aligned with the parameters in the specification. A mistake in these parameters might disrupt the operability of the whole pooler. The operator doesn't validate the value of any option.","title":"PgBouncer configuration options"},{"location":"connection_pooling/#monitoring","text":"The PgBouncer implementation of the Pooler comes with a default Prometheus exporter. It makes available several metrics having the cnpg_pgbouncer_ prefix by running: SHOW LISTS (prefix: cnpg_pgbouncer_lists ) SHOW POOLS (prefix: cnpg_pgbouncer_pools ) SHOW STATS (prefix: cnpg_pgbouncer_stats ) Like the CloudNativePG instance, the exporter runs on port 9127 of each pod running PgBouncer and also provides metrics related to the Go runtime (with the prefix go_* ). Info You can inspect the exported metrics on a pod running PgBouncer. For instructions, see How to inspect the exported metrics . Make sure that you use the correct IP and the 9127 port. This example shows the output for cnpg_pgbouncer metrics: # HELP cnpg_pgbouncer_collection_duration_seconds Collection time duration in seconds # TYPE cnpg_pgbouncer_collection_duration_seconds gauge cnpg_pgbouncer_collection_duration_seconds{collector=\"Collect.up\"} 0.002338805 # HELP cnpg_pgbouncer_collection_errors_total Total errors occurred accessing PostgreSQL for metrics. # TYPE cnpg_pgbouncer_collection_errors_total counter cnpg_pgbouncer_collection_errors_total{collector=\"sql: Scan error on column index 16, name \\\"load_balance_hosts\\\": converting NULL to int is unsupported\"} 5 # HELP cnpg_pgbouncer_collections_total Total number of times PostgreSQL was accessed for metrics. # TYPE cnpg_pgbouncer_collections_total counter cnpg_pgbouncer_collections_total 5 # HELP cnpg_pgbouncer_last_collection_error 1 if the last collection ended with error, 0 otherwise. # TYPE cnpg_pgbouncer_last_collection_error gauge cnpg_pgbouncer_last_collection_error 0 # HELP cnpg_pgbouncer_lists_databases Count of databases. # TYPE cnpg_pgbouncer_lists_databases gauge cnpg_pgbouncer_lists_databases 1 # HELP cnpg_pgbouncer_lists_dns_names Count of DNS names in the cache. # TYPE cnpg_pgbouncer_lists_dns_names gauge cnpg_pgbouncer_lists_dns_names 0 # HELP cnpg_pgbouncer_lists_dns_pending Not used. # TYPE cnpg_pgbouncer_lists_dns_pending gauge cnpg_pgbouncer_lists_dns_pending 0 # HELP cnpg_pgbouncer_lists_dns_queries Count of in-flight DNS queries. # TYPE cnpg_pgbouncer_lists_dns_queries gauge cnpg_pgbouncer_lists_dns_queries 0 # HELP cnpg_pgbouncer_lists_dns_zones Count of DNS zones in the cache. # TYPE cnpg_pgbouncer_lists_dns_zones gauge cnpg_pgbouncer_lists_dns_zones 0 # HELP cnpg_pgbouncer_lists_free_clients Count of free clients. # TYPE cnpg_pgbouncer_lists_free_clients gauge cnpg_pgbouncer_lists_free_clients 49 # HELP cnpg_pgbouncer_lists_free_servers Count of free servers. # TYPE cnpg_pgbouncer_lists_free_servers gauge cnpg_pgbouncer_lists_free_servers 0 # HELP cnpg_pgbouncer_lists_login_clients Count of clients in login state. # TYPE cnpg_pgbouncer_lists_login_clients gauge cnpg_pgbouncer_lists_login_clients 0 # HELP cnpg_pgbouncer_lists_pools Count of pools. # TYPE cnpg_pgbouncer_lists_pools gauge cnpg_pgbouncer_lists_pools 1 # HELP cnpg_pgbouncer_lists_used_clients Count of used clients. # TYPE cnpg_pgbouncer_lists_used_clients gauge cnpg_pgbouncer_lists_used_clients 1 # HELP cnpg_pgbouncer_lists_used_servers Count of used servers. # TYPE cnpg_pgbouncer_lists_used_servers gauge cnpg_pgbouncer_lists_used_servers 0 # HELP cnpg_pgbouncer_lists_users Count of users. # TYPE cnpg_pgbouncer_lists_users gauge cnpg_pgbouncer_lists_users 2 # HELP cnpg_pgbouncer_pools_cl_active Client connections that are linked to server connection and can process queries. # TYPE cnpg_pgbouncer_pools_cl_active gauge cnpg_pgbouncer_pools_cl_active{database=\"pgbouncer\",user=\"pgbouncer\"} 1 # HELP cnpg_pgbouncer_pools_cl_active_cancel_req Client connections that have forwarded query cancellations to the server and are waiting for the server response. # TYPE cnpg_pgbouncer_pools_cl_active_cancel_req gauge cnpg_pgbouncer_pools_cl_active_cancel_req{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_cl_cancel_req Client connections that have not forwarded query cancellations to the server yet. # TYPE cnpg_pgbouncer_pools_cl_cancel_req gauge cnpg_pgbouncer_pools_cl_cancel_req{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_cl_waiting Client connections that have sent queries but have not yet got a server connection. # TYPE cnpg_pgbouncer_pools_cl_waiting gauge cnpg_pgbouncer_pools_cl_waiting{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_cl_waiting_cancel_req Client connections that have not forwarded query cancellations to the server yet. # TYPE cnpg_pgbouncer_pools_cl_waiting_cancel_req gauge cnpg_pgbouncer_pools_cl_waiting_cancel_req{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_load_balance_hosts Number of hosts not load balancing between hosts # TYPE cnpg_pgbouncer_pools_load_balance_hosts gauge cnpg_pgbouncer_pools_load_balance_hosts{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_maxwait How long the first (oldest) client in the queue has waited, in seconds. If this starts increasing, then the current pool of servers does not handle requests quickly enough. The reason may be either an overloaded server or just too small of a pool_size setting. # TYPE cnpg_pgbouncer_pools_maxwait gauge cnpg_pgbouncer_pools_maxwait{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_maxwait_us Microsecond part of the maximum waiting time. # TYPE cnpg_pgbouncer_pools_maxwait_us gauge cnpg_pgbouncer_pools_maxwait_us{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_pool_mode The pooling mode in use. 1 for session, 2 for transaction, 3 for statement, -1 if unknown # TYPE cnpg_pgbouncer_pools_pool_mode gauge cnpg_pgbouncer_pools_pool_mode{database=\"pgbouncer\",user=\"pgbouncer\"} 3 # HELP cnpg_pgbouncer_pools_sv_active Server connections that are linked to a client. # TYPE cnpg_pgbouncer_pools_sv_active gauge cnpg_pgbouncer_pools_sv_active{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_active_cancel Server connections that are currently forwarding a cancel request # TYPE cnpg_pgbouncer_pools_sv_active_cancel gauge cnpg_pgbouncer_pools_sv_active_cancel{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_idle Server connections that are unused and immediately usable for client queries. # TYPE cnpg_pgbouncer_pools_sv_idle gauge cnpg_pgbouncer_pools_sv_idle{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_login Server connections currently in the process of logging in. # TYPE cnpg_pgbouncer_pools_sv_login gauge cnpg_pgbouncer_pools_sv_login{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_tested Server connections that are currently running either server_reset_query or server_check_query. # TYPE cnpg_pgbouncer_pools_sv_tested gauge cnpg_pgbouncer_pools_sv_tested{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_used Server connections that have been idle for more than server_check_delay, so they need server_check_query to run on them before they can be used again. # TYPE cnpg_pgbouncer_pools_sv_used gauge cnpg_pgbouncer_pools_sv_used{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_pools_sv_wait_cancels Servers that normally could become idle, but are waiting to do so until all in-flight cancel requests have completed that were sent to cancel a query on this server. # TYPE cnpg_pgbouncer_pools_sv_wait_cancels gauge cnpg_pgbouncer_pools_sv_wait_cancels{database=\"pgbouncer\",user=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_bind_count Average number of prepared statements readied for execution by clients and forwarded to PostgreSQL by pgbouncer. # TYPE cnpg_pgbouncer_stats_avg_bind_count gauge cnpg_pgbouncer_stats_avg_bind_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_client_parse_count Average number of prepared statements created by clients. # TYPE cnpg_pgbouncer_stats_avg_client_parse_count gauge cnpg_pgbouncer_stats_avg_client_parse_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_query_count Average queries per second in last stat period. # TYPE cnpg_pgbouncer_stats_avg_query_count gauge cnpg_pgbouncer_stats_avg_query_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_query_time Average query duration, in microseconds. # TYPE cnpg_pgbouncer_stats_avg_query_time gauge cnpg_pgbouncer_stats_avg_query_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_recv Average received (from clients) bytes per second. # TYPE cnpg_pgbouncer_stats_avg_recv gauge cnpg_pgbouncer_stats_avg_recv{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_sent Average sent (to clients) bytes per second. # TYPE cnpg_pgbouncer_stats_avg_sent gauge cnpg_pgbouncer_stats_avg_sent{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_server_parse_count Average number of prepared statements created by pgbouncer on a server. # TYPE cnpg_pgbouncer_stats_avg_server_parse_count gauge cnpg_pgbouncer_stats_avg_server_parse_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_wait_time Time spent by clients waiting for a server, in microseconds (average per second). # TYPE cnpg_pgbouncer_stats_avg_wait_time gauge cnpg_pgbouncer_stats_avg_wait_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_xact_count Average transactions per second in last stat period. # TYPE cnpg_pgbouncer_stats_avg_xact_count gauge cnpg_pgbouncer_stats_avg_xact_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_avg_xact_time Average transaction duration, in microseconds. # TYPE cnpg_pgbouncer_stats_avg_xact_time gauge cnpg_pgbouncer_stats_avg_xact_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_bind_count Total number of prepared statements readied for execution by clients and forwarded to PostgreSQL by pgbouncer # TYPE cnpg_pgbouncer_stats_total_bind_count gauge cnpg_pgbouncer_stats_total_bind_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_client_parse_count Total number of prepared statements created by clients. # TYPE cnpg_pgbouncer_stats_total_client_parse_count gauge cnpg_pgbouncer_stats_total_client_parse_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_query_count Total number of SQL queries pooled by pgbouncer. # TYPE cnpg_pgbouncer_stats_total_query_count gauge cnpg_pgbouncer_stats_total_query_count{database=\"pgbouncer\"} 15 # HELP cnpg_pgbouncer_stats_total_query_time Total number of microseconds spent by pgbouncer when actively connected to PostgreSQL, executing queries. # TYPE cnpg_pgbouncer_stats_total_query_time gauge cnpg_pgbouncer_stats_total_query_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_received Total volume in bytes of network traffic received by pgbouncer. # TYPE cnpg_pgbouncer_stats_total_received gauge cnpg_pgbouncer_stats_total_received{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_sent Total volume in bytes of network traffic sent by pgbouncer. # TYPE cnpg_pgbouncer_stats_total_sent gauge cnpg_pgbouncer_stats_total_sent{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_server_parse_count Total number of prepared statements created by pgbouncer on a server. # TYPE cnpg_pgbouncer_stats_total_server_parse_count gauge cnpg_pgbouncer_stats_total_server_parse_count{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_wait_time Time spent by clients waiting for a server, in microseconds. # TYPE cnpg_pgbouncer_stats_total_wait_time gauge cnpg_pgbouncer_stats_total_wait_time{database=\"pgbouncer\"} 0 # HELP cnpg_pgbouncer_stats_total_xact_count Total number of SQL transactions pooled by pgbouncer. # TYPE cnpg_pgbouncer_stats_total_xact_count gauge cnpg_pgbouncer_stats_total_xact_count{database=\"pgbouncer\"} 15 # HELP cnpg_pgbouncer_stats_total_xact_time Total number of microseconds spent by pgbouncer when connected to PostgreSQL in a transaction, either idle in transaction or executing queries. # TYPE cnpg_pgbouncer_stats_total_xact_time gauge cnpg_pgbouncer_stats_total_xact_time{database=\"pgbouncer\"} 0 Info For a better understanding of the metrics please refer to the PgBouncer documentation. As for clusters, a specific pooler can be monitored using the Prometheus operator's resource PodMonitor . A PodMonitor correctly pointing to a pooler can be created by the operator by setting .spec.monitoring.enablePodMonitor to true in the Pooler resource. The default is false . Important Any change to PodMonitor created automatically is overridden by the operator at the next reconciliation cycle. If you need to customize it, you can do so as shown in the following example. To deploy a PodMonitor for a specific pooler manually, you can define it as follows and change it as needed: apiVersion: monitoring.coreos.com/v1 kind: PodMonitor metadata: name: spec: selector: matchLabels: cnpg.io/poolerName: podMetricsEndpoints: - port: metrics","title":"Monitoring"},{"location":"connection_pooling/#logging","text":"Logs are directly sent to standard output, in JSON format, like in the following example: { \"level\": \"info\", \"ts\": SECONDS.MICROSECONDS, \"msg\": \"record\", \"pipe\": \"stderr\", \"record\": { \"timestamp\": \"YYYY-MM-DD HH:MM:SS.MS UTC\", \"pid\": \"\", \"level\": \"LOG\", \"msg\": \"kernel file descriptor limit: 1048576 (hard: 1048576); max_client_conn: 100, max expected fd use: 112\" } }","title":"Logging"},{"location":"connection_pooling/#pausing-connections","text":"The Pooler specification allows you to take advantage of PgBouncer's PAUSE and RESUME commands, using only declarative configuration. You can ado this using the paused option, which by default is set to false . When set to true , the operator internally invokes the PAUSE command in PgBouncer, which: Closes all active connections toward the PostgreSQL server, after waiting for the queries to complete Pauses any new connection coming from the client When the paused option is reset to false , the operator invokes the RESUME command in PgBouncer, reopening the taps toward the PostgreSQL service defined in the Pooler resource. PAUSE For more information, see PAUSE in the PgBouncer documentation . Important In future versions, the switchover operation will be fully integrated with the PgBouncer pooler and take advantage of the PAUSE / RESUME features to reduce the perceived downtime by client applications. Currently, you can achieve the same results by setting the paused attribute to true , issuing the switchover command through the cnpg plugin , and then restoring the paused attribute to false .","title":"Pausing connections"},{"location":"connection_pooling/#limitations","text":"","title":"Limitations"},{"location":"connection_pooling/#single-postgresql-cluster","text":"The current implementation of the pooler is designed to work as part of a specific CloudNativePG cluster (a service). It isn't currently possible to create a pooler that spans multiple clusters.","title":"Single PostgreSQL cluster"},{"location":"connection_pooling/#controlled-configurability","text":"CloudNativePG transparently manages several configuration options that are used for the PgBouncer layer to communicate with PostgreSQL. Such options aren't configurable from outside and include TLS certificates, authentication settings, the databases section, and the users section. Also, considering the specific use case for the single PostgreSQL cluster, the adopted criteria is to explicitly list the options that can be configured by users. Note The adopted solution likely addresses the majority of use cases. It leaves room for the future implementation of a separate operator for PgBouncer to complete the gamma with more advanced and customized scenarios.","title":"Controlled configurability"},{"location":"container_images/","text":"Container Image Requirements The CloudNativePG operator for Kubernetes is designed to work with any compatible container image of PostgreSQL that complies with the following requirements: PostgreSQL executables that must be in the path: initdb postgres pg_ctl pg_controldata pg_basebackup Barman Cloud executables that must be in the path: barman-cloud-backup barman-cloud-backup-delete barman-cloud-backup-list barman-cloud-check-wal-archive barman-cloud-restore barman-cloud-wal-archive barman-cloud-wal-restore PGAudit extension installed (optional - only if PGAudit is required in the deployed clusters) Appropriate locale settings du (optional, for kubectl cnpg status ) Important Only PostgreSQL versions supported by the PGDG are allowed. No entry point and/or command is required in the image definition, as CloudNativePG overrides it with its instance manager. Warning Application Container Images will be used by CloudNativePG in a Primary with multiple/optional Hot Standby Servers Architecture only. The CloudNativePG community provides and supports public PostgreSQL container images that work with CloudNativePG, and publishes them on ghcr.io . Image Tag Requirements To ensure the operator makes informed decisions, it must accurately detect the PostgreSQL major version. This detection can occur in two ways: Utilizing the major field of the imageCatalogRef , if defined. Auto-detecting the major version from the image tag of the imageName if not explicitly specified. For auto-detection to work, the image tag must adhere to a specific format. It should commence with a valid PostgreSQL major version number (e.g., 15.6 or 16), optionally followed by a dot and the patch level. Following this, the tag can include any character combination valid and accepted in a Docker tag, preceded by a dot, an underscore, or a minus sign. Examples of accepted image tags: 12.1 13.3.2.1-1 13.4 14 15.5-10 16.0 Warning latest is not considered a valid tag for the image. Note Image tag requirements do no apply for images defined in a catalog.","title":"Container Image Requirements"},{"location":"container_images/#container-image-requirements","text":"The CloudNativePG operator for Kubernetes is designed to work with any compatible container image of PostgreSQL that complies with the following requirements: PostgreSQL executables that must be in the path: initdb postgres pg_ctl pg_controldata pg_basebackup Barman Cloud executables that must be in the path: barman-cloud-backup barman-cloud-backup-delete barman-cloud-backup-list barman-cloud-check-wal-archive barman-cloud-restore barman-cloud-wal-archive barman-cloud-wal-restore PGAudit extension installed (optional - only if PGAudit is required in the deployed clusters) Appropriate locale settings du (optional, for kubectl cnpg status ) Important Only PostgreSQL versions supported by the PGDG are allowed. No entry point and/or command is required in the image definition, as CloudNativePG overrides it with its instance manager. Warning Application Container Images will be used by CloudNativePG in a Primary with multiple/optional Hot Standby Servers Architecture only. The CloudNativePG community provides and supports public PostgreSQL container images that work with CloudNativePG, and publishes them on ghcr.io .","title":"Container Image Requirements"},{"location":"container_images/#image-tag-requirements","text":"To ensure the operator makes informed decisions, it must accurately detect the PostgreSQL major version. This detection can occur in two ways: Utilizing the major field of the imageCatalogRef , if defined. Auto-detecting the major version from the image tag of the imageName if not explicitly specified. For auto-detection to work, the image tag must adhere to a specific format. It should commence with a valid PostgreSQL major version number (e.g., 15.6 or 16), optionally followed by a dot and the patch level. Following this, the tag can include any character combination valid and accepted in a Docker tag, preceded by a dot, an underscore, or a minus sign. Examples of accepted image tags: 12.1 13.3.2.1-1 13.4 14 15.5-10 16.0 Warning latest is not considered a valid tag for the image. Note Image tag requirements do no apply for images defined in a catalog.","title":"Image Tag Requirements"},{"location":"controller/","text":"Custom Pod Controller Kubernetes uses the Controller pattern to align the current cluster state with the desired one. Stateful applications are usually managed with the StatefulSet controller, which creates and reconciles a set of Pods built from the same specification, and assigns them a sticky identity. CloudNativePG implements its own custom controller to manage PostgreSQL instances, instead of relying on the StatefulSet controller. While bringing more complexity to the implementation, this design choice provides the operator with more flexibility on how we manage the cluster, while being transparent on the topology of PostgreSQL clusters. Like many choices in the design realm, different ones lead to other compromises. The following sections discuss a few points where we believe this design choice has made the implementation of CloudNativePG more reliable, and easier to understand. PVC resizing This is a well known limitation of StatefulSet : it does not support resizing PVCs. This is inconvenient for a database. Resizing volumes requires convoluted workarounds. In contrast, CloudNativePG leverages the configured storage class to manage the underlying PVCs directly, and can handle PVC resizing if the storage class supports it. Primary Instances versus Replicas The StatefulSet controller is designed to create a set of Pods from just one template. Given that we use one Pod per PostgreSQL instance, we have two kinds of Pods: primary instance (only one) replicas (multiple, optional) This difference is relevant when deciding the correct deployment strategy to execute for a given operation. Some operations should be performed on the replicas first, and then on the primary, but only after an updated replica is promoted as the new primary. For example, when you want to apply a different PostgreSQL image version, or when you increase configuration parameters like max_connections (which are treated specially by PostgreSQL because CloudNativePG uses hot standby replicas ). While doing that, CloudNativePG considers the PostgreSQL instance's role - and not just its serial number. Sometimes the operator needs to follow the opposite process: work on the primary first and then on the replicas. For example, when you lower max_connections . In that case, CloudNativePG will: apply the new setting to the primary instance restart it apply the new setting on the replicas The StatefulSet controller, being application-independent, can't incorporate this behavior, which is specific to PostgreSQL's native replication technology. Coherence of PVCs PostgreSQL instances can be configured to work with multiple PVCs: this is how WAL storage can be separated from PGDATA . The two data stores need to be coherent from the PostgreSQL point of view, as they're used simultaneously. If you delete the PVC corresponding to the WAL storage of an instance, the PVC where PGDATA is stored will not be usable anymore. This behavior is specific to PostgreSQL and is not implemented in the StatefulSet controller - the latter not being application specific. After the user dropped a PVC, a StatefulSet would just recreate it, leading to a corrupted PostgreSQL instance. CloudNativePG would instead classify the remaining PVC as unusable, and start creating a new pair of PVCs for another instance to join the cluster correctly. Local storage, remote storage, and database size Sometimes you need to take down a Kubernetes node to do an upgrade. After the upgrade, depending on your upgrade strategy, the updated node could go up again, or a new node could replace it. Supposing the unavailable node was hosting a PostgreSQL instance, depending on your database size and your cloud infrastructure, you may prefer to choose one of the following actions: drop the PVC and the Pod residing on the downed node; create a new PVC cloning the data from another PVC; after that, schedule a Pod for it drop the Pod, schedule the Pod in a different node, and mount the PVC from there leave the Pod and the PVC as they are, and wait for the node to be back up. The first solution is practical when your database size permits, allowing you to immediately bring back the desired number of replicas. The second solution is only feasible when you're not using the storage of the local node, and re-mounting the PVC in another host is possible in a reasonable amount of time (which only you and your organization know). The third solution is appropriate when the database is big and uses local node storage for maximum performance and data durability. The CloudNativePG controller implements all these strategies so that the user can select the preferred behavior at the cluster level (read the \"Kubernetes upgrade\" section for details). Being generic, the StatefulSet doesn't allow this level of customization.","title":"Custom Pod Controller"},{"location":"controller/#custom-pod-controller","text":"Kubernetes uses the Controller pattern to align the current cluster state with the desired one. Stateful applications are usually managed with the StatefulSet controller, which creates and reconciles a set of Pods built from the same specification, and assigns them a sticky identity. CloudNativePG implements its own custom controller to manage PostgreSQL instances, instead of relying on the StatefulSet controller. While bringing more complexity to the implementation, this design choice provides the operator with more flexibility on how we manage the cluster, while being transparent on the topology of PostgreSQL clusters. Like many choices in the design realm, different ones lead to other compromises. The following sections discuss a few points where we believe this design choice has made the implementation of CloudNativePG more reliable, and easier to understand.","title":"Custom Pod Controller"},{"location":"controller/#pvc-resizing","text":"This is a well known limitation of StatefulSet : it does not support resizing PVCs. This is inconvenient for a database. Resizing volumes requires convoluted workarounds. In contrast, CloudNativePG leverages the configured storage class to manage the underlying PVCs directly, and can handle PVC resizing if the storage class supports it.","title":"PVC resizing"},{"location":"controller/#primary-instances-versus-replicas","text":"The StatefulSet controller is designed to create a set of Pods from just one template. Given that we use one Pod per PostgreSQL instance, we have two kinds of Pods: primary instance (only one) replicas (multiple, optional) This difference is relevant when deciding the correct deployment strategy to execute for a given operation. Some operations should be performed on the replicas first, and then on the primary, but only after an updated replica is promoted as the new primary. For example, when you want to apply a different PostgreSQL image version, or when you increase configuration parameters like max_connections (which are treated specially by PostgreSQL because CloudNativePG uses hot standby replicas ). While doing that, CloudNativePG considers the PostgreSQL instance's role - and not just its serial number. Sometimes the operator needs to follow the opposite process: work on the primary first and then on the replicas. For example, when you lower max_connections . In that case, CloudNativePG will: apply the new setting to the primary instance restart it apply the new setting on the replicas The StatefulSet controller, being application-independent, can't incorporate this behavior, which is specific to PostgreSQL's native replication technology.","title":"Primary Instances versus Replicas"},{"location":"controller/#coherence-of-pvcs","text":"PostgreSQL instances can be configured to work with multiple PVCs: this is how WAL storage can be separated from PGDATA . The two data stores need to be coherent from the PostgreSQL point of view, as they're used simultaneously. If you delete the PVC corresponding to the WAL storage of an instance, the PVC where PGDATA is stored will not be usable anymore. This behavior is specific to PostgreSQL and is not implemented in the StatefulSet controller - the latter not being application specific. After the user dropped a PVC, a StatefulSet would just recreate it, leading to a corrupted PostgreSQL instance. CloudNativePG would instead classify the remaining PVC as unusable, and start creating a new pair of PVCs for another instance to join the cluster correctly.","title":"Coherence of PVCs"},{"location":"controller/#local-storage-remote-storage-and-database-size","text":"Sometimes you need to take down a Kubernetes node to do an upgrade. After the upgrade, depending on your upgrade strategy, the updated node could go up again, or a new node could replace it. Supposing the unavailable node was hosting a PostgreSQL instance, depending on your database size and your cloud infrastructure, you may prefer to choose one of the following actions: drop the PVC and the Pod residing on the downed node; create a new PVC cloning the data from another PVC; after that, schedule a Pod for it drop the Pod, schedule the Pod in a different node, and mount the PVC from there leave the Pod and the PVC as they are, and wait for the node to be back up. The first solution is practical when your database size permits, allowing you to immediately bring back the desired number of replicas. The second solution is only feasible when you're not using the storage of the local node, and re-mounting the PVC in another host is possible in a reasonable amount of time (which only you and your organization know). The third solution is appropriate when the database is big and uses local node storage for maximum performance and data durability. The CloudNativePG controller implements all these strategies so that the user can select the preferred behavior at the cluster level (read the \"Kubernetes upgrade\" section for details). Being generic, the StatefulSet doesn't allow this level of customization.","title":"Local storage, remote storage, and database size"},{"location":"database_import/","text":"Importing Postgres databases This section describes how to import one or more existing PostgreSQL databases inside a brand new CloudNativePG cluster. The import operation is based on the concept of online logical backups in PostgreSQL, and relies on pg_dump via a network connection to the origin host, and pg_restore . Thanks to native Multi-Version Concurrency Control (MVCC) and snapshots, PostgreSQL enables taking consistent backups over the network, in a concurrent manner, without stopping any write activity. Logical backups are also the most common, flexible and reliable technique to perform major upgrades of PostgreSQL versions. As a result, the instructions in this section are suitable for both: importing one or more databases from an existing PostgreSQL instance, even outside Kubernetes importing the database from any PostgreSQL version to one that is either the same or newer, enabling major upgrades of PostgreSQL (e.g. from version 13.x to version 17.x) Warning When performing major upgrades of PostgreSQL you are responsible for making sure that applications are compatible with the new version and that the upgrade path of the objects contained in the database (including extensions) is feasible. In both cases, the operation is performed on a consistent snapshot of the origin database. Important For this reason we suggest to stop write operations on the source before the final import in the Cluster resource, as changes done to the source database after the start of the backup will not be in the destination cluster - hence why this feature is referred to as \"offline import\" or \"offline major upgrade\". How it works Conceptually, the import requires you to create a new cluster from scratch ( destination cluster ), using the initdb bootstrap method , and then complete the initdb.import subsection to import objects from an existing Postgres cluster ( source cluster ). As per PostgreSQL recommendation, we suggest that the PostgreSQL major version of the destination cluster is greater or equal than the one of the source cluster . CloudNativePG provides two main ways to import objects from the source cluster into the destination cluster: microservice approach : the destination cluster is designed to host a single application database owned by the specified application user, as recommended by the CloudNativePG project monolith approach : the destination cluster is designed to host multiple databases and different users, imported from the source cluster The first import method is available via the microservice type, the second via the monolith type. Warning It is your responsibility to ensure that the destination cluster can access the source cluster with a superuser or a user having enough privileges to take a logical backup with pg_dump . Please refer to the PostgreSQL documentation on pg_dump for further information. The microservice type With the microservice approach, you can specify a single database you want to import from the source cluster into the destination cluster. The operation is performed in 4 steps: initdb bootstrap of the new cluster export of the selected database (in initdb.import.databases ) using pg_dump -Fd import of the database using pg_restore --no-acl --no-owner into the initdb.database (application database) owned by the initdb.owner user cleanup of the database dump file optional execution of the user defined SQL queries in the application database via the postImportApplicationSQL parameter execution of ANALYZE VERBOSE on the imported database In the figure below, a single PostgreSQL cluster containing N databases is imported into separate CloudNativePG clusters, with each cluster using a microservice import for one of the N source databases. For example, the YAML below creates a new 3 instance PostgreSQL cluster (latest available major version at the time the operator was released) called cluster-microservice that imports the angus database from the cluster-pg96 cluster (with the unsupported PostgreSQL 9.6), by connecting to the postgres database using the postgres user, via the password stored in the cluster-pg96-superuser secret. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-microservice spec: instances: 3 bootstrap: initdb: import: type: microservice databases: - angus source: externalCluster: cluster-pg96 #postImportApplicationSQL: #- | # INSERT YOUR SQL QUERIES HERE storage: size: 1Gi externalClusters: - name: cluster-pg96 connectionParameters: # Use the correct IP or host name for the source database host: pg96.local user: postgres dbname: postgres password: name: cluster-pg96-superuser key: password Warning The example above deliberately uses a source database running a version of PostgreSQL that is not supported anymore by the Community, and consequently by CloudNativePG. Data export from the source instance is performed using the version of pg_dump in the destination cluster, which must be a supported one, and equal or greater than the source one. Based on our experience, this way of exporting data should work on older and unsupported versions of Postgres too, giving you the chance to move your legacy data to a better system, inside Kubernetes. This is the main reason why we used 9.6 in the examples of this section. We'd be interested to hear from you, should you experience any issues in this area. There are a few things you need to be aware of when using the microservice type: It requires an externalCluster that points to an existing PostgreSQL instance containing the data to import (for more information, please refer to \"The externalClusters section\" ) Traffic must be allowed between the Kubernetes cluster and the externalCluster during the operation Connection to the source database must be granted with the specified user that needs to run pg_dump and read roles information ( superuser is OK) Currently, the pg_dump -Fd result is stored temporarily inside the dumps folder in the PGDATA volume, so there should be enough available space to temporarily contain the dump result on the assigned node, as well as the restored data and indexes. Once the import operation is completed, this folder is automatically deleted by the operator. Only one database can be specified inside the initdb.import.databases array Roles are not imported - and as such they cannot be specified inside initdb.import.roles Hint The microservice approach adheres to CloudNativePG conventions and defaults for the destination cluster. If you do not set initdb.database or initdb.owner for the destination cluster, both parameters will default to app . The monolith type With the monolith approach, you can specify a set of roles and databases you want to import from the source cluster into the destination cluster. The operation is performed in the following steps: initdb bootstrap of the new cluster export and import of the selected roles export of the selected databases (in initdb.import.databases ), one at a time, using pg_dump -Fd create each of the selected databases and import data using pg_restore run ANALYZE on each imported database cleanup of the database dump files For example, the YAML below creates a new 3 instance PostgreSQL cluster (latest available major version at the time the operator was released) called cluster-monolith that imports the accountant and the bank_user roles, as well as the accounting , banking , resort databases from the cluster-pg96 cluster (with the unsupported PostgreSQL 9.6), by connecting to the postgres database using the postgres user, via the password stored in the cluster-pg96-superuser secret. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-monolith spec: instances: 3 bootstrap: initdb: import: type: monolith databases: - accounting - banking - resort roles: - accountant - bank_user source: externalCluster: cluster-pg96 storage: size: 1Gi externalClusters: - name: cluster-pg96 connectionParameters: # Use the correct IP or host name for the source database host: pg96.local user: postgres dbname: postgres sslmode: require password: name: cluster-pg96-superuser key: password There are a few things you need to be aware of when using the monolith type: It requires an externalCluster that points to an existing PostgreSQL instance containing the data to import (for more information, please refer to \"The externalClusters section\" ) Traffic must be allowed between the Kubernetes cluster and the externalCluster during the operation Connection to the source database must be granted with the specified user that needs to run pg_dump and retrieve roles information ( superuser is OK) Currently, the pg_dump -Fd result is stored temporarily inside the dumps folder in the PGDATA volume of the destination cluster's instances, so there should be enough available space to temporarily contain the dump result on the assigned node, as well as the restored data and indexes. Once the import operation is completed, this folder is automatically deleted by the operator. At least one database to be specified in the initdb.import.databases array Any role that is required by the imported databases must be specified inside initdb.import.roles , with the limitations below: The following roles, if present, are not imported: postgres , streaming_replica , cnpg_pooler_pgbouncer The SUPERUSER option is removed from any imported role Wildcard \"*\" can be used as the only element in the databases and/or roles arrays to import every object of the kind; When matching databases the wildcard will ignore the postgres database, template databases, and those databases not allowing connections After the clone procedure is done, ANALYZE VERBOSE is executed for every database. The postImportApplicationSQL field is not supported Hint The databases and their owners are preserved exactly as they exist in the source cluster\u2014no app database or user will be created during import. If your bootstrap.initdb stanza specifies custom database and owner values that do not match any of the databases or users being imported, the instance manager will create a new, empty application database and owner role with those specified names, while leaving the imported databases and owners unchanged. A practical example There is nothing to stop you from using the monolith approach to import a single database. It is interesting to see how the results of doing so would differ from using the microservice approach. Given a source cluster, for example the following, with a database named mydb owned by role me : apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 1 postgresql: pg_hba: - host all all all trust storage: size: 1Gi bootstrap: initdb: database: mydb owner: me We can import it via microservice : apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-microservice spec: instances: 1 storage: size: 1Gi bootstrap: initdb: import: type: microservice databases: - mydb source: externalCluster: cluster-example externalClusters: - name: cluster-example connectionParameters: host: cluster-example-rw dbname: postgres as well as via monolith: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-monolith spec: instances: 1 storage: size: 1Gi bootstrap: initdb: import: type: monolith databases: - mydb roles: - me source: externalCluster: cluster-example externalClusters: - name: cluster-example connectionParameters: host: cluster-example-rw dbname: postgres In both cases, the database's contents will be imported, but: In the microservice case, the imported database's name and owner both become app , or whichever configuration for the fields database and owner are set in the bootstrap.initdb stanza. In the monolith case, the database and owner are kept exactly as in the source cluster, i.e. mydb and me respectively. No app database nor user will be created. If there are custom settings for database and owner in the bootstrap.initdb stanza that don't match the source databases/owners to import, the instance manager will create a new empty application database and owner role, but will leave the imported databases/owners intact. Import optimizations During the logical import of a database, CloudNativePG optimizes the configuration of PostgreSQL in order to prioritize speed versus data durability, by forcing: archive_mode to off fsync to off full_page_writes to off max_wal_senders to 0 wal_level to minimal Before completing the import job, CloudNativePG restores the expected configuration, then runs initdb --sync-only to ensure that data is permanently written on disk. Important WAL archiving, if requested, and WAL level will be honored after the database import process has completed. Similarly, replicas will be cloned after the bootstrap phase, when the actual cluster resource starts. There are other optimizations you can do during the import phase. Although this topic is beyond the scope of CloudNativePG, we recommend that you reduce unnecessary writes in the checkpoint area by tuning Postgres GUCs like shared_buffers , max_wal_size , checkpoint_timeout directly in the Cluster configuration. Customizing pg_dump and pg_restore Behavior You can customize the behavior of pg_dump and pg_restore by specifying additional options using the pgDumpExtraOptions and pgRestoreExtraOptions parameters. For instance, you can enable parallel jobs to speed up data import/export processes, as shown in the following example: # bootstrap: initdb: import: type: microservice databases: - app source: externalCluster: cluster-example pgDumpExtraOptions: - '--jobs=2' pgRestoreExtraOptions: - '--jobs=2' # Warning Use the pgDumpExtraOptions and pgRestoreExtraOptions fields with caution and at your own risk. These options are not validated or verified by the operator, and some configurations may conflict with its intended functionality or behavior. Always test thoroughly in a safe and controlled environment before applying them in production. Online Import and Upgrades Logical replication offers a powerful way to import any PostgreSQL database accessible over the network using the following approach: Import Bootstrap with Schema-Only Option : Initialize the schema in the target database before replication begins. Subscription Resource : Set up continuous replication to synchronize data changes. This technique can also be leveraged for performing major PostgreSQL upgrades with minimal downtime, making it ideal for seamless migrations and system upgrades. For more details, including limitations and best practices, refer to the Logical Replication section in the documentation.","title":"Importing Postgres databases"},{"location":"database_import/#importing-postgres-databases","text":"This section describes how to import one or more existing PostgreSQL databases inside a brand new CloudNativePG cluster. The import operation is based on the concept of online logical backups in PostgreSQL, and relies on pg_dump via a network connection to the origin host, and pg_restore . Thanks to native Multi-Version Concurrency Control (MVCC) and snapshots, PostgreSQL enables taking consistent backups over the network, in a concurrent manner, without stopping any write activity. Logical backups are also the most common, flexible and reliable technique to perform major upgrades of PostgreSQL versions. As a result, the instructions in this section are suitable for both: importing one or more databases from an existing PostgreSQL instance, even outside Kubernetes importing the database from any PostgreSQL version to one that is either the same or newer, enabling major upgrades of PostgreSQL (e.g. from version 13.x to version 17.x) Warning When performing major upgrades of PostgreSQL you are responsible for making sure that applications are compatible with the new version and that the upgrade path of the objects contained in the database (including extensions) is feasible. In both cases, the operation is performed on a consistent snapshot of the origin database. Important For this reason we suggest to stop write operations on the source before the final import in the Cluster resource, as changes done to the source database after the start of the backup will not be in the destination cluster - hence why this feature is referred to as \"offline import\" or \"offline major upgrade\".","title":"Importing Postgres databases"},{"location":"database_import/#how-it-works","text":"Conceptually, the import requires you to create a new cluster from scratch ( destination cluster ), using the initdb bootstrap method , and then complete the initdb.import subsection to import objects from an existing Postgres cluster ( source cluster ). As per PostgreSQL recommendation, we suggest that the PostgreSQL major version of the destination cluster is greater or equal than the one of the source cluster . CloudNativePG provides two main ways to import objects from the source cluster into the destination cluster: microservice approach : the destination cluster is designed to host a single application database owned by the specified application user, as recommended by the CloudNativePG project monolith approach : the destination cluster is designed to host multiple databases and different users, imported from the source cluster The first import method is available via the microservice type, the second via the monolith type. Warning It is your responsibility to ensure that the destination cluster can access the source cluster with a superuser or a user having enough privileges to take a logical backup with pg_dump . Please refer to the PostgreSQL documentation on pg_dump for further information.","title":"How it works"},{"location":"database_import/#the-microservice-type","text":"With the microservice approach, you can specify a single database you want to import from the source cluster into the destination cluster. The operation is performed in 4 steps: initdb bootstrap of the new cluster export of the selected database (in initdb.import.databases ) using pg_dump -Fd import of the database using pg_restore --no-acl --no-owner into the initdb.database (application database) owned by the initdb.owner user cleanup of the database dump file optional execution of the user defined SQL queries in the application database via the postImportApplicationSQL parameter execution of ANALYZE VERBOSE on the imported database In the figure below, a single PostgreSQL cluster containing N databases is imported into separate CloudNativePG clusters, with each cluster using a microservice import for one of the N source databases. For example, the YAML below creates a new 3 instance PostgreSQL cluster (latest available major version at the time the operator was released) called cluster-microservice that imports the angus database from the cluster-pg96 cluster (with the unsupported PostgreSQL 9.6), by connecting to the postgres database using the postgres user, via the password stored in the cluster-pg96-superuser secret. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-microservice spec: instances: 3 bootstrap: initdb: import: type: microservice databases: - angus source: externalCluster: cluster-pg96 #postImportApplicationSQL: #- | # INSERT YOUR SQL QUERIES HERE storage: size: 1Gi externalClusters: - name: cluster-pg96 connectionParameters: # Use the correct IP or host name for the source database host: pg96.local user: postgres dbname: postgres password: name: cluster-pg96-superuser key: password Warning The example above deliberately uses a source database running a version of PostgreSQL that is not supported anymore by the Community, and consequently by CloudNativePG. Data export from the source instance is performed using the version of pg_dump in the destination cluster, which must be a supported one, and equal or greater than the source one. Based on our experience, this way of exporting data should work on older and unsupported versions of Postgres too, giving you the chance to move your legacy data to a better system, inside Kubernetes. This is the main reason why we used 9.6 in the examples of this section. We'd be interested to hear from you, should you experience any issues in this area. There are a few things you need to be aware of when using the microservice type: It requires an externalCluster that points to an existing PostgreSQL instance containing the data to import (for more information, please refer to \"The externalClusters section\" ) Traffic must be allowed between the Kubernetes cluster and the externalCluster during the operation Connection to the source database must be granted with the specified user that needs to run pg_dump and read roles information ( superuser is OK) Currently, the pg_dump -Fd result is stored temporarily inside the dumps folder in the PGDATA volume, so there should be enough available space to temporarily contain the dump result on the assigned node, as well as the restored data and indexes. Once the import operation is completed, this folder is automatically deleted by the operator. Only one database can be specified inside the initdb.import.databases array Roles are not imported - and as such they cannot be specified inside initdb.import.roles Hint The microservice approach adheres to CloudNativePG conventions and defaults for the destination cluster. If you do not set initdb.database or initdb.owner for the destination cluster, both parameters will default to app .","title":"The microservice type"},{"location":"database_import/#the-monolith-type","text":"With the monolith approach, you can specify a set of roles and databases you want to import from the source cluster into the destination cluster. The operation is performed in the following steps: initdb bootstrap of the new cluster export and import of the selected roles export of the selected databases (in initdb.import.databases ), one at a time, using pg_dump -Fd create each of the selected databases and import data using pg_restore run ANALYZE on each imported database cleanup of the database dump files For example, the YAML below creates a new 3 instance PostgreSQL cluster (latest available major version at the time the operator was released) called cluster-monolith that imports the accountant and the bank_user roles, as well as the accounting , banking , resort databases from the cluster-pg96 cluster (with the unsupported PostgreSQL 9.6), by connecting to the postgres database using the postgres user, via the password stored in the cluster-pg96-superuser secret. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-monolith spec: instances: 3 bootstrap: initdb: import: type: monolith databases: - accounting - banking - resort roles: - accountant - bank_user source: externalCluster: cluster-pg96 storage: size: 1Gi externalClusters: - name: cluster-pg96 connectionParameters: # Use the correct IP or host name for the source database host: pg96.local user: postgres dbname: postgres sslmode: require password: name: cluster-pg96-superuser key: password There are a few things you need to be aware of when using the monolith type: It requires an externalCluster that points to an existing PostgreSQL instance containing the data to import (for more information, please refer to \"The externalClusters section\" ) Traffic must be allowed between the Kubernetes cluster and the externalCluster during the operation Connection to the source database must be granted with the specified user that needs to run pg_dump and retrieve roles information ( superuser is OK) Currently, the pg_dump -Fd result is stored temporarily inside the dumps folder in the PGDATA volume of the destination cluster's instances, so there should be enough available space to temporarily contain the dump result on the assigned node, as well as the restored data and indexes. Once the import operation is completed, this folder is automatically deleted by the operator. At least one database to be specified in the initdb.import.databases array Any role that is required by the imported databases must be specified inside initdb.import.roles , with the limitations below: The following roles, if present, are not imported: postgres , streaming_replica , cnpg_pooler_pgbouncer The SUPERUSER option is removed from any imported role Wildcard \"*\" can be used as the only element in the databases and/or roles arrays to import every object of the kind; When matching databases the wildcard will ignore the postgres database, template databases, and those databases not allowing connections After the clone procedure is done, ANALYZE VERBOSE is executed for every database. The postImportApplicationSQL field is not supported Hint The databases and their owners are preserved exactly as they exist in the source cluster\u2014no app database or user will be created during import. If your bootstrap.initdb stanza specifies custom database and owner values that do not match any of the databases or users being imported, the instance manager will create a new, empty application database and owner role with those specified names, while leaving the imported databases and owners unchanged.","title":"The monolith type"},{"location":"database_import/#a-practical-example","text":"There is nothing to stop you from using the monolith approach to import a single database. It is interesting to see how the results of doing so would differ from using the microservice approach. Given a source cluster, for example the following, with a database named mydb owned by role me : apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 1 postgresql: pg_hba: - host all all all trust storage: size: 1Gi bootstrap: initdb: database: mydb owner: me We can import it via microservice : apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-microservice spec: instances: 1 storage: size: 1Gi bootstrap: initdb: import: type: microservice databases: - mydb source: externalCluster: cluster-example externalClusters: - name: cluster-example connectionParameters: host: cluster-example-rw dbname: postgres as well as via monolith: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example-monolith spec: instances: 1 storage: size: 1Gi bootstrap: initdb: import: type: monolith databases: - mydb roles: - me source: externalCluster: cluster-example externalClusters: - name: cluster-example connectionParameters: host: cluster-example-rw dbname: postgres In both cases, the database's contents will be imported, but: In the microservice case, the imported database's name and owner both become app , or whichever configuration for the fields database and owner are set in the bootstrap.initdb stanza. In the monolith case, the database and owner are kept exactly as in the source cluster, i.e. mydb and me respectively. No app database nor user will be created. If there are custom settings for database and owner in the bootstrap.initdb stanza that don't match the source databases/owners to import, the instance manager will create a new empty application database and owner role, but will leave the imported databases/owners intact.","title":"A practical example"},{"location":"database_import/#import-optimizations","text":"During the logical import of a database, CloudNativePG optimizes the configuration of PostgreSQL in order to prioritize speed versus data durability, by forcing: archive_mode to off fsync to off full_page_writes to off max_wal_senders to 0 wal_level to minimal Before completing the import job, CloudNativePG restores the expected configuration, then runs initdb --sync-only to ensure that data is permanently written on disk. Important WAL archiving, if requested, and WAL level will be honored after the database import process has completed. Similarly, replicas will be cloned after the bootstrap phase, when the actual cluster resource starts. There are other optimizations you can do during the import phase. Although this topic is beyond the scope of CloudNativePG, we recommend that you reduce unnecessary writes in the checkpoint area by tuning Postgres GUCs like shared_buffers , max_wal_size , checkpoint_timeout directly in the Cluster configuration.","title":"Import optimizations"},{"location":"database_import/#customizing-pg_dump-and-pg_restore-behavior","text":"You can customize the behavior of pg_dump and pg_restore by specifying additional options using the pgDumpExtraOptions and pgRestoreExtraOptions parameters. For instance, you can enable parallel jobs to speed up data import/export processes, as shown in the following example: # bootstrap: initdb: import: type: microservice databases: - app source: externalCluster: cluster-example pgDumpExtraOptions: - '--jobs=2' pgRestoreExtraOptions: - '--jobs=2' # Warning Use the pgDumpExtraOptions and pgRestoreExtraOptions fields with caution and at your own risk. These options are not validated or verified by the operator, and some configurations may conflict with its intended functionality or behavior. Always test thoroughly in a safe and controlled environment before applying them in production.","title":"Customizing pg_dump and pg_restore Behavior"},{"location":"database_import/#online-import-and-upgrades","text":"Logical replication offers a powerful way to import any PostgreSQL database accessible over the network using the following approach: Import Bootstrap with Schema-Only Option : Initialize the schema in the target database before replication begins. Subscription Resource : Set up continuous replication to synchronize data changes. This technique can also be leveraged for performing major PostgreSQL upgrades with minimal downtime, making it ideal for seamless migrations and system upgrades. For more details, including limitations and best practices, refer to the Logical Replication section in the documentation.","title":"Online Import and Upgrades"},{"location":"declarative_database_management/","text":"PostgreSQL Database Management CloudNativePG simplifies PostgreSQL database provisioning by automatically creating an application database named app by default. This default behavior is explained in the \"Bootstrap an Empty Cluster\" section. For more advanced use cases, CloudNativePG introduces declarative database management , which empowers users to define and control the lifecycle of PostgreSQL databases using the Database Custom Resource Definition (CRD). This method seamlessly integrates with Kubernetes, providing a scalable, automated, and consistent approach to managing PostgreSQL databases. Key Concepts Scope of Management Important CloudNativePG manages global objects in PostgreSQL clusters, such as databases, roles, and tablespaces. However, it does not manage the content of databases (e.g., schemas and tables). For database content, specialized tools or the applications themselves should be used. Declarative Database Manifest The following example demonstrates how a Database resource interacts with a Cluster : apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: name: cluster-example-one spec: name: one owner: app cluster: name: cluster-example When applied, this manifest creates a Database object called cluster-example-one requesting a database named one , owned by the app role, in the cluster-example PostgreSQL cluster. Info Please refer to the API reference the full list of attributes you can define for each Database object. Required Fields in the Database Manifest metadata.name : Unique name of the Kubernetes object within its namespace. spec.name : Name of the database as it will appear in PostgreSQL. spec.owner : PostgreSQL role that owns the database. spec.cluster.name : Name of the target PostgreSQL cluster. The Database object must reference a specific Cluster , determining where the database will be created. It is managed by the cluster's primary instance, ensuring the database is created or updated as needed. Info The distinction between metadata.name and spec.name allows multiple Database resources to reference databases with the same name across different CloudNativePG clusters in the same Kubernetes namespace. Reserved Database Names PostgreSQL automatically creates databases such as postgres , template0 , and template1 . These names are reserved and cannot be used for new Database objects in CloudNativePG. Important Creating a Database with spec.name set to postgres , template0 , or template1 is not allowed. Reconciliation and Status Once a Database object is reconciled successfully: status.applied will be set to true . status.observedGeneration will match the metadata.generation of the last applied configuration. Example of a reconciled Database object: apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: generation: 1 name: cluster-example-one spec: cluster: name: cluster-example name: one owner: app status: observedGeneration: 1 applied: true If an error occurs during reconciliation, status.applied will be false , and an error message will be included in the status.message field. Deleting a Database CloudNativePG supports two methods for database deletion: Using the delete reclaim policy Declaratively setting the database's ensure field to absent Deleting via delete Reclaim Policy The databaseReclaimPolicy field determines the behavior when a Database object is deleted: retain (default): The database remains in PostgreSQL for manual management. delete : The database is automatically removed from PostgreSQL. Example: apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: name: cluster-example-two spec: databaseReclaimPolicy: delete name: two owner: app cluster: name: cluster-example Deleting this Database object will automatically remove the two database from the cluster-example cluster. Declaratively Setting ensure: absent To remove a database, set the ensure field to absent like in the following example:. apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: name: cluster-example-database-to-drop spec: cluster: name: cluster-example name: database-to-drop owner: app ensure: absent This manifest ensures that the database-to-drop database is removed from the cluster-example cluster. Limitations and Caveats Renaming a database While CloudNativePG adheres to PostgreSQL\u2019s CREATE DATABASE and ALTER DATABASE commands, renaming databases is not supported . Attempting to modify spec.name in an existing Database object will result in rejection by Kubernetes. Creating vs. Altering a Database For new databases, CloudNativePG uses the CREATE DATABASE statement. For existing databases, ALTER DATABASE is used to apply changes. It is important to note that there are some differences between these two Postgres commands: in particular, the options accepted by ALTER are a subset of those accepted by CREATE . Warning Some fields, such as encoding and collation settings, are immutable in PostgreSQL. Attempts to modify these fields on existing databases will be ignored. Replica Clusters Database objects declared on replica clusters cannot be enforced, as replicas lack write privileges. These objects will remain in a pending state until the replica is promoted. Conflict Resolution If two Database objects in the same namespace manage the same PostgreSQL database (i.e., identical spec.name and spec.cluster.name ), the second object will be rejected. Example status message: status: applied: false message: 'reconciliation error: database \"one\" is already managed by Database object \"cluster-example-one\"' Postgres Version Differences CloudNativePG adheres to PostgreSQL's capabilities. For example, features like ICU_RULES introduced in PostgreSQL 16 are unavailable in earlier versions. Errors from PostgreSQL will be reflected in the Database object's status . Manual Changes CloudNativePG does not overwrite manual changes to databases. Once reconciled, a Database object will not be reapplied unless its metadata.generation changes, giving flexibility for direct PostgreSQL modifications.","title":"PostgreSQL Database Management"},{"location":"declarative_database_management/#postgresql-database-management","text":"CloudNativePG simplifies PostgreSQL database provisioning by automatically creating an application database named app by default. This default behavior is explained in the \"Bootstrap an Empty Cluster\" section. For more advanced use cases, CloudNativePG introduces declarative database management , which empowers users to define and control the lifecycle of PostgreSQL databases using the Database Custom Resource Definition (CRD). This method seamlessly integrates with Kubernetes, providing a scalable, automated, and consistent approach to managing PostgreSQL databases.","title":"PostgreSQL Database Management"},{"location":"declarative_database_management/#key-concepts","text":"","title":"Key Concepts"},{"location":"declarative_database_management/#scope-of-management","text":"Important CloudNativePG manages global objects in PostgreSQL clusters, such as databases, roles, and tablespaces. However, it does not manage the content of databases (e.g., schemas and tables). For database content, specialized tools or the applications themselves should be used.","title":"Scope of Management"},{"location":"declarative_database_management/#declarative-database-manifest","text":"The following example demonstrates how a Database resource interacts with a Cluster : apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: name: cluster-example-one spec: name: one owner: app cluster: name: cluster-example When applied, this manifest creates a Database object called cluster-example-one requesting a database named one , owned by the app role, in the cluster-example PostgreSQL cluster. Info Please refer to the API reference the full list of attributes you can define for each Database object.","title":"Declarative Database Manifest"},{"location":"declarative_database_management/#required-fields-in-the-database-manifest","text":"metadata.name : Unique name of the Kubernetes object within its namespace. spec.name : Name of the database as it will appear in PostgreSQL. spec.owner : PostgreSQL role that owns the database. spec.cluster.name : Name of the target PostgreSQL cluster. The Database object must reference a specific Cluster , determining where the database will be created. It is managed by the cluster's primary instance, ensuring the database is created or updated as needed. Info The distinction between metadata.name and spec.name allows multiple Database resources to reference databases with the same name across different CloudNativePG clusters in the same Kubernetes namespace.","title":"Required Fields in the Database Manifest"},{"location":"declarative_database_management/#reserved-database-names","text":"PostgreSQL automatically creates databases such as postgres , template0 , and template1 . These names are reserved and cannot be used for new Database objects in CloudNativePG. Important Creating a Database with spec.name set to postgres , template0 , or template1 is not allowed.","title":"Reserved Database Names"},{"location":"declarative_database_management/#reconciliation-and-status","text":"Once a Database object is reconciled successfully: status.applied will be set to true . status.observedGeneration will match the metadata.generation of the last applied configuration. Example of a reconciled Database object: apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: generation: 1 name: cluster-example-one spec: cluster: name: cluster-example name: one owner: app status: observedGeneration: 1 applied: true If an error occurs during reconciliation, status.applied will be false , and an error message will be included in the status.message field.","title":"Reconciliation and Status"},{"location":"declarative_database_management/#deleting-a-database","text":"CloudNativePG supports two methods for database deletion: Using the delete reclaim policy Declaratively setting the database's ensure field to absent","title":"Deleting a Database"},{"location":"declarative_database_management/#deleting-via-delete-reclaim-policy","text":"The databaseReclaimPolicy field determines the behavior when a Database object is deleted: retain (default): The database remains in PostgreSQL for manual management. delete : The database is automatically removed from PostgreSQL. Example: apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: name: cluster-example-two spec: databaseReclaimPolicy: delete name: two owner: app cluster: name: cluster-example Deleting this Database object will automatically remove the two database from the cluster-example cluster.","title":"Deleting via delete Reclaim Policy"},{"location":"declarative_database_management/#declaratively-setting-ensure-absent","text":"To remove a database, set the ensure field to absent like in the following example:. apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: name: cluster-example-database-to-drop spec: cluster: name: cluster-example name: database-to-drop owner: app ensure: absent This manifest ensures that the database-to-drop database is removed from the cluster-example cluster.","title":"Declaratively Setting ensure: absent"},{"location":"declarative_database_management/#limitations-and-caveats","text":"","title":"Limitations and Caveats"},{"location":"declarative_database_management/#renaming-a-database","text":"While CloudNativePG adheres to PostgreSQL\u2019s CREATE DATABASE and ALTER DATABASE commands, renaming databases is not supported . Attempting to modify spec.name in an existing Database object will result in rejection by Kubernetes.","title":"Renaming a database"},{"location":"declarative_database_management/#creating-vs-altering-a-database","text":"For new databases, CloudNativePG uses the CREATE DATABASE statement. For existing databases, ALTER DATABASE is used to apply changes. It is important to note that there are some differences between these two Postgres commands: in particular, the options accepted by ALTER are a subset of those accepted by CREATE . Warning Some fields, such as encoding and collation settings, are immutable in PostgreSQL. Attempts to modify these fields on existing databases will be ignored.","title":"Creating vs. Altering a Database"},{"location":"declarative_database_management/#replica-clusters","text":"Database objects declared on replica clusters cannot be enforced, as replicas lack write privileges. These objects will remain in a pending state until the replica is promoted.","title":"Replica Clusters"},{"location":"declarative_database_management/#conflict-resolution","text":"If two Database objects in the same namespace manage the same PostgreSQL database (i.e., identical spec.name and spec.cluster.name ), the second object will be rejected. Example status message: status: applied: false message: 'reconciliation error: database \"one\" is already managed by Database object \"cluster-example-one\"'","title":"Conflict Resolution"},{"location":"declarative_database_management/#postgres-version-differences","text":"CloudNativePG adheres to PostgreSQL's capabilities. For example, features like ICU_RULES introduced in PostgreSQL 16 are unavailable in earlier versions. Errors from PostgreSQL will be reflected in the Database object's status .","title":"Postgres Version Differences"},{"location":"declarative_database_management/#manual-changes","text":"CloudNativePG does not overwrite manual changes to databases. Once reconciled, a Database object will not be reapplied unless its metadata.generation changes, giving flexibility for direct PostgreSQL modifications.","title":"Manual Changes"},{"location":"declarative_hibernation/","text":"Declarative hibernation CloudNativePG is designed to keep PostgreSQL clusters up, running and available anytime. There are some kinds of workloads that require the database to be up only when the workload is active. Batch-driven solutions are one such case. In batch-driven solutions, the database needs to be up only when the batch process is running. The declarative hibernation feature enables saving CPU power by removing the database Pods, while keeping the database PVCs. Note Declarative hibernation is different from the existing implementation of imperative hibernation via the cnpg plugin . Imperative hibernation shuts down all Postgres instances in the High Availability cluster, and keeps a static copy of the PVCs of the primary that contain PGDATA and WALs. The plugin enables to exit the hibernation phase, by resuming the primary and then recreating all the replicas - if they exist. Hibernation To hibernate a cluster, set the cnpg.io/hibernation=on annotation: $ kubectl annotate cluster --overwrite cnpg.io/hibernation=on A hibernated cluster won't have any running Pods, while the PVCs are retained so that the cluster can be rehydrated at a later time. Replica PVCs will be kept in addition to the primary's PVC. The hibernation procedure will delete the primary Pod and then the replica Pods, avoiding switchover, to ensure the replicas are kept in sync. The hibernation status can be monitored by looking for the cnpg.io/hibernation condition: $ kubectl get cluster -o \"jsonpath={.status.conditions[?(.type==\\\"cnpg.io/hibernation\\\")]}\" { \"lastTransitionTime\":\"2023-03-05T16:43:35Z\", \"message\":\"Cluster has been hibernated\", \"reason\":\"Hibernated\", \"status\":\"True\", \"type\":\"cnpg.io/hibernation\" } The hibernation status can also be read with the status sub-command of the cnpg plugin for kubectl : $ kubectl cnpg status Cluster Summary Name: cluster-example Namespace: default PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:17.5 Primary instance: cluster-example-2 Status: Cluster in healthy state Instances: 3 Ready instances: 0 Hibernation Status Hibernated Message Cluster has been hibernated Time 2023-03-05 16:43:35 +0000 UTC [..] Rehydration To rehydrate a cluster, either set the cnpg.io/hibernation annotation to off : $ kubectl annotate cluster --overwrite cnpg.io/hibernation=off Or, just unset it altogether: $ kubectl annotate cluster cnpg.io/hibernation- The Pods will be recreated and the cluster will resume operation.","title":"Declarative hibernation"},{"location":"declarative_hibernation/#declarative-hibernation","text":"CloudNativePG is designed to keep PostgreSQL clusters up, running and available anytime. There are some kinds of workloads that require the database to be up only when the workload is active. Batch-driven solutions are one such case. In batch-driven solutions, the database needs to be up only when the batch process is running. The declarative hibernation feature enables saving CPU power by removing the database Pods, while keeping the database PVCs. Note Declarative hibernation is different from the existing implementation of imperative hibernation via the cnpg plugin . Imperative hibernation shuts down all Postgres instances in the High Availability cluster, and keeps a static copy of the PVCs of the primary that contain PGDATA and WALs. The plugin enables to exit the hibernation phase, by resuming the primary and then recreating all the replicas - if they exist.","title":"Declarative hibernation"},{"location":"declarative_hibernation/#hibernation","text":"To hibernate a cluster, set the cnpg.io/hibernation=on annotation: $ kubectl annotate cluster --overwrite cnpg.io/hibernation=on A hibernated cluster won't have any running Pods, while the PVCs are retained so that the cluster can be rehydrated at a later time. Replica PVCs will be kept in addition to the primary's PVC. The hibernation procedure will delete the primary Pod and then the replica Pods, avoiding switchover, to ensure the replicas are kept in sync. The hibernation status can be monitored by looking for the cnpg.io/hibernation condition: $ kubectl get cluster -o \"jsonpath={.status.conditions[?(.type==\\\"cnpg.io/hibernation\\\")]}\" { \"lastTransitionTime\":\"2023-03-05T16:43:35Z\", \"message\":\"Cluster has been hibernated\", \"reason\":\"Hibernated\", \"status\":\"True\", \"type\":\"cnpg.io/hibernation\" } The hibernation status can also be read with the status sub-command of the cnpg plugin for kubectl : $ kubectl cnpg status Cluster Summary Name: cluster-example Namespace: default PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:17.5 Primary instance: cluster-example-2 Status: Cluster in healthy state Instances: 3 Ready instances: 0 Hibernation Status Hibernated Message Cluster has been hibernated Time 2023-03-05 16:43:35 +0000 UTC [..]","title":"Hibernation"},{"location":"declarative_hibernation/#rehydration","text":"To rehydrate a cluster, either set the cnpg.io/hibernation annotation to off : $ kubectl annotate cluster --overwrite cnpg.io/hibernation=off Or, just unset it altogether: $ kubectl annotate cluster cnpg.io/hibernation- The Pods will be recreated and the cluster will resume operation.","title":"Rehydration"},{"location":"declarative_role_management/","text":"PostgreSQL Role Management From its inception, CloudNativePG has managed the creation of specific roles required in PostgreSQL instances: some reserved users, such as the postgres superuser, streaming_replica and cnpg_pooler_pgbouncer (when the PgBouncer Pooler is used) The application user, set as the low-privilege owner of the application database This process is described in the \"Bootstrap\" section. With the managed stanza in the cluster spec, CloudNativePG now provides full lifecycle management for roles specified in .spec.managed.roles . This feature enables declarative management of existing roles, as well as the creation of new roles if they are not already present in the database. Role creation will occur after the database bootstrapping is complete. An example manifest for a cluster with declarative role management can be found in the file cluster-example-with-roles.yaml . Here is an excerpt from that file: apiVersion: postgresql.cnpg.io/v1 kind: Cluster spec: managed: roles: - name: dante ensure: present comment: Dante Alighieri login: true superuser: false inRoles: - pg_monitor - pg_signal_backend The role specification in .spec.managed.roles adheres to the PostgreSQL structure and naming conventions . Please refer to the API reference for the full list of attributes you can define for each role. A few points are worth noting: The ensure attribute is not part of PostgreSQL. It enables declarative role management to create and remove roles. The two possible values are present (the default) and absent . The inherit attribute is true by default, following PostgreSQL conventions. The connectionLimit attribute defaults to -1, in line with PostgreSQL conventions. Role membership with inRoles defaults to no memberships. Declarative role management ensures that PostgreSQL instances align with the spec. If a user modifies role attributes directly in the database, the CloudNativePG operator will revert those changes during the next reconciliation cycle. Password management The declarative role management feature includes reconciling of role passwords. Passwords are managed in fundamentally different ways in the Kubernetes world and in PostgreSQL, and as a result there are a few things to note. Managed role configurations may optionally specify the name of a Secret where the username and password are stored (encoded in Base64 as is usual in Kubernetes). For example: managed: roles: - name: dante ensure: present [\u2026 snipped \u2026] passwordSecret: name: cluster-example-dante This would assume the existence of a Secret called cluster-example-dante , containing a username and password. The username should match the role we are setting the password for. For example, : apiVersion: v1 data: username: ZGFudGU= password: ZGFudGU= kind: Secret metadata: name: cluster-example-dante labels: cnpg.io/reload: \"true\" type: kubernetes.io/basic-auth If there is no passwordSecret specified for a role, the instance manager will not try to CREATE / ALTER the role with a password. This keeps with PostgreSQL conventions, where ALTER will not update passwords unless directed to with WITH PASSWORD . If a role was initially created with a password, and we would like to set the password to NULL in PostgreSQL, this necessitates being explicit on the part of the user of CloudNativePG. To distinguish \"no password provided in spec\" from \"set the password to NULL\", the field DisablePassword should be used. Imagine we decided we would like to have no password on the dante role in the database. In such case we would specify the following: managed: roles: - name: dante ensure: present [\u2026 snipped \u2026] disablePassword: true NOTE: it is considered an error to set both passwordSecret and disablePassword on a given role. This configuration will be rejected by the validation webhook. Password expiry, VALID UNTIL The VALID UNTIL role attribute in PostgreSQL controls password expiry. Roles created without VALID UNTIL specified get NULL by default in PostgreSQL, meaning that their password will never expire. PostgreSQL uses a timestamp type for VALID UNTIL , which includes support for the value 'infinity' indicating that the password never expires. Please see the PostgreSQL documentation for reference. With declarative role management, the validUntil attribute for managed roles controls password expiry. validUntil can only take: a Kubernetes timestamp, or be omitted (defaulting to null ) In the first case, the given validUntil timestamp will be set in the database as the VALID UNTIL attribute of the role. In the second case (omitted validUntil ) the operator will ensure password never expires, mirroring the behavior of PostgreSQL. Specifically: in case of new role, it will omit the VALID UNTIL clause in the role creation statement in case of existing role, it will set VALID UNTIL to infinity if VALID UNTIL was not set to NULL in the database (this is due to PostgreSQL not allowing VALID UNTIL NULL in the ALTER ROLE SQL statement) Warning New roles created without passwordSecret will have a NULL password inside PostgreSQL. Password hashed You can also provide pre-encrypted passwords by specifying the password in MD5/SCRAM-SHA-256 hash format: kind: Secret type: kubernetes.io/basic-auth metadata: name: cluster-example-cavalcanti labels: cnpg.io/reload: \"true\" apiVersion: v1 stringData: username: cavalcanti password: SCRAM-SHA-256$:$: Unrealizable role configurations In PostgreSQL, in some cases, commands cannot be honored by the database and will be rejected. Please refer to the PostgreSQL documentation on error codes for details. Role operations can produce such fundamental errors. Two examples: We ask PostgreSQL to create the role petrarca as a member of the role (group) poets , but poets does not exist. We ask PostgreSQL to drop the role dante , but the role dante is the owner of the database inferno . These fundamental errors cannot be fixed by the database, nor the CloudNativePG operator, without clarification from the human administrator. The two examples above could be fixed by creating the role poets or dropping the database inferno respectively, but they might have originated due to human error, and in such case, the \"fix\" proposed might be the wrong thing to do. CloudNativePG will record when such fundamental errors occur, and will display them in the cluster Status. Which segues into\u2026 Status of managed roles The Cluster status includes a section for the managed roles' status, as shown below: status: [\u2026snipped\u2026] managedRolesStatus: byStatus: not-managed: - app pending-reconciliation: - dante - petrarca reconciled: - ariosto reserved: - postgres - streaming_replica cannotReconcile: dante: - 'could not perform DELETE on role dante: owner of database inferno' petrarca: - 'could not perform UPDATE_MEMBERSHIPS on role petrarca: role \"poets\" does not exist' Note the special sub-section cannotReconcile for operations the database (and CloudNativePG) cannot honor, and which require human intervention. This section covers roles reserved for operator use and those that are not under declarative management, providing a comprehensive view of the roles in the database instances. The kubectl plugin also shows the status of managed roles in its status sub-command: Managed roles status Status Roles ------ ----- pending-reconciliation petrarca reconciled app,dante reserved postgres,streaming_replica Irreconcilable roles Role Errors ---- ------ petrarca could not perform UPDATE_MEMBERSHIPS on role petrarca: role \"poets\" does not exist Important In terms of backward compatibility, declarative role management is designed to ignore roles that exist in the database but are not included in the spec. The lifecycle of these roles will continue to be managed within PostgreSQL, allowing CloudNativePG users to adopt this feature at their convenience.","title":"PostgreSQL Role Management"},{"location":"declarative_role_management/#postgresql-role-management","text":"From its inception, CloudNativePG has managed the creation of specific roles required in PostgreSQL instances: some reserved users, such as the postgres superuser, streaming_replica and cnpg_pooler_pgbouncer (when the PgBouncer Pooler is used) The application user, set as the low-privilege owner of the application database This process is described in the \"Bootstrap\" section. With the managed stanza in the cluster spec, CloudNativePG now provides full lifecycle management for roles specified in .spec.managed.roles . This feature enables declarative management of existing roles, as well as the creation of new roles if they are not already present in the database. Role creation will occur after the database bootstrapping is complete. An example manifest for a cluster with declarative role management can be found in the file cluster-example-with-roles.yaml . Here is an excerpt from that file: apiVersion: postgresql.cnpg.io/v1 kind: Cluster spec: managed: roles: - name: dante ensure: present comment: Dante Alighieri login: true superuser: false inRoles: - pg_monitor - pg_signal_backend The role specification in .spec.managed.roles adheres to the PostgreSQL structure and naming conventions . Please refer to the API reference for the full list of attributes you can define for each role. A few points are worth noting: The ensure attribute is not part of PostgreSQL. It enables declarative role management to create and remove roles. The two possible values are present (the default) and absent . The inherit attribute is true by default, following PostgreSQL conventions. The connectionLimit attribute defaults to -1, in line with PostgreSQL conventions. Role membership with inRoles defaults to no memberships. Declarative role management ensures that PostgreSQL instances align with the spec. If a user modifies role attributes directly in the database, the CloudNativePG operator will revert those changes during the next reconciliation cycle.","title":"PostgreSQL Role Management"},{"location":"declarative_role_management/#password-management","text":"The declarative role management feature includes reconciling of role passwords. Passwords are managed in fundamentally different ways in the Kubernetes world and in PostgreSQL, and as a result there are a few things to note. Managed role configurations may optionally specify the name of a Secret where the username and password are stored (encoded in Base64 as is usual in Kubernetes). For example: managed: roles: - name: dante ensure: present [\u2026 snipped \u2026] passwordSecret: name: cluster-example-dante This would assume the existence of a Secret called cluster-example-dante , containing a username and password. The username should match the role we are setting the password for. For example, : apiVersion: v1 data: username: ZGFudGU= password: ZGFudGU= kind: Secret metadata: name: cluster-example-dante labels: cnpg.io/reload: \"true\" type: kubernetes.io/basic-auth If there is no passwordSecret specified for a role, the instance manager will not try to CREATE / ALTER the role with a password. This keeps with PostgreSQL conventions, where ALTER will not update passwords unless directed to with WITH PASSWORD . If a role was initially created with a password, and we would like to set the password to NULL in PostgreSQL, this necessitates being explicit on the part of the user of CloudNativePG. To distinguish \"no password provided in spec\" from \"set the password to NULL\", the field DisablePassword should be used. Imagine we decided we would like to have no password on the dante role in the database. In such case we would specify the following: managed: roles: - name: dante ensure: present [\u2026 snipped \u2026] disablePassword: true NOTE: it is considered an error to set both passwordSecret and disablePassword on a given role. This configuration will be rejected by the validation webhook.","title":"Password management"},{"location":"declarative_role_management/#password-expiry-valid-until","text":"The VALID UNTIL role attribute in PostgreSQL controls password expiry. Roles created without VALID UNTIL specified get NULL by default in PostgreSQL, meaning that their password will never expire. PostgreSQL uses a timestamp type for VALID UNTIL , which includes support for the value 'infinity' indicating that the password never expires. Please see the PostgreSQL documentation for reference. With declarative role management, the validUntil attribute for managed roles controls password expiry. validUntil can only take: a Kubernetes timestamp, or be omitted (defaulting to null ) In the first case, the given validUntil timestamp will be set in the database as the VALID UNTIL attribute of the role. In the second case (omitted validUntil ) the operator will ensure password never expires, mirroring the behavior of PostgreSQL. Specifically: in case of new role, it will omit the VALID UNTIL clause in the role creation statement in case of existing role, it will set VALID UNTIL to infinity if VALID UNTIL was not set to NULL in the database (this is due to PostgreSQL not allowing VALID UNTIL NULL in the ALTER ROLE SQL statement) Warning New roles created without passwordSecret will have a NULL password inside PostgreSQL.","title":"Password expiry, VALID UNTIL"},{"location":"declarative_role_management/#password-hashed","text":"You can also provide pre-encrypted passwords by specifying the password in MD5/SCRAM-SHA-256 hash format: kind: Secret type: kubernetes.io/basic-auth metadata: name: cluster-example-cavalcanti labels: cnpg.io/reload: \"true\" apiVersion: v1 stringData: username: cavalcanti password: SCRAM-SHA-256$:$:","title":"Password hashed"},{"location":"declarative_role_management/#unrealizable-role-configurations","text":"In PostgreSQL, in some cases, commands cannot be honored by the database and will be rejected. Please refer to the PostgreSQL documentation on error codes for details. Role operations can produce such fundamental errors. Two examples: We ask PostgreSQL to create the role petrarca as a member of the role (group) poets , but poets does not exist. We ask PostgreSQL to drop the role dante , but the role dante is the owner of the database inferno . These fundamental errors cannot be fixed by the database, nor the CloudNativePG operator, without clarification from the human administrator. The two examples above could be fixed by creating the role poets or dropping the database inferno respectively, but they might have originated due to human error, and in such case, the \"fix\" proposed might be the wrong thing to do. CloudNativePG will record when such fundamental errors occur, and will display them in the cluster Status. Which segues into\u2026","title":"Unrealizable role configurations"},{"location":"declarative_role_management/#status-of-managed-roles","text":"The Cluster status includes a section for the managed roles' status, as shown below: status: [\u2026snipped\u2026] managedRolesStatus: byStatus: not-managed: - app pending-reconciliation: - dante - petrarca reconciled: - ariosto reserved: - postgres - streaming_replica cannotReconcile: dante: - 'could not perform DELETE on role dante: owner of database inferno' petrarca: - 'could not perform UPDATE_MEMBERSHIPS on role petrarca: role \"poets\" does not exist' Note the special sub-section cannotReconcile for operations the database (and CloudNativePG) cannot honor, and which require human intervention. This section covers roles reserved for operator use and those that are not under declarative management, providing a comprehensive view of the roles in the database instances. The kubectl plugin also shows the status of managed roles in its status sub-command: Managed roles status Status Roles ------ ----- pending-reconciliation petrarca reconciled app,dante reserved postgres,streaming_replica Irreconcilable roles Role Errors ---- ------ petrarca could not perform UPDATE_MEMBERSHIPS on role petrarca: role \"poets\" does not exist Important In terms of backward compatibility, declarative role management is designed to ignore roles that exist in the database but are not included in the spec. The lifecycle of these roles will continue to be managed within PostgreSQL, allowing CloudNativePG users to adopt this feature at their convenience.","title":"Status of managed roles"},{"location":"e2e/","text":"End-to-End Tests CloudNativePG is automatically tested after each commit via a suite of End-to-end (E2E) tests (or integration tests) which ensure that the operator correctly deploys and manages PostgreSQL clusters. Kubernetes versions 1.25 through 1.29, and PostgreSQL versions 12 through 16, are tested for each commit, helping detect bugs at an early stage of the development process. For each tested version of Kubernetes and PostgreSQL, a Kubernetes cluster is created using kind , run on the GitHub Actions platform, and the following suite of E2E tests are performed on that cluster: Basic: Installation of the operator Creation of a Cluster Usage of a persistent volume for data storage Service connectivity: Connection via services, including read-only Connection via user-provided server and/or client certificates PgBouncer Self-healing: Failover Switchover Primary endpoint switch in case of failover in less than 10 seconds Primary endpoint switch in case of switchover in less than 20 seconds Recover from a degraded state in less than 60 seconds PVC Deletion Corrupted PVC Backup and Restore: Backup and restore from Volume Snapshots Backup and ScheduledBackups execution using Barman Cloud on S3 Backup and ScheduledBackups execution using Barman Cloud on Azure blob storage Restore from backup using Barman Cloud on S3 Restore from backup using Barman Cloud on Azure blob storage Point-in-time recovery (PITR) on Azure, S3 storage Wal-Restore (sequential / parallel) Operator: Operator Deployment Operator configuration via ConfigMap Operator pod deletion Operator pod eviction Operator upgrade Operator High Availability Observability: Metrics collection PgBouncer Metrics JSON log format Replication: Replication Slots Synchronous replication Scale-up and scale-down of a Cluster Logical replication via declarative Publication / Subscription Replica clusters Bootstrapping a replica cluster from backup Bootstrapping a replica cluster via streaming Bootstrapping via volume snapshots Detaching a replica cluster Plugin: Cluster Hibernation using CNPG plugin Fencing Creation of a connection certificate Postgres Configuration: Manage PostgreSQL configuration changes Rolling updates when changing PostgreSQL images Rolling updates when changing ImageCatalog/ClusterImageCatalog images Rolling updates on hot standby sensitive parameter changes Database initialization via InitDB Pod Scheduling: Tolerations and taints Pod affinity using NodeSelector Rolling updates on PodSpec drift detection In-place upgrades Multi-Arch availability Cluster Metadata: ConfigMap for Cluster Labels and Annotations Object metadata Recovery: Data corruption pg_basebackup Importing Databases: Microservice approach Monolith approach Storage: Storage expansion Dedicated PG_WAL persistent volume Security: AppArmor annotation propagation. Executed only on Azure environment Maintenance: Node Drain with maintenance window Node Drain with single-instance cluster with/without Pod Disruption Budgets Hibernation Declarative hibernation / rehydration Volume snapshots Backup/restore for cold and online snapshots Point-in-time recovery (PITR) for cold and online snapshots Backups via plugin for cold and online snapshots Declarative backups for cold and online snapshots Managed Roles Creation and update of managed roles Password maintenance using Kubernetes secrets Tablespaces Declarative creation of tablespaces Declarative creation of temporary tablespaces Backup / recovery from object storage Backup / recovery from volume snapshots Declarative databases Declarative creation of databases with default (retain) reclaim policy Declarative creation of databases with delete reclaim policy","title":"End-to-End Tests"},{"location":"e2e/#end-to-end-tests","text":"CloudNativePG is automatically tested after each commit via a suite of End-to-end (E2E) tests (or integration tests) which ensure that the operator correctly deploys and manages PostgreSQL clusters. Kubernetes versions 1.25 through 1.29, and PostgreSQL versions 12 through 16, are tested for each commit, helping detect bugs at an early stage of the development process. For each tested version of Kubernetes and PostgreSQL, a Kubernetes cluster is created using kind , run on the GitHub Actions platform, and the following suite of E2E tests are performed on that cluster: Basic: Installation of the operator Creation of a Cluster Usage of a persistent volume for data storage Service connectivity: Connection via services, including read-only Connection via user-provided server and/or client certificates PgBouncer Self-healing: Failover Switchover Primary endpoint switch in case of failover in less than 10 seconds Primary endpoint switch in case of switchover in less than 20 seconds Recover from a degraded state in less than 60 seconds PVC Deletion Corrupted PVC Backup and Restore: Backup and restore from Volume Snapshots Backup and ScheduledBackups execution using Barman Cloud on S3 Backup and ScheduledBackups execution using Barman Cloud on Azure blob storage Restore from backup using Barman Cloud on S3 Restore from backup using Barman Cloud on Azure blob storage Point-in-time recovery (PITR) on Azure, S3 storage Wal-Restore (sequential / parallel) Operator: Operator Deployment Operator configuration via ConfigMap Operator pod deletion Operator pod eviction Operator upgrade Operator High Availability Observability: Metrics collection PgBouncer Metrics JSON log format Replication: Replication Slots Synchronous replication Scale-up and scale-down of a Cluster Logical replication via declarative Publication / Subscription Replica clusters Bootstrapping a replica cluster from backup Bootstrapping a replica cluster via streaming Bootstrapping via volume snapshots Detaching a replica cluster Plugin: Cluster Hibernation using CNPG plugin Fencing Creation of a connection certificate Postgres Configuration: Manage PostgreSQL configuration changes Rolling updates when changing PostgreSQL images Rolling updates when changing ImageCatalog/ClusterImageCatalog images Rolling updates on hot standby sensitive parameter changes Database initialization via InitDB Pod Scheduling: Tolerations and taints Pod affinity using NodeSelector Rolling updates on PodSpec drift detection In-place upgrades Multi-Arch availability Cluster Metadata: ConfigMap for Cluster Labels and Annotations Object metadata Recovery: Data corruption pg_basebackup Importing Databases: Microservice approach Monolith approach Storage: Storage expansion Dedicated PG_WAL persistent volume Security: AppArmor annotation propagation. Executed only on Azure environment Maintenance: Node Drain with maintenance window Node Drain with single-instance cluster with/without Pod Disruption Budgets Hibernation Declarative hibernation / rehydration Volume snapshots Backup/restore for cold and online snapshots Point-in-time recovery (PITR) for cold and online snapshots Backups via plugin for cold and online snapshots Declarative backups for cold and online snapshots Managed Roles Creation and update of managed roles Password maintenance using Kubernetes secrets Tablespaces Declarative creation of tablespaces Declarative creation of temporary tablespaces Backup / recovery from object storage Backup / recovery from volume snapshots Declarative databases Declarative creation of databases with default (retain) reclaim policy Declarative creation of databases with delete reclaim policy","title":"End-to-End Tests"},{"location":"failover/","text":"Automated failover In the case of unexpected errors on the primary for longer than the .spec.failoverDelay (by default 0 seconds), the cluster will go into failover mode . This may happen, for example, when: The primary pod has a disk failure The primary pod is deleted The postgres container on the primary has any kind of sustained failure In the failover scenario, the primary cannot be assumed to be working properly. After cases like the ones above, the readiness probe for the primary pod will start failing. This will be picked up in the controller's reconciliation loop. The controller will initiate the failover process, in two steps: First, it will mark the TargetPrimary as pending . This change of state will force the primary pod to shutdown, to ensure the WAL receivers on the replicas will stop. The cluster will be marked in failover phase (\"Failing over\"). Once all WAL receivers are stopped, there will be a leader election, and a new primary will be named. The chosen instance will initiate promotion to primary, and, after this is completed, the cluster will resume normal operations. Meanwhile, the former primary pod will restart, detect that it is no longer the primary, and become a replica node. Important The two-phase procedure helps ensure the WAL receivers can stop in an orderly fashion, and that the failing primary will not start streaming WALs again upon restart. These safeguards prevent timeline discrepancies between the new primary and the replicas. During the time the failing primary is being shut down: It will first try a PostgreSQL's fast shutdown with .spec.switchoverDelay seconds as timeout. This graceful shutdown will attempt to archive pending WALs. If the fast shutdown fails, or its timeout is exceeded, a PostgreSQL's immediate shutdown is initiated. Info \"Fast\" mode does not wait for PostgreSQL clients to disconnect and will terminate an online backup in progress. All active transactions are rolled back and clients are forcibly disconnected, then the server is shut down. \"Immediate\" mode will abort all PostgreSQL server processes immediately, without a clean shutdown. RTO and RPO impact Failover may result in the service being impacted ( RTO ) and/or data being lost ( RPO ): During the time when the primary has started to fail, and before the controller starts failover procedures, queries in transit, WAL writes, checkpoints and similar operations, may fail. Once the fast shutdown command has been issued, the cluster will no longer accept connections, so service will be impacted but no data will be lost. If the fast shutdown fails, the immediate shutdown will stop any pending processes, including WAL writing. Data may be lost. During the time the primary is shutting down and a new primary hasn't yet started, the cluster will operate without a primary and thus be impaired - but with no data loss. Note The timeout that controls fast shutdown is set by .spec.switchoverDelay , as in the case of a switchover. Increasing the time for fast shutdown is safer from an RPO point of view, but possibly delays the return to normal operation - negatively affecting RTO. Warning As already mentioned in the \"Instance Manager\" section when explaining the switchover process, the .spec.switchoverDelay option affects the RPO and RTO of your PostgreSQL database. Setting it to a low value, might favor RTO over RPO but lead to data loss at cluster level and/or backup level. On the contrary, setting it to a high value, might remove the risk of data loss while leaving the cluster without an active primary for a longer time during the switchover. Delayed failover As anticipated above, the .spec.failoverDelay option allows you to delay the start of the failover procedure by a number of seconds after the primary has been detected to be unhealthy. By default, this setting is set to 0 , triggering the failover procedure immediately. Sometimes failing over to a new primary can be more disruptive than waiting for the primary to come back online. This is especially true of network disruptions where multiple tiers are affected (i.e., downstream logical subscribers) or when the time to perform the failover is longer than the expected outage. Enabling a new configuration option to delay failover provides a mechanism to prevent premature failover for short-lived network or node instability.","title":"Automated failover"},{"location":"failover/#automated-failover","text":"In the case of unexpected errors on the primary for longer than the .spec.failoverDelay (by default 0 seconds), the cluster will go into failover mode . This may happen, for example, when: The primary pod has a disk failure The primary pod is deleted The postgres container on the primary has any kind of sustained failure In the failover scenario, the primary cannot be assumed to be working properly. After cases like the ones above, the readiness probe for the primary pod will start failing. This will be picked up in the controller's reconciliation loop. The controller will initiate the failover process, in two steps: First, it will mark the TargetPrimary as pending . This change of state will force the primary pod to shutdown, to ensure the WAL receivers on the replicas will stop. The cluster will be marked in failover phase (\"Failing over\"). Once all WAL receivers are stopped, there will be a leader election, and a new primary will be named. The chosen instance will initiate promotion to primary, and, after this is completed, the cluster will resume normal operations. Meanwhile, the former primary pod will restart, detect that it is no longer the primary, and become a replica node. Important The two-phase procedure helps ensure the WAL receivers can stop in an orderly fashion, and that the failing primary will not start streaming WALs again upon restart. These safeguards prevent timeline discrepancies between the new primary and the replicas. During the time the failing primary is being shut down: It will first try a PostgreSQL's fast shutdown with .spec.switchoverDelay seconds as timeout. This graceful shutdown will attempt to archive pending WALs. If the fast shutdown fails, or its timeout is exceeded, a PostgreSQL's immediate shutdown is initiated. Info \"Fast\" mode does not wait for PostgreSQL clients to disconnect and will terminate an online backup in progress. All active transactions are rolled back and clients are forcibly disconnected, then the server is shut down. \"Immediate\" mode will abort all PostgreSQL server processes immediately, without a clean shutdown.","title":"Automated failover"},{"location":"failover/#rto-and-rpo-impact","text":"Failover may result in the service being impacted ( RTO ) and/or data being lost ( RPO ): During the time when the primary has started to fail, and before the controller starts failover procedures, queries in transit, WAL writes, checkpoints and similar operations, may fail. Once the fast shutdown command has been issued, the cluster will no longer accept connections, so service will be impacted but no data will be lost. If the fast shutdown fails, the immediate shutdown will stop any pending processes, including WAL writing. Data may be lost. During the time the primary is shutting down and a new primary hasn't yet started, the cluster will operate without a primary and thus be impaired - but with no data loss. Note The timeout that controls fast shutdown is set by .spec.switchoverDelay , as in the case of a switchover. Increasing the time for fast shutdown is safer from an RPO point of view, but possibly delays the return to normal operation - negatively affecting RTO. Warning As already mentioned in the \"Instance Manager\" section when explaining the switchover process, the .spec.switchoverDelay option affects the RPO and RTO of your PostgreSQL database. Setting it to a low value, might favor RTO over RPO but lead to data loss at cluster level and/or backup level. On the contrary, setting it to a high value, might remove the risk of data loss while leaving the cluster without an active primary for a longer time during the switchover.","title":"RTO and RPO impact"},{"location":"failover/#delayed-failover","text":"As anticipated above, the .spec.failoverDelay option allows you to delay the start of the failover procedure by a number of seconds after the primary has been detected to be unhealthy. By default, this setting is set to 0 , triggering the failover procedure immediately. Sometimes failing over to a new primary can be more disruptive than waiting for the primary to come back online. This is especially true of network disruptions where multiple tiers are affected (i.e., downstream logical subscribers) or when the time to perform the failover is longer than the expected outage. Enabling a new configuration option to delay failover provides a mechanism to prevent premature failover for short-lived network or node instability.","title":"Delayed failover"},{"location":"failure_modes/","text":"Failure Modes Note In previous versions of CloudNativePG, this page included specific failure scenarios. Since these largely follow standard Kubernetes behavior, we have streamlined the content to avoid duplication of information that belongs to the underlying Kubernetes stack and is not specific to CloudNativePG. CloudNativePG adheres to standard Kubernetes principles for self-healing and high availability. We assume familiarity with core Kubernetes concepts such as storage classes, PVCs, nodes, and Pods. For CloudNativePG-specific details, refer to the \"Postgres Instance Manager\" section , which covers startup, liveness, and readiness probes, as well as the self-healing section below. Important If you are running CloudNativePG in production, we strongly recommend seeking professional support . Self-Healing Primary Failure If the primary Pod fails: The operator promotes the most up-to-date standby with the lowest replication lag. The -rw service is updated to point to the new primary. The failed Pod is removed from the -r and -rw services. Standby Pods begin replicating from the new primary. The former primary uses pg_rewind to re-synchronize if its PVC is available; otherwise, a new standby is created from a backup of the new primary. Standby Failure If a standby Pod fails: It is removed from the -r and -ro services. The Pod is restarted using its PVC if available; otherwise, a new Pod is created from a backup of the current primary. Once ready, the Pod is re-added to the -r and -ro services. Manual Intervention For failure scenarios not covered by automated recovery, manual intervention may be required. Important Do not perform manual operations without professional support . Disabling Reconciliation To temporarily disable the reconciliation loop for a PostgreSQL cluster, use the cnpg.io/reconciliationLoop annotation: metadata: name: cluster-example-no-reconcile annotations: cnpg.io/reconciliationLoop: \"disabled\" spec: # ... Use this annotation with extreme caution and only during emergency operations. Warning This annotation should be removed as soon as the issue is resolved. Leaving it in place prevents the operator from executing self-healing actions, including failover.","title":"Failure Modes"},{"location":"failure_modes/#failure-modes","text":"Note In previous versions of CloudNativePG, this page included specific failure scenarios. Since these largely follow standard Kubernetes behavior, we have streamlined the content to avoid duplication of information that belongs to the underlying Kubernetes stack and is not specific to CloudNativePG. CloudNativePG adheres to standard Kubernetes principles for self-healing and high availability. We assume familiarity with core Kubernetes concepts such as storage classes, PVCs, nodes, and Pods. For CloudNativePG-specific details, refer to the \"Postgres Instance Manager\" section , which covers startup, liveness, and readiness probes, as well as the self-healing section below. Important If you are running CloudNativePG in production, we strongly recommend seeking professional support .","title":"Failure Modes"},{"location":"failure_modes/#self-healing","text":"","title":"Self-Healing"},{"location":"failure_modes/#primary-failure","text":"If the primary Pod fails: The operator promotes the most up-to-date standby with the lowest replication lag. The -rw service is updated to point to the new primary. The failed Pod is removed from the -r and -rw services. Standby Pods begin replicating from the new primary. The former primary uses pg_rewind to re-synchronize if its PVC is available; otherwise, a new standby is created from a backup of the new primary.","title":"Primary Failure"},{"location":"failure_modes/#standby-failure","text":"If a standby Pod fails: It is removed from the -r and -ro services. The Pod is restarted using its PVC if available; otherwise, a new Pod is created from a backup of the current primary. Once ready, the Pod is re-added to the -r and -ro services.","title":"Standby Failure"},{"location":"failure_modes/#manual-intervention","text":"For failure scenarios not covered by automated recovery, manual intervention may be required. Important Do not perform manual operations without professional support .","title":"Manual Intervention"},{"location":"failure_modes/#disabling-reconciliation","text":"To temporarily disable the reconciliation loop for a PostgreSQL cluster, use the cnpg.io/reconciliationLoop annotation: metadata: name: cluster-example-no-reconcile annotations: cnpg.io/reconciliationLoop: \"disabled\" spec: # ... Use this annotation with extreme caution and only during emergency operations. Warning This annotation should be removed as soon as the issue is resolved. Leaving it in place prevents the operator from executing self-healing actions, including failover.","title":"Disabling Reconciliation"},{"location":"faq/","text":"Frequently Asked Questions (FAQ) Running PostgreSQL in Kubernetes Everyone knows that stateful workloads like PostgreSQL cannot run in Kubernetes. Why do you say the contrary? An independent research survey commissioned by the Data on Kubernetes Community in September 2021 revealed that half of the respondents run most of their production workloads on Kubernetes. 90% of them believe that Kubernetes is ready for stateful workloads, and 70% of them run databases in production. Databases like Postgres. However, according to them, significant challenges remain, such as the knowledge gap (Kubernetes and Cloud Native, in general, have a steep learning curve) and the quality of Kubernetes operators. The latter is the reason why we believe that an operator like CloudNativePG highly contributes to the success of your project. For database fanatics like us, a real game-changer has been the introduction of the support for local persistent volumes in Kubernetes 1.14 in April 2019 . CloudNativePG is built on immutable application containers. What does it mean? According to the microservice architectural pattern, a container is designed to run a single application or process. As a result, such container images are built to run the main application as the single entry point (the so-called PID 1 process). In Kubernetes terms, the application is referred to as workload. Workloads can be stateless like a web application server or stateful like a database. Mapping this concept to PostgreSQL, an immutable application container is a single \"postgres\" process that is running and tied to a single and specific version - the one in the immutable container image. No other processes such as SSH or systemd, or syslog are allowed. Immutable Application Containers are in contrast with Mutable System Containers, which are still a very common way to interpret and use containers. Immutable means that a container won't be modified during its life: no updates, no patches, no configuration changes. If you must update the application code or apply a patch, you build a new image and redeploy it. Immutability makes deployments safer and more repeatable. For more information, please refer to \"Why EDB chose immutable application containers\" . What does Cloud Native mean? The Cloud Native Computing Foundation defines the term \" Cloud Native \". However, since the start of the Cloud Native PostgreSQL/CloudNativePG operator at 2ndQuadrant, the development team has been interpreting Cloud Native as three main concepts: An existing, healthy, genuine, and prosperous DevOps culture, founded on people, as well as principles and processes, which enables teams and organizations (as teams of teams) to continuously change so to innovate and accelerate the delivery of outcomes and produce value for the business in safer, more efficient, and more engaging ways A microservice architecture that is based on Immutable Application Containers A way to manage and orchestrate these containers, such as Kubernetes Currently, the standard de facto for container orchestration is Kubernetes, which automates the deployment, administration and scalability of Cloud Native Applications. Another definition of Cloud Native that resonates with us is the one defined by Ibryam and Hu\u00df in \"Kubernetes Patterns\", published by O'Reilly : Principles, Patterns, Tools to automate containerized microservices at scale Can I run CloudNativePG on bare metal Kubernetes? Yes, definitely. You can run Kubernetes on bare metal. And you can dedicate one or more physical worker nodes with locally attached storage to PostgreSQL workloads for maximum and predictable I/O performance. The actual Cloud Native PostgreSQL project, from which CloudNativePG originated, was born after a pilot project in 2019 that benchmarked storage and PostgreSQL on the same bare metal server, first directly in Linux, and then inside Kubernetes. As expected, the experiment showed only negligible performance impact introduced by the container running in Kubernetes through local persistent volumes, allowing the Cloud Native initiative to continue. Why should I use PostgreSQL replication instead of file system replication? Please read the \"Architecture: Synchronizing the state\" section. Why should I use an operator instead of running PostgreSQL as a container? The most basic approach to running PostgreSQL in Kubernetes is to have a pod, which is the smallest unit of deployment in Kubernetes, running a Postgres container with no replica. The volume hosting the Postgres data directory is mounted on the pod, and it usually resides on network storage. In this case, Kubernetes restarts the pod in case of a problem or moves it to another Kubernetes node. The most sophisticated approach is to run PostgreSQL using an operator. An operator is an extension of the Kubernetes controller and defines how a complex application works in business continuity contexts. The operator pattern is currently state of the art in Kubernetes for this purpose. An operator simulates the work of a human operator in an automated and programmatic way. Postgres is a complex application, and an operator not only needs to deploy a cluster (the first step), but also properly react after unexpected events. The typical example is that of a failover. An operator relies on Kubernetes for capabilities like self-healing, scalability, replication, high availability, backup, recovery, updates, access, resource control, storage management, and so on. It also facilitates the integration of a PostgreSQL cluster in the log management and monitoring infrastructure. CloudNativePG enables the definition of the desired state of a PostgreSQL cluster via declarative configuration. Kubernetes continuously makes sure that the current state of the infrastructure matches the desired one through reconciliation loops initiated by the Kubernetes controller. If the desired state and the actual state don't match, reconciliation loops trigger self-healing procedures. That's where an operator like CloudNativePG comes into play. Are there any other operators for Postgres out there? Yes, of course. And our advice is that you look at all of them and compare them with CloudNativePG before making your decision. You will see that most of these operators use an external failover management tool (Patroni or similar) and rely on StatefulSets. Here is a non exhaustive list, in chronological order from their publication on GitHub: Crunchy Data Postgres Operator (2017) Zalando Postgres Operator (2017) Stackgres (2020) Percona Operator for PostgreSQL (2021) Kubegres (2021) Feel free to report any relevant missing entry as a PR. Info The Data on Kubernetes Community (which includes some of our maintainers) is working on an independent and vendor neutral project to list the operators called Operator Feature Matrix . You say that CloudNativePG is a fully declarative operator. What do you mean by that? The easiest way is to explain declarative configuration through an example that highlights the differences with imperative configuration. In an imperative context, the state is defined as a series of tasks to be executed in sequence. So, we can get a three-node PostgreSQL cluster by creating the first instance, configuring the replication, cloning a second instance, and the third one. In a declarative approach, the state of a system is defined using configuration, namely: there's a PostgreSQL 13 cluster with two replicas. This approach highly simplifies change management operations, and when these are stored in source control systems like Git, it enables the Infrastructure as Code capability. And Kubernetes takes it farther than deployment, as it makes sure that our request is fulfilled at any time. What are the required skills to run PostgreSQL on Kubernetes? Running PostgreSQL on Kubernetes requires both PostgreSQL and Kubernetes skills in your DevOps team. The best experience is when database administrators familiarize themselves with Kubernetes core concepts and are able to interact with Kubernetes administrators. Our advice is for everyone that wants to fully exploit Cloud Native PostgreSQL to acquire the \"Certified Kubernetes Administrator (CKA)\" status from the CNCF certification program. Why isn't CloudNativePG using StatefulSets? CloudNativePG does not rely on StatefulSet resources, and instead manages the underlying PVCs directly by leveraging the selected storage class for dynamic provisioning. Please refer to the \"Custom Pod Controller\" section for details and reasons behind this decision. High availability What happens to the PostgreSQL clusters when the operator pod dies or it is not available for a certain amount of time? The CloudNativePG operator, among other things, is responsible for self-healing capabilities. As such, they might not be available during an outage of the operator. However, assuming that the outage does not affect the nodes where PostgreSQL clusters are running, the database will continue to serve normal operations, through the relevant Kubernetes services. Moreover, the instance manager , which runs inside each PostgreSQL pod will still work, making sure that the database server is up, including accessory services like logging, export of metrics, continuous archiving of WAL files, etc. To summarize: an outage of the operator does not necessarily imply a PostgreSQL database outage; it's like running a database without a DBA or system administrator. What are the reasons behind CloudNativePG not relying on a failover management tool like Patroni, repmgr, or Stolon? Although part of the team that develops CloudNativePG has been heavily involved in repmgr in the past, we decided to take a different approach and directly extend the Kubernetes controller and rely on the Kubernetes API server to hold the status of a Postgres cluster, and use it as the only source of truth to: control High Availability of a Postgres cluster primarily via automated failover and switchover, coordinating itself with the instance manager control the Kubernetes services, that is the entry points for your applications Should I manually resync a former primary with the new one following a failover? No. The operator does that automatically for you, and relies on pg_rewind to synchronize the former primary with the new one. Database management Why should I use PostgreSQL? We believe that PostgreSQL is the equivalent in the database area of what Linux represents in the operating system space. The current latest major version of Postgres is version 16, which ships out of the box: native streaming replication, both physical and logical continuous hot backup and point in time recovery declarative partitioning for horizontal table partitioning, which is a very well-known technique in the database area to improve vertical scalability on a single instance extensibility, with extensions like PostGIS for geographical databases parallel queries for vertical scalability JSON support, unleashing the multi-model hybrid database for both structured and unstructured data queried via standard SQL And so on ... How many databases should be hosted in a single PostgreSQL instance? Our recommendation is to dedicate a single PostgreSQL cluster (intended as primary and multiple standby servers) to a single database, entirely managed by a single microservice application. However, by leveraging the \"postgres\" superuser, it is possible to create as many users and databases as desired (subject to the available resources). The reason for this recommendation lies in the Cloud Native concept, based on microservices. In a pure microservice architecture, the microservice itself should own the data it manages exclusively. These could be flat files, queues, key-value stores, or, in our case, a PostgreSQL relational database containing both structured and unstructured data. The general idea is that only the microservice can access the database, including schema management and migrations. CloudNativePG has been designed to work this way out of the box, by default creating an application user and an application database owned by the aforementioned application user. Reserving a PostgreSQL instance to a single microservice owned database, enhances: resource management: in PostgreSQL, CPU, and memory constrained resources are generally handled at the instance level, not the database level, making it easier to integrate it with Kubernetes resource management policies at the pod level physical continuous backup and Point-In-Time-Recovery (PITR): given that PostgreSQL handles continuous backup and recovery at the instance level, having one database per instance simplifies PITR operations, differentiates retention policy management, and increases data protection of backups application updates: enable each application to decide their update policies without impacting other databases owned by different applications database updates: each application can decide which PostgreSQL version to use, and independently, when to upgrade to a different major version of PostgreSQL and at what conditions (e.g., cutover time) Is there an upper limit in database size for not considering Kubernetes? No, as Kubernetes is no different from virtual machines and bare metal as far as this is regarded. Practically, however, it depends on the available resources of your Kubernetes cluster. Our advice with very large databases (VLDB) is to consider a shared nothing architecture, where a Kubernetes worker node is dedicated to a single Postgres instance, with dedicated storage. We proved that this extreme architectural pattern works when we benchmarked running PostgreSQL on bare metal Kubernetes with local persistent volumes . Tablespaces and horizontal partitioning are data modeling techniques that you can use to improve the vertical scalability of you databases. How can I specify a time zone in the PostgreSQL cluster? PostgreSQL has an extensive support for time zones, as explained in the official documentation: Date time data types Client connections config options Although time zones can even be used at session, transaction and even as part of a query in PostgreSQL, a very common way is to set them up globally. With CloudNativePG you can configure the cluster level time zone in the .spec.postgresql.parameters section as in the following example: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: pg-italy spec: instances: 1 postgresql: parameters: timezone: \"Europe/Rome\" storage: size: 1Gi The time zone can be verified with: $ kubectl exec -ti pg-italy-1 -c postgres -- psql -x -c \"SHOW timezone\" -[ RECORD 1 ]--------- TimeZone | Europe/Rome What is the recommended architecture for best business continuity outcomes? As covered in the \"Architecture\" section, the main recommendation is to adopt shared nothing architectures as much as possible, by leveraging the native capabilities and resources that Kubernetes provides in a single cluster, namely: availability zones: spread your instances across different availability zones in the same Kubernetes cluster worker nodes: as a consequence, make sure that your Postgres instances reside on different Kubernetes worker nodes storage: use dedicated storage for each worker node running Postgres Use at least one standby, preferably at least two, so that you can configure synchronous replication in the cluster, introducing RPO =0 for high availability. If you do not have availability zones - normally the case of on-premise installations - separate on worker nodes and storage. Properly setup continuous backup on a local/regional object store. The same architecture that is in a single Kubernetes cluster can be replicated in another Kubernetes cluster (normally in another geographical area or region) through the replica cluster feature, providing disaster recovery and high availability at global scale. You can use the WAL archive in the primary object store to feed the replica in the other region, without having to provide a streaming connection, if the default maximum RPO of 5 minutes is enough for you. How can instances be stopped or started? Please look at \"Fencing\" or \"Hibernation\" . What are the global objects such as roles and databases that are automatically created by CloudNativePG? The operator automatically creates a user for the application (by default called app ) and a database for the application (by default called app ) which is owned by the aforementioned user. This way, the database is ready for a microservice adoption, as developers can control migrations using the app user, without requiring superuser access. Teams can then create another user for read-write operations through the \"Declarative role management\" feature and assign the required GRANT to the tables.","title":"Frequently Asked Questions (FAQ)"},{"location":"faq/#frequently-asked-questions-faq","text":"","title":"Frequently Asked Questions (FAQ)"},{"location":"faq/#running-postgresql-in-kubernetes","text":"Everyone knows that stateful workloads like PostgreSQL cannot run in Kubernetes. Why do you say the contrary? An independent research survey commissioned by the Data on Kubernetes Community in September 2021 revealed that half of the respondents run most of their production workloads on Kubernetes. 90% of them believe that Kubernetes is ready for stateful workloads, and 70% of them run databases in production. Databases like Postgres. However, according to them, significant challenges remain, such as the knowledge gap (Kubernetes and Cloud Native, in general, have a steep learning curve) and the quality of Kubernetes operators. The latter is the reason why we believe that an operator like CloudNativePG highly contributes to the success of your project. For database fanatics like us, a real game-changer has been the introduction of the support for local persistent volumes in Kubernetes 1.14 in April 2019 . CloudNativePG is built on immutable application containers. What does it mean? According to the microservice architectural pattern, a container is designed to run a single application or process. As a result, such container images are built to run the main application as the single entry point (the so-called PID 1 process). In Kubernetes terms, the application is referred to as workload. Workloads can be stateless like a web application server or stateful like a database. Mapping this concept to PostgreSQL, an immutable application container is a single \"postgres\" process that is running and tied to a single and specific version - the one in the immutable container image. No other processes such as SSH or systemd, or syslog are allowed. Immutable Application Containers are in contrast with Mutable System Containers, which are still a very common way to interpret and use containers. Immutable means that a container won't be modified during its life: no updates, no patches, no configuration changes. If you must update the application code or apply a patch, you build a new image and redeploy it. Immutability makes deployments safer and more repeatable. For more information, please refer to \"Why EDB chose immutable application containers\" . What does Cloud Native mean? The Cloud Native Computing Foundation defines the term \" Cloud Native \". However, since the start of the Cloud Native PostgreSQL/CloudNativePG operator at 2ndQuadrant, the development team has been interpreting Cloud Native as three main concepts: An existing, healthy, genuine, and prosperous DevOps culture, founded on people, as well as principles and processes, which enables teams and organizations (as teams of teams) to continuously change so to innovate and accelerate the delivery of outcomes and produce value for the business in safer, more efficient, and more engaging ways A microservice architecture that is based on Immutable Application Containers A way to manage and orchestrate these containers, such as Kubernetes Currently, the standard de facto for container orchestration is Kubernetes, which automates the deployment, administration and scalability of Cloud Native Applications. Another definition of Cloud Native that resonates with us is the one defined by Ibryam and Hu\u00df in \"Kubernetes Patterns\", published by O'Reilly : Principles, Patterns, Tools to automate containerized microservices at scale Can I run CloudNativePG on bare metal Kubernetes? Yes, definitely. You can run Kubernetes on bare metal. And you can dedicate one or more physical worker nodes with locally attached storage to PostgreSQL workloads for maximum and predictable I/O performance. The actual Cloud Native PostgreSQL project, from which CloudNativePG originated, was born after a pilot project in 2019 that benchmarked storage and PostgreSQL on the same bare metal server, first directly in Linux, and then inside Kubernetes. As expected, the experiment showed only negligible performance impact introduced by the container running in Kubernetes through local persistent volumes, allowing the Cloud Native initiative to continue. Why should I use PostgreSQL replication instead of file system replication? Please read the \"Architecture: Synchronizing the state\" section. Why should I use an operator instead of running PostgreSQL as a container? The most basic approach to running PostgreSQL in Kubernetes is to have a pod, which is the smallest unit of deployment in Kubernetes, running a Postgres container with no replica. The volume hosting the Postgres data directory is mounted on the pod, and it usually resides on network storage. In this case, Kubernetes restarts the pod in case of a problem or moves it to another Kubernetes node. The most sophisticated approach is to run PostgreSQL using an operator. An operator is an extension of the Kubernetes controller and defines how a complex application works in business continuity contexts. The operator pattern is currently state of the art in Kubernetes for this purpose. An operator simulates the work of a human operator in an automated and programmatic way. Postgres is a complex application, and an operator not only needs to deploy a cluster (the first step), but also properly react after unexpected events. The typical example is that of a failover. An operator relies on Kubernetes for capabilities like self-healing, scalability, replication, high availability, backup, recovery, updates, access, resource control, storage management, and so on. It also facilitates the integration of a PostgreSQL cluster in the log management and monitoring infrastructure. CloudNativePG enables the definition of the desired state of a PostgreSQL cluster via declarative configuration. Kubernetes continuously makes sure that the current state of the infrastructure matches the desired one through reconciliation loops initiated by the Kubernetes controller. If the desired state and the actual state don't match, reconciliation loops trigger self-healing procedures. That's where an operator like CloudNativePG comes into play. Are there any other operators for Postgres out there? Yes, of course. And our advice is that you look at all of them and compare them with CloudNativePG before making your decision. You will see that most of these operators use an external failover management tool (Patroni or similar) and rely on StatefulSets. Here is a non exhaustive list, in chronological order from their publication on GitHub: Crunchy Data Postgres Operator (2017) Zalando Postgres Operator (2017) Stackgres (2020) Percona Operator for PostgreSQL (2021) Kubegres (2021) Feel free to report any relevant missing entry as a PR. Info The Data on Kubernetes Community (which includes some of our maintainers) is working on an independent and vendor neutral project to list the operators called Operator Feature Matrix . You say that CloudNativePG is a fully declarative operator. What do you mean by that? The easiest way is to explain declarative configuration through an example that highlights the differences with imperative configuration. In an imperative context, the state is defined as a series of tasks to be executed in sequence. So, we can get a three-node PostgreSQL cluster by creating the first instance, configuring the replication, cloning a second instance, and the third one. In a declarative approach, the state of a system is defined using configuration, namely: there's a PostgreSQL 13 cluster with two replicas. This approach highly simplifies change management operations, and when these are stored in source control systems like Git, it enables the Infrastructure as Code capability. And Kubernetes takes it farther than deployment, as it makes sure that our request is fulfilled at any time. What are the required skills to run PostgreSQL on Kubernetes? Running PostgreSQL on Kubernetes requires both PostgreSQL and Kubernetes skills in your DevOps team. The best experience is when database administrators familiarize themselves with Kubernetes core concepts and are able to interact with Kubernetes administrators. Our advice is for everyone that wants to fully exploit Cloud Native PostgreSQL to acquire the \"Certified Kubernetes Administrator (CKA)\" status from the CNCF certification program. Why isn't CloudNativePG using StatefulSets? CloudNativePG does not rely on StatefulSet resources, and instead manages the underlying PVCs directly by leveraging the selected storage class for dynamic provisioning. Please refer to the \"Custom Pod Controller\" section for details and reasons behind this decision.","title":"Running PostgreSQL in Kubernetes"},{"location":"faq/#high-availability","text":"What happens to the PostgreSQL clusters when the operator pod dies or it is not available for a certain amount of time? The CloudNativePG operator, among other things, is responsible for self-healing capabilities. As such, they might not be available during an outage of the operator. However, assuming that the outage does not affect the nodes where PostgreSQL clusters are running, the database will continue to serve normal operations, through the relevant Kubernetes services. Moreover, the instance manager , which runs inside each PostgreSQL pod will still work, making sure that the database server is up, including accessory services like logging, export of metrics, continuous archiving of WAL files, etc. To summarize: an outage of the operator does not necessarily imply a PostgreSQL database outage; it's like running a database without a DBA or system administrator. What are the reasons behind CloudNativePG not relying on a failover management tool like Patroni, repmgr, or Stolon? Although part of the team that develops CloudNativePG has been heavily involved in repmgr in the past, we decided to take a different approach and directly extend the Kubernetes controller and rely on the Kubernetes API server to hold the status of a Postgres cluster, and use it as the only source of truth to: control High Availability of a Postgres cluster primarily via automated failover and switchover, coordinating itself with the instance manager control the Kubernetes services, that is the entry points for your applications Should I manually resync a former primary with the new one following a failover? No. The operator does that automatically for you, and relies on pg_rewind to synchronize the former primary with the new one.","title":"High availability"},{"location":"faq/#database-management","text":"Why should I use PostgreSQL? We believe that PostgreSQL is the equivalent in the database area of what Linux represents in the operating system space. The current latest major version of Postgres is version 16, which ships out of the box: native streaming replication, both physical and logical continuous hot backup and point in time recovery declarative partitioning for horizontal table partitioning, which is a very well-known technique in the database area to improve vertical scalability on a single instance extensibility, with extensions like PostGIS for geographical databases parallel queries for vertical scalability JSON support, unleashing the multi-model hybrid database for both structured and unstructured data queried via standard SQL And so on ... How many databases should be hosted in a single PostgreSQL instance? Our recommendation is to dedicate a single PostgreSQL cluster (intended as primary and multiple standby servers) to a single database, entirely managed by a single microservice application. However, by leveraging the \"postgres\" superuser, it is possible to create as many users and databases as desired (subject to the available resources). The reason for this recommendation lies in the Cloud Native concept, based on microservices. In a pure microservice architecture, the microservice itself should own the data it manages exclusively. These could be flat files, queues, key-value stores, or, in our case, a PostgreSQL relational database containing both structured and unstructured data. The general idea is that only the microservice can access the database, including schema management and migrations. CloudNativePG has been designed to work this way out of the box, by default creating an application user and an application database owned by the aforementioned application user. Reserving a PostgreSQL instance to a single microservice owned database, enhances: resource management: in PostgreSQL, CPU, and memory constrained resources are generally handled at the instance level, not the database level, making it easier to integrate it with Kubernetes resource management policies at the pod level physical continuous backup and Point-In-Time-Recovery (PITR): given that PostgreSQL handles continuous backup and recovery at the instance level, having one database per instance simplifies PITR operations, differentiates retention policy management, and increases data protection of backups application updates: enable each application to decide their update policies without impacting other databases owned by different applications database updates: each application can decide which PostgreSQL version to use, and independently, when to upgrade to a different major version of PostgreSQL and at what conditions (e.g., cutover time) Is there an upper limit in database size for not considering Kubernetes? No, as Kubernetes is no different from virtual machines and bare metal as far as this is regarded. Practically, however, it depends on the available resources of your Kubernetes cluster. Our advice with very large databases (VLDB) is to consider a shared nothing architecture, where a Kubernetes worker node is dedicated to a single Postgres instance, with dedicated storage. We proved that this extreme architectural pattern works when we benchmarked running PostgreSQL on bare metal Kubernetes with local persistent volumes . Tablespaces and horizontal partitioning are data modeling techniques that you can use to improve the vertical scalability of you databases. How can I specify a time zone in the PostgreSQL cluster? PostgreSQL has an extensive support for time zones, as explained in the official documentation: Date time data types Client connections config options Although time zones can even be used at session, transaction and even as part of a query in PostgreSQL, a very common way is to set them up globally. With CloudNativePG you can configure the cluster level time zone in the .spec.postgresql.parameters section as in the following example: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: pg-italy spec: instances: 1 postgresql: parameters: timezone: \"Europe/Rome\" storage: size: 1Gi The time zone can be verified with: $ kubectl exec -ti pg-italy-1 -c postgres -- psql -x -c \"SHOW timezone\" -[ RECORD 1 ]--------- TimeZone | Europe/Rome What is the recommended architecture for best business continuity outcomes? As covered in the \"Architecture\" section, the main recommendation is to adopt shared nothing architectures as much as possible, by leveraging the native capabilities and resources that Kubernetes provides in a single cluster, namely: availability zones: spread your instances across different availability zones in the same Kubernetes cluster worker nodes: as a consequence, make sure that your Postgres instances reside on different Kubernetes worker nodes storage: use dedicated storage for each worker node running Postgres Use at least one standby, preferably at least two, so that you can configure synchronous replication in the cluster, introducing RPO =0 for high availability. If you do not have availability zones - normally the case of on-premise installations - separate on worker nodes and storage. Properly setup continuous backup on a local/regional object store. The same architecture that is in a single Kubernetes cluster can be replicated in another Kubernetes cluster (normally in another geographical area or region) through the replica cluster feature, providing disaster recovery and high availability at global scale. You can use the WAL archive in the primary object store to feed the replica in the other region, without having to provide a streaming connection, if the default maximum RPO of 5 minutes is enough for you. How can instances be stopped or started? Please look at \"Fencing\" or \"Hibernation\" . What are the global objects such as roles and databases that are automatically created by CloudNativePG? The operator automatically creates a user for the application (by default called app ) and a database for the application (by default called app ) which is owned by the aforementioned user. This way, the database is ready for a microservice adoption, as developers can control migrations using the app user, without requiring superuser access. Teams can then create another user for read-write operations through the \"Declarative role management\" feature and assign the required GRANT to the tables.","title":"Database management"},{"location":"fencing/","text":"Fencing Fencing in CloudNativePG is the ultimate process of protecting the data in one, more, or even all instances of a PostgreSQL cluster when they appear to be malfunctioning. When an instance is fenced, the PostgreSQL server process ( postmaster ) is guaranteed to be shut down, while the pod is kept running. This makes sure that, until the fence is lifted, data on the pod is not modified by PostgreSQL and that the file system can be investigated for debugging and troubleshooting purposes. How to fence instances In CloudNativePG you can fence: a specific instance a list of instances an entire Postgres Cluster Fencing is controlled through the content of the cnpg.io/fencedInstances annotation, which expects a JSON formatted list of instance names. If the annotation is set to '[\"*\"]' , a singleton list with a wildcard, the whole cluster is fenced. If the annotation is set to an empty JSON list, the operator behaves as if the annotation was not set. For example: cnpg.io/fencedInstances: '[\"cluster-example-1\"]' will fence just the cluster-example-1 instance cnpg.io/fencedInstances: '[\"cluster-example-1\",\"cluster-example-2\"]' will fence the cluster-example-1 and cluster-example-2 instances cnpg.io/fencedInstances: '[\"*\"]' will fence every instance in the cluster. The annotation can be manually set on the Kubernetes object, for example via the kubectl annotate command, or in a transparent way using the kubectl cnpg fencing on subcommand: # to fence only one instance kubectl cnpg fencing on cluster-example 1 # to fence all the instances in a Cluster kubectl cnpg fencing on cluster-example \"*\" Here is an example of a Cluster with an instance that was previously fenced: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: annotations: cnpg.io/fencedInstances: '[\"cluster-example-1\"]' [...] How to lift fencing Fencing can be lifted by clearing the annotation, or set it to a different value. As for fencing, this can be done either manually with kubectl annotate , or using the kubectl cnpg fencing subcommand as follows: # to lift the fencing only for one instance # N.B.: at the moment this won't work if the whole cluster was fenced previously, # in that case you will have to manually set the annotation as explained above kubectl cnpg fencing off cluster-example 1 # to lift the fencing for all the instances in a Cluster kubectl cnpg fencing off cluster-example \"*\" How fencing works Once an instance is set for fencing, the procedure to shut down the postmaster process is initiated, identical to the one of the switchover. This consists of an initial fast shutdown with a timeout set to .spec.switchoverDelay , followed by an immediate shutdown. Then: the Pod will be kept alive the Pod won't be marked as Ready all the changes that don't require the Postgres instance to be up will be reconciled, including: configuration files certificates and all the cryptographic material metrics will not be collected, except cnpg_collector_fencing_on which will be set to 1 Warning If a primary instance is fenced, its postmaster process is shut down but no failover is performed, interrupting the operativity of the applications. When the fence will be lifted, the primary instance will be started up again without performing a failover. Given that, we advise users to fence primary instances only if strictly required. If a fenced instance is deleted, the pod will be recreated normally, but the postmaster won't be started. This can be extremely helpful when instances are Crashlooping .","title":"Fencing"},{"location":"fencing/#fencing","text":"Fencing in CloudNativePG is the ultimate process of protecting the data in one, more, or even all instances of a PostgreSQL cluster when they appear to be malfunctioning. When an instance is fenced, the PostgreSQL server process ( postmaster ) is guaranteed to be shut down, while the pod is kept running. This makes sure that, until the fence is lifted, data on the pod is not modified by PostgreSQL and that the file system can be investigated for debugging and troubleshooting purposes.","title":"Fencing"},{"location":"fencing/#how-to-fence-instances","text":"In CloudNativePG you can fence: a specific instance a list of instances an entire Postgres Cluster Fencing is controlled through the content of the cnpg.io/fencedInstances annotation, which expects a JSON formatted list of instance names. If the annotation is set to '[\"*\"]' , a singleton list with a wildcard, the whole cluster is fenced. If the annotation is set to an empty JSON list, the operator behaves as if the annotation was not set. For example: cnpg.io/fencedInstances: '[\"cluster-example-1\"]' will fence just the cluster-example-1 instance cnpg.io/fencedInstances: '[\"cluster-example-1\",\"cluster-example-2\"]' will fence the cluster-example-1 and cluster-example-2 instances cnpg.io/fencedInstances: '[\"*\"]' will fence every instance in the cluster. The annotation can be manually set on the Kubernetes object, for example via the kubectl annotate command, or in a transparent way using the kubectl cnpg fencing on subcommand: # to fence only one instance kubectl cnpg fencing on cluster-example 1 # to fence all the instances in a Cluster kubectl cnpg fencing on cluster-example \"*\" Here is an example of a Cluster with an instance that was previously fenced: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: annotations: cnpg.io/fencedInstances: '[\"cluster-example-1\"]' [...]","title":"How to fence instances"},{"location":"fencing/#how-to-lift-fencing","text":"Fencing can be lifted by clearing the annotation, or set it to a different value. As for fencing, this can be done either manually with kubectl annotate , or using the kubectl cnpg fencing subcommand as follows: # to lift the fencing only for one instance # N.B.: at the moment this won't work if the whole cluster was fenced previously, # in that case you will have to manually set the annotation as explained above kubectl cnpg fencing off cluster-example 1 # to lift the fencing for all the instances in a Cluster kubectl cnpg fencing off cluster-example \"*\"","title":"How to lift fencing"},{"location":"fencing/#how-fencing-works","text":"Once an instance is set for fencing, the procedure to shut down the postmaster process is initiated, identical to the one of the switchover. This consists of an initial fast shutdown with a timeout set to .spec.switchoverDelay , followed by an immediate shutdown. Then: the Pod will be kept alive the Pod won't be marked as Ready all the changes that don't require the Postgres instance to be up will be reconciled, including: configuration files certificates and all the cryptographic material metrics will not be collected, except cnpg_collector_fencing_on which will be set to 1 Warning If a primary instance is fenced, its postmaster process is shut down but no failover is performed, interrupting the operativity of the applications. When the fence will be lifted, the primary instance will be started up again without performing a failover. Given that, we advise users to fence primary instances only if strictly required. If a fenced instance is deleted, the pod will be recreated normally, but the postmaster won't be started. This can be extremely helpful when instances are Crashlooping .","title":"How fencing works"},{"location":"image_catalog/","text":"Image Catalog ImageCatalog and ClusterImageCatalog are essential resources that empower you to define images for creating a Cluster . The key distinction lies in their scope: an ImageCatalog is namespaced, while a ClusterImageCatalog is cluster-scoped. Both share a common structure, comprising a list of images, each equipped with a major field indicating the major version of the image. Warning The operator places trust in the user-defined major version and refrains from conducting any PostgreSQL version detection. It is the user's responsibility to ensure alignment between the declared major version in the catalog and the PostgreSQL image. The major field's value must remain unique within a catalog, preventing duplication across images. Distinct catalogs, however, may expose different images under the same major value. Example of a Namespaced ImageCatalog : apiVersion: postgresql.cnpg.io/v1 kind: ImageCatalog metadata: name: postgresql namespace: default spec: images: - major: 15 image: ghcr.io/cloudnative-pg/postgresql:15.6 - major: 16 image: ghcr.io/cloudnative-pg/postgresql:16.8 - major: 17 image: ghcr.io/cloudnative-pg/postgresql:17.5 Example of a Cluster-Wide Catalog using ClusterImageCatalog Resource: apiVersion: postgresql.cnpg.io/v1 kind: ClusterImageCatalog metadata: name: postgresql spec: images: - major: 15 image: ghcr.io/cloudnative-pg/postgresql:15.6 - major: 16 image: ghcr.io/cloudnative-pg/postgresql:16.8 - major: 17 image: ghcr.io/cloudnative-pg/postgresql:17.5 A Cluster resource has the flexibility to reference either an ImageCatalog or a ClusterImageCatalog to precisely specify the desired image. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 3 imageCatalogRef: apiGroup: postgresql.cnpg.io kind: ImageCatalog name: postgresql major: 16 storage: size: 1Gi Clusters utilizing these catalogs maintain continuous monitoring. Any alterations to the images within a catalog trigger automatic updates for all associated clusters referencing that specific entry. CloudNativePG Catalogs The CloudNativePG project maintains ClusterImageCatalogs for the images it provides. These catalogs are regularly updated with the latest images for each major version. By applying the ClusterImageCatalog.yaml file from the CloudNativePG project's GitHub repositories, cluster administrators can ensure that their clusters are automatically updated to the latest version within the specified major release. PostgreSQL Container Images You can install the latest version of the cluster catalog for the PostgreSQL Container Images ( cloudnative-pg/postgres-containers repository) with: kubectl apply \\ -f https://raw.githubusercontent.com/cloudnative-pg/postgres-containers/main/Debian/ClusterImageCatalog-bookworm.yaml PostGIS Container Images You can install the latest version of the cluster catalog for the PostGIS Container Images ( cloudnative-pg/postgis-containers repository) with: kubectl apply \\ -f https://raw.githubusercontent.com/cloudnative-pg/postgis-containers/main/PostGIS/ClusterImageCatalog.yaml","title":"Image Catalog"},{"location":"image_catalog/#image-catalog","text":"ImageCatalog and ClusterImageCatalog are essential resources that empower you to define images for creating a Cluster . The key distinction lies in their scope: an ImageCatalog is namespaced, while a ClusterImageCatalog is cluster-scoped. Both share a common structure, comprising a list of images, each equipped with a major field indicating the major version of the image. Warning The operator places trust in the user-defined major version and refrains from conducting any PostgreSQL version detection. It is the user's responsibility to ensure alignment between the declared major version in the catalog and the PostgreSQL image. The major field's value must remain unique within a catalog, preventing duplication across images. Distinct catalogs, however, may expose different images under the same major value. Example of a Namespaced ImageCatalog : apiVersion: postgresql.cnpg.io/v1 kind: ImageCatalog metadata: name: postgresql namespace: default spec: images: - major: 15 image: ghcr.io/cloudnative-pg/postgresql:15.6 - major: 16 image: ghcr.io/cloudnative-pg/postgresql:16.8 - major: 17 image: ghcr.io/cloudnative-pg/postgresql:17.5 Example of a Cluster-Wide Catalog using ClusterImageCatalog Resource: apiVersion: postgresql.cnpg.io/v1 kind: ClusterImageCatalog metadata: name: postgresql spec: images: - major: 15 image: ghcr.io/cloudnative-pg/postgresql:15.6 - major: 16 image: ghcr.io/cloudnative-pg/postgresql:16.8 - major: 17 image: ghcr.io/cloudnative-pg/postgresql:17.5 A Cluster resource has the flexibility to reference either an ImageCatalog or a ClusterImageCatalog to precisely specify the desired image. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 3 imageCatalogRef: apiGroup: postgresql.cnpg.io kind: ImageCatalog name: postgresql major: 16 storage: size: 1Gi Clusters utilizing these catalogs maintain continuous monitoring. Any alterations to the images within a catalog trigger automatic updates for all associated clusters referencing that specific entry.","title":"Image Catalog"},{"location":"image_catalog/#cloudnativepg-catalogs","text":"The CloudNativePG project maintains ClusterImageCatalogs for the images it provides. These catalogs are regularly updated with the latest images for each major version. By applying the ClusterImageCatalog.yaml file from the CloudNativePG project's GitHub repositories, cluster administrators can ensure that their clusters are automatically updated to the latest version within the specified major release.","title":"CloudNativePG Catalogs"},{"location":"image_catalog/#postgresql-container-images","text":"You can install the latest version of the cluster catalog for the PostgreSQL Container Images ( cloudnative-pg/postgres-containers repository) with: kubectl apply \\ -f https://raw.githubusercontent.com/cloudnative-pg/postgres-containers/main/Debian/ClusterImageCatalog-bookworm.yaml","title":"PostgreSQL Container Images"},{"location":"image_catalog/#postgis-container-images","text":"You can install the latest version of the cluster catalog for the PostGIS Container Images ( cloudnative-pg/postgis-containers repository) with: kubectl apply \\ -f https://raw.githubusercontent.com/cloudnative-pg/postgis-containers/main/PostGIS/ClusterImageCatalog.yaml","title":"PostGIS Container Images"},{"location":"installation_upgrade/","text":"Installation and upgrades Installation on Kubernetes Directly using the operator manifest The operator can be installed like any other resource in Kubernetes, through a YAML manifest applied via kubectl . You can install the latest operator manifest for this minor release as follows: kubectl apply --server-side -f \\ https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.25/releases/cnpg-1.25.3.yaml You can verify that with: kubectl get deployment -n cnpg-system cnpg-controller-manager Using the cnpg plugin for kubectl You can use the cnpg plugin to override the default configuration options that are in the static manifests. For example, to generate the default latest manifest but change the watch namespaces to only be a specific namespace, you could run: kubectl cnpg install generate \\ --watch-namespace \"specific-namespace\" \\ > cnpg_for_specific_namespace.yaml Please refer to \" cnpg plugin\" documentation for a more comprehensive example. Warning If you are deploying CloudNativePG on GKE and get an error ( ... failed to call webhook... ), be aware that by default traffic between worker nodes and control plane is blocked by the firewall except for a few specific ports, as explained in the official docs and by this issue . You'll need to either change the targetPort in the webhook service, to be one of the allowed ones, or open the webhooks' port ( 9443 ) on the firewall. Testing the latest development snapshot If you want to test or evaluate the latest development snapshot of CloudNativePG before the next official patch release, you can download the manifests from the cloudnative-pg/artifacts which provides easy access to the current trunk (main) as well as to each supported release. For example, you can install the latest snapshot of the operator with: curl -sSfL \\ https://raw.githubusercontent.com/cloudnative-pg/artifacts/main/manifests/operator-manifest.yaml | \\ kubectl apply --server-side -f - If you are instead looking for the latest snapshot of the operator for this specific minor release, you can just run: curl -sSfL \\ https://raw.githubusercontent.com/cloudnative-pg/artifacts/release-1.25/manifests/operator-manifest.yaml | \\ kubectl apply --server-side -f - Important Snapshots are not supported by the CloudNativePG Community, and are not intended for use in production. Using the Helm Chart The operator can be installed using the provided Helm chart . Using OLM CloudNativePG can also be installed via the Operator Lifecycle Manager (OLM) directly from OperatorHub.io . For deployments on Red Hat OpenShift, EDB offers and fully supports a certified version of CloudNativePG, available through the Red Hat OpenShift Container Platform . Details about the deployment In Kubernetes, the operator is by default installed in the cnpg-system namespace as a Kubernetes Deployment . The name of this deployment depends on the installation method. When installed through the manifest or the cnpg plugin, it is called cnpg-controller-manager by default. When installed via Helm, the default name is cnpg-cloudnative-pg . Note With Helm you can customize the name of the deployment via the fullnameOverride field in the \"values.yaml\" file . You can get more information using the describe command in kubectl : $ kubectl get deployments -n cnpg-system NAME READY UP-TO-DATE AVAILABLE AGE 1/1 1 1 18m kubectl describe deploy \\ -n cnpg-system \\ As with any Deployment, it sits on top of a ReplicaSet and supports rolling upgrades. The default configuration of the CloudNativePG operator comes with a Deployment of a single replica, which is suitable for most installations. In case the node where the pod is running is not reachable anymore, the pod will be rescheduled on another node. If you require high availability at the operator level, it is possible to specify multiple replicas in the Deployment configuration - given that the operator supports leader election. Also, you can take advantage of taints and tolerations to make sure that the operator does not run on the same nodes where the actual PostgreSQL clusters are running (this might even include the control plane for self-managed Kubernetes installations). Operator configuration You can change the default behavior of the operator by overriding some default options. For more information, please refer to the \"Operator configuration\" section. Upgrades Important Please carefully read the release notes before performing an upgrade as some versions might require extra steps. Upgrading CloudNativePG operator is a two-step process: upgrade the controller and the related Kubernetes resources upgrade the instance manager running in every PostgreSQL pod Unless differently stated in the release notes, the first step is normally done by applying the manifest of the newer version for plain Kubernetes installations, or using the native package manager of the used distribution (please follow the instructions in the above sections). The second step is automatically triggered after updating the controller. By default, this initiates a rolling update of every deployed PostgreSQL cluster, upgrading one instance at a time to use the new instance manager. The rolling update concludes with a switchover, which is governed by the primaryUpdateStrategy option. The default value, unsupervised , completes the switchover automatically. If set to supervised , the user must manually promote the new primary instance using the cnpg plugin for kubectl . Rolling updates This process is discussed in-depth on the Rolling Updates page. Important In case primaryUpdateStrategy is set to the default value of unsupervised , an upgrade of the operator will trigger a switchover on your PostgreSQL cluster, causing a (normally negligible) downtime. If your PostgreSQL Cluster has only one instance, the instance will be automatically restarted as supervised value is not supported for primaryUpdateStrategy . In either case, your applications will have to reconnect to PostgreSQL. The default rolling update behavior can be replaced with in-place updates of the instance manager. This approach does not require a restart of the PostgreSQL instance, thereby avoiding a switchover within the cluster. This feature, which is disabled by default, is described in detail below. Spread Upgrades By default, all PostgreSQL clusters are rolled out simultaneously, which may lead to a spike in resource usage, especially when managing multiple clusters. CloudNativePG provides two configuration options at the operator level that allow you to introduce delays between cluster roll-outs or even between instances within the same cluster, helping to distribute resource usage over time: CLUSTERS_ROLLOUT_DELAY : Defines the number of seconds to wait between roll-outs of different PostgreSQL clusters (default: 0 ). INSTANCES_ROLLOUT_DELAY : Defines the number of seconds to wait between roll-outs of individual instances within the same PostgreSQL cluster (default: 0 ). In-place updates of the instance manager By default, CloudNativePG issues a rolling update of the cluster every time the operator is updated. The new instance manager shipped with the operator is added to each PostgreSQL pod via an init container. However, this behavior can be changed via configuration to enable in-place updates of the instance manager, which is the PID 1 process that keeps the container alive. Internally, each instance manager in CloudNativePG supports the injection of a new executable that replaces the existing one after successfully completing an integrity verification phase and gracefully terminating all internal processes. Upon restarting with the new binary, the instance manager seamlessly adopts the already running postmaster . As a result, the PostgreSQL process is unaffected by the update, refraining from the need to perform a switchover. The other side of the coin, is that the Pod is changed after the start, breaking the pure concept of immutability. You can enable this feature by setting the ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES environment variable to 'true' in the operator configuration . The in-place upgrade process will not change the init container image inside the Pods. Therefore, the Pod definition will not reflect the current version of the operator. Compatibility among versions CloudNativePG follows semantic versioning. Every release of the operator within the same API version is compatible with the previous one. The current API version is v1, corresponding to versions 1.x.y of the operator. In addition to new features, new versions of the operator contain bug fixes and stability enhancements. Because of this, we strongly encourage users to upgrade to the latest version of the operator , as each version is released in order to maintain the most secure and stable Postgres environment. CloudNativePG currently releases new versions of the operator at least monthly. If you are unable to apply updates as each version becomes available, we recommend upgrading through each version in sequential order to come current periodically and not skipping versions. The release notes page contains a detailed list of the changes introduced in every released version of CloudNativePG, and it must be read before upgrading to a newer version of the software. Most versions are directly upgradable and in that case, applying the newer manifest for plain Kubernetes installations or using the native package manager of the chosen distribution is enough. When versions are not directly upgradable, the old version needs to be removed before installing the new one. This won't affect user data but only the operator itself. Upgrading to 1.25 from a previous minor version Important We strongly recommend that all CloudNativePG users upgrade to version 1.25.1 or at least to the latest stable version of the minor release you are currently using (namely 1.24.x). Warning Every time you are upgrading to a higher minor release, make sure you go through the release notes and upgrade instructions of all the intermediate minor releases. For example, if you want to move from 1.23.x to 1.25, make sure you go through the release notes and upgrade instructions for 1.24 and 1.25. No changes to existing 1.24 cluster configurations are required when upgrading to 1.25. Upgrading to 1.24 from a previous minor version From Replica Clusters to Distributed Topology One of the key enhancements in CloudNativePG 1.24.0 is the upgrade of the replica cluster feature. The former replica cluster feature, now referred to as the \"Standalone Replica Cluster,\" is no longer recommended for Disaster Recovery (DR) and High Availability (HA) scenarios that span multiple Kubernetes clusters. Standalone replica clusters are best suited for read-only workloads, such as reporting, OLAP, or creating development environments with test data. For DR and HA purposes, CloudNativePG now introduces the Distributed Topology strategy for replica clusters. This new strategy allows you to build PostgreSQL clusters across private, public, hybrid, and multi-cloud environments, spanning multiple regions and potentially different cloud providers. It also provides an API to control the switchover operation, ensuring that only one cluster acts as the primary at any given time. This Distributed Topology strategy enhances resilience and scalability, making it a robust solution for modern, distributed applications that require high availability and disaster recovery capabilities across diverse infrastructure setups. You can seamlessly transition from a previous replica cluster configuration to a distributed topology by modifying all the Cluster resources involved in the distributed PostgreSQL setup. Ensure the following steps are taken: Configure the externalClusters section to include all the clusters involved in the distributed topology. We strongly suggest using the same configuration across all Cluster resources for maintainability and consistency. Configure the primary and source fields in the .spec.replica stanza to reflect the distributed topology. The primary field should contain the name of the current primary cluster in the distributed topology, while the source field should contain the name of the cluster each Cluster resource is replicating from. It is important to note that the enabled field, which was previously set to true or false , should now be unset (default). For more information, please refer to the \"Distributed Topology\" section for replica clusters .","title":"Installation and upgrades"},{"location":"installation_upgrade/#installation-and-upgrades","text":"","title":"Installation and upgrades"},{"location":"installation_upgrade/#installation-on-kubernetes","text":"","title":"Installation on Kubernetes"},{"location":"installation_upgrade/#directly-using-the-operator-manifest","text":"The operator can be installed like any other resource in Kubernetes, through a YAML manifest applied via kubectl . You can install the latest operator manifest for this minor release as follows: kubectl apply --server-side -f \\ https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.25/releases/cnpg-1.25.3.yaml You can verify that with: kubectl get deployment -n cnpg-system cnpg-controller-manager","title":"Directly using the operator manifest"},{"location":"installation_upgrade/#using-the-cnpg-plugin-for-kubectl","text":"You can use the cnpg plugin to override the default configuration options that are in the static manifests. For example, to generate the default latest manifest but change the watch namespaces to only be a specific namespace, you could run: kubectl cnpg install generate \\ --watch-namespace \"specific-namespace\" \\ > cnpg_for_specific_namespace.yaml Please refer to \" cnpg plugin\" documentation for a more comprehensive example. Warning If you are deploying CloudNativePG on GKE and get an error ( ... failed to call webhook... ), be aware that by default traffic between worker nodes and control plane is blocked by the firewall except for a few specific ports, as explained in the official docs and by this issue . You'll need to either change the targetPort in the webhook service, to be one of the allowed ones, or open the webhooks' port ( 9443 ) on the firewall.","title":"Using the cnpg plugin for kubectl"},{"location":"installation_upgrade/#testing-the-latest-development-snapshot","text":"If you want to test or evaluate the latest development snapshot of CloudNativePG before the next official patch release, you can download the manifests from the cloudnative-pg/artifacts which provides easy access to the current trunk (main) as well as to each supported release. For example, you can install the latest snapshot of the operator with: curl -sSfL \\ https://raw.githubusercontent.com/cloudnative-pg/artifacts/main/manifests/operator-manifest.yaml | \\ kubectl apply --server-side -f - If you are instead looking for the latest snapshot of the operator for this specific minor release, you can just run: curl -sSfL \\ https://raw.githubusercontent.com/cloudnative-pg/artifacts/release-1.25/manifests/operator-manifest.yaml | \\ kubectl apply --server-side -f - Important Snapshots are not supported by the CloudNativePG Community, and are not intended for use in production.","title":"Testing the latest development snapshot"},{"location":"installation_upgrade/#using-the-helm-chart","text":"The operator can be installed using the provided Helm chart .","title":"Using the Helm Chart"},{"location":"installation_upgrade/#using-olm","text":"CloudNativePG can also be installed via the Operator Lifecycle Manager (OLM) directly from OperatorHub.io . For deployments on Red Hat OpenShift, EDB offers and fully supports a certified version of CloudNativePG, available through the Red Hat OpenShift Container Platform .","title":"Using OLM"},{"location":"installation_upgrade/#details-about-the-deployment","text":"In Kubernetes, the operator is by default installed in the cnpg-system namespace as a Kubernetes Deployment . The name of this deployment depends on the installation method. When installed through the manifest or the cnpg plugin, it is called cnpg-controller-manager by default. When installed via Helm, the default name is cnpg-cloudnative-pg . Note With Helm you can customize the name of the deployment via the fullnameOverride field in the \"values.yaml\" file . You can get more information using the describe command in kubectl : $ kubectl get deployments -n cnpg-system NAME READY UP-TO-DATE AVAILABLE AGE 1/1 1 1 18m kubectl describe deploy \\ -n cnpg-system \\ As with any Deployment, it sits on top of a ReplicaSet and supports rolling upgrades. The default configuration of the CloudNativePG operator comes with a Deployment of a single replica, which is suitable for most installations. In case the node where the pod is running is not reachable anymore, the pod will be rescheduled on another node. If you require high availability at the operator level, it is possible to specify multiple replicas in the Deployment configuration - given that the operator supports leader election. Also, you can take advantage of taints and tolerations to make sure that the operator does not run on the same nodes where the actual PostgreSQL clusters are running (this might even include the control plane for self-managed Kubernetes installations). Operator configuration You can change the default behavior of the operator by overriding some default options. For more information, please refer to the \"Operator configuration\" section.","title":"Details about the deployment"},{"location":"installation_upgrade/#upgrades","text":"Important Please carefully read the release notes before performing an upgrade as some versions might require extra steps. Upgrading CloudNativePG operator is a two-step process: upgrade the controller and the related Kubernetes resources upgrade the instance manager running in every PostgreSQL pod Unless differently stated in the release notes, the first step is normally done by applying the manifest of the newer version for plain Kubernetes installations, or using the native package manager of the used distribution (please follow the instructions in the above sections). The second step is automatically triggered after updating the controller. By default, this initiates a rolling update of every deployed PostgreSQL cluster, upgrading one instance at a time to use the new instance manager. The rolling update concludes with a switchover, which is governed by the primaryUpdateStrategy option. The default value, unsupervised , completes the switchover automatically. If set to supervised , the user must manually promote the new primary instance using the cnpg plugin for kubectl . Rolling updates This process is discussed in-depth on the Rolling Updates page. Important In case primaryUpdateStrategy is set to the default value of unsupervised , an upgrade of the operator will trigger a switchover on your PostgreSQL cluster, causing a (normally negligible) downtime. If your PostgreSQL Cluster has only one instance, the instance will be automatically restarted as supervised value is not supported for primaryUpdateStrategy . In either case, your applications will have to reconnect to PostgreSQL. The default rolling update behavior can be replaced with in-place updates of the instance manager. This approach does not require a restart of the PostgreSQL instance, thereby avoiding a switchover within the cluster. This feature, which is disabled by default, is described in detail below.","title":"Upgrades"},{"location":"installation_upgrade/#spread-upgrades","text":"By default, all PostgreSQL clusters are rolled out simultaneously, which may lead to a spike in resource usage, especially when managing multiple clusters. CloudNativePG provides two configuration options at the operator level that allow you to introduce delays between cluster roll-outs or even between instances within the same cluster, helping to distribute resource usage over time: CLUSTERS_ROLLOUT_DELAY : Defines the number of seconds to wait between roll-outs of different PostgreSQL clusters (default: 0 ). INSTANCES_ROLLOUT_DELAY : Defines the number of seconds to wait between roll-outs of individual instances within the same PostgreSQL cluster (default: 0 ).","title":"Spread Upgrades"},{"location":"installation_upgrade/#in-place-updates-of-the-instance-manager","text":"By default, CloudNativePG issues a rolling update of the cluster every time the operator is updated. The new instance manager shipped with the operator is added to each PostgreSQL pod via an init container. However, this behavior can be changed via configuration to enable in-place updates of the instance manager, which is the PID 1 process that keeps the container alive. Internally, each instance manager in CloudNativePG supports the injection of a new executable that replaces the existing one after successfully completing an integrity verification phase and gracefully terminating all internal processes. Upon restarting with the new binary, the instance manager seamlessly adopts the already running postmaster . As a result, the PostgreSQL process is unaffected by the update, refraining from the need to perform a switchover. The other side of the coin, is that the Pod is changed after the start, breaking the pure concept of immutability. You can enable this feature by setting the ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES environment variable to 'true' in the operator configuration . The in-place upgrade process will not change the init container image inside the Pods. Therefore, the Pod definition will not reflect the current version of the operator.","title":"In-place updates of the instance manager"},{"location":"installation_upgrade/#compatibility-among-versions","text":"CloudNativePG follows semantic versioning. Every release of the operator within the same API version is compatible with the previous one. The current API version is v1, corresponding to versions 1.x.y of the operator. In addition to new features, new versions of the operator contain bug fixes and stability enhancements. Because of this, we strongly encourage users to upgrade to the latest version of the operator , as each version is released in order to maintain the most secure and stable Postgres environment. CloudNativePG currently releases new versions of the operator at least monthly. If you are unable to apply updates as each version becomes available, we recommend upgrading through each version in sequential order to come current periodically and not skipping versions. The release notes page contains a detailed list of the changes introduced in every released version of CloudNativePG, and it must be read before upgrading to a newer version of the software. Most versions are directly upgradable and in that case, applying the newer manifest for plain Kubernetes installations or using the native package manager of the chosen distribution is enough. When versions are not directly upgradable, the old version needs to be removed before installing the new one. This won't affect user data but only the operator itself.","title":"Compatibility among versions"},{"location":"installation_upgrade/#upgrading-to-125-from-a-previous-minor-version","text":"Important We strongly recommend that all CloudNativePG users upgrade to version 1.25.1 or at least to the latest stable version of the minor release you are currently using (namely 1.24.x). Warning Every time you are upgrading to a higher minor release, make sure you go through the release notes and upgrade instructions of all the intermediate minor releases. For example, if you want to move from 1.23.x to 1.25, make sure you go through the release notes and upgrade instructions for 1.24 and 1.25. No changes to existing 1.24 cluster configurations are required when upgrading to 1.25.","title":"Upgrading to 1.25 from a previous minor version"},{"location":"installation_upgrade/#upgrading-to-124-from-a-previous-minor-version","text":"","title":"Upgrading to 1.24 from a previous minor version"},{"location":"installation_upgrade/#from-replica-clusters-to-distributed-topology","text":"One of the key enhancements in CloudNativePG 1.24.0 is the upgrade of the replica cluster feature. The former replica cluster feature, now referred to as the \"Standalone Replica Cluster,\" is no longer recommended for Disaster Recovery (DR) and High Availability (HA) scenarios that span multiple Kubernetes clusters. Standalone replica clusters are best suited for read-only workloads, such as reporting, OLAP, or creating development environments with test data. For DR and HA purposes, CloudNativePG now introduces the Distributed Topology strategy for replica clusters. This new strategy allows you to build PostgreSQL clusters across private, public, hybrid, and multi-cloud environments, spanning multiple regions and potentially different cloud providers. It also provides an API to control the switchover operation, ensuring that only one cluster acts as the primary at any given time. This Distributed Topology strategy enhances resilience and scalability, making it a robust solution for modern, distributed applications that require high availability and disaster recovery capabilities across diverse infrastructure setups. You can seamlessly transition from a previous replica cluster configuration to a distributed topology by modifying all the Cluster resources involved in the distributed PostgreSQL setup. Ensure the following steps are taken: Configure the externalClusters section to include all the clusters involved in the distributed topology. We strongly suggest using the same configuration across all Cluster resources for maintainability and consistency. Configure the primary and source fields in the .spec.replica stanza to reflect the distributed topology. The primary field should contain the name of the current primary cluster in the distributed topology, while the source field should contain the name of the cluster each Cluster resource is replicating from. It is important to note that the enabled field, which was previously set to true or false , should now be unset (default). For more information, please refer to the \"Distributed Topology\" section for replica clusters .","title":"From Replica Clusters to Distributed Topology"},{"location":"instance_manager/","text":"Postgres instance manager CloudNativePG does not rely on an external tool for failover management. It simply relies on the Kubernetes API server and a native key component called: the Postgres instance manager . The instance manager takes care of the entire lifecycle of the PostgreSQL server process (also known as postmaster ). When you create a new cluster, the operator makes a Pod per instance. The field .spec.instances specifies how many instances to create. Each Pod will start the instance manager as the parent process (PID 1) for the main container, which in turn runs the PostgreSQL instance. During the lifetime of the Pod, the instance manager acts as a backend to handle the startup, liveness and readiness probes . Startup, Liveness, and Readiness Probes CloudNativePG leverages PostgreSQL's pg_isready to implement Kubernetes startup, liveness, and readiness probes. Startup Probe The startup probe ensures that a PostgreSQL instance, whether a primary or standby, has fully started according to pg_isready . While the startup probe is running, the liveness and readiness probes remain disabled. Following Kubernetes standards, if the startup probe fails, the kubelet will terminate the container, which will then be restarted. The startup probe provided by CloudNativePG is configurable via the parameter .spec.startDelay , which specifies the maximum time, in seconds, allowed for the startup probe to succeed. At a minimum, the probe requires pg_isready to return 0 or 1 . By default, the startDelay is set to 3600 seconds. It is recommended to adjust this setting based on the time PostgreSQL needs to fully initialize in your specific environment. Warning Setting .spec.startDelay too low can cause the liveness probe to activate prematurely, potentially resulting in unnecessary Pod restarts if PostgreSQL hasn\u2019t fully initialized. CloudNativePG configures the startup probe with the following default parameters: failureThreshold: FAILURE_THRESHOLD periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 The failureThreshold value is automatically calculated by dividing startDelay by periodSeconds . You can customize any of the probe settings in the .spec.probes.startup section of your configuration. Warning Be sure that any custom probe settings are tailored to your cluster's operational requirements to avoid unintended disruptions. Info For more details on probe configuration, refer to the probe API documentation . If you manually specify .spec.probes.startup.failureThreshold , it will override the default behavior and disable the automatic use of startDelay . For example, the following configuration explicitly sets custom probe parameters, bypassing startDelay : # ... snip spec: probes: startup: periodSeconds: 3 timeoutSeconds: 3 failureThreshold: 10 Liveness Probe The liveness probe begins after the startup probe successfully completes. Its primary role is to ensure the PostgreSQL instance\u2014whether primary or standby\u2014is operating correctly. This is achieved using the pg_isready utility. Both exit codes 0 (indicating the server is accepting connections) and 1 (indicating the server is rejecting connections, such as during startup or a smart shutdown) are treated as valid outcomes. Following Kubernetes standards, if the liveness probe fails, the kubelet will terminate the container, which will then be restarted. The amount of time before a Pod is classified as not alive is configurable via the .spec.livenessProbeTimeout parameter. CloudNativePG configures the liveness probe with the following default parameters: failureThreshold: FAILURE_THRESHOLD periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 The failureThreshold value is automatically calculated by dividing livenessProbeTimeout by periodSeconds . By default, .spec.livenessProbeTimeout is set to 30 seconds. This means the liveness probe will report a failure if it detects three consecutive probe failures, with a 10-second interval between each check. You can customize any of the probe settings in the .spec.probes.liveness section of your configuration. Warning Be sure that any custom probe settings are tailored to your cluster's operational requirements to avoid unintended disruptions. Info For more details on probe configuration, refer to the probe API documentation . If you manually specify .spec.probes.liveness.failureThreshold , it will override the default behavior and disable the automatic use of livenessProbeTimeout . For example, the following configuration explicitly sets custom probe parameters, bypassing livenessProbeTimeout : # ... snip spec: probes: liveness: periodSeconds: 3 timeoutSeconds: 3 failureThreshold: 10 Readiness Probe The readiness probe begins once the startup probe has successfully completed. Its purpose is to check whether the PostgreSQL instance is ready to accept traffic and serve requests. For streaming replicas, it also requires that they have connected to the source at least once. Following Kubernetes standards, if the readiness probe fails, the pod will be marked unready and will not receive traffic from any services. CloudNativePG uses the following default configuration for the readiness probe: failureThreshold: 3 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 If the default settings do not suit your requirements, you can fully customize the readiness probe by specifying parameters in the .spec.probes.readiness stanza. For example: # ... snip spec: probes: readiness: periodSeconds: 3 timeoutSeconds: 3 failureThreshold: 10 Warning Ensure that any custom probe settings are aligned with your cluster\u2019s operational requirements to prevent unintended disruptions. Info For more information on configuring probes, see the probe API . Shutdown control When a Pod running Postgres is deleted, either manually or by Kubernetes following a node drain operation, the kubelet will send a termination signal to the instance manager, and the instance manager will take care of shutting down PostgreSQL in an appropriate way. The .spec.smartShutdownTimeout and .spec.stopDelay options, expressed in seconds, control the amount of time given to PostgreSQL to shut down. The values default to 180 and 1800 seconds, respectively. The shutdown procedure is composed of two steps: The instance manager requests a smart shut down, disallowing any new connection to PostgreSQL. This step will last for up to .spec.smartShutdownTimeout seconds. If PostgreSQL is still up, the instance manager requests a fast shut down, terminating any existing connection and exiting promptly. If the instance is archiving and/or streaming WAL files, the process will wait for up to the remaining time set in .spec.stopDelay to complete the operation and then forcibly shut down. Such a timeout needs to be at least 15 seconds. Important In order to avoid any data loss in the Postgres cluster, which impacts the database RPO , don't delete the Pod where the primary instance is running. In this case, perform a switchover to another instance first. Shutdown of the primary during a switchover During a switchover, the shutdown procedure is slightly different from the general case. Indeed, the operator requires the former primary to issue a fast shut down before the selected new primary can be promoted, in order to ensure that all the data are available on the new primary. For this reason, the .spec.switchoverDelay , expressed in seconds, controls the time given to the former primary to shut down gracefully and archive all the WAL files. By default it is set to 3600 (1 hour). Warning The .spec.switchoverDelay option affects the RPO and RTO of your PostgreSQL database. Setting it to a low value, might favor RTO over RPO but lead to data loss at cluster level and/or backup level. On the contrary, setting it to a high value, might remove the risk of data loss while leaving the cluster without an active primary for a longer time during the switchover. Failover In case of primary pod failure, the cluster will go into failover mode. Please refer to the \"Failover\" section for details. Disk Full Failure Storage exhaustion is a well known issue for PostgreSQL clusters. The PostgreSQL documentation highlights the possible failure scenarios and the importance of monitoring disk usage to prevent it from becoming full. The same applies to CloudNativePG and Kubernetes as well: the \"Monitoring\" section provides details on checking the disk space used by WAL segments and standard metrics on disk usage exported to Prometheus. Important In a production system, it is critical to monitor the database continuously. Exhausted disk storage can lead to a database server shutdown. Note The detection of exhausted storage relies on a storage class that accurately reports disk size and usage. This may not be the case in simulated Kubernetes environments like Kind or with test storage class implementations such as csi-driver-host-path . If the disk containing the WALs becomes full and no more WAL segments can be stored, PostgreSQL will stop working. CloudNativePG correctly detects this issue by verifying that there is enough space to store the next WAL segment, and avoids triggering a failover, which could complicate recovery. That allows a human administrator to address the root cause. In such a case, if supported by the storage class, the quickest course of action is currently to: Expand the storage size of the full PVC Increase the size in the Cluster resource to the same value Once the issue is resolved and there is sufficient free space for WAL segments, the Pod will restart and the cluster will become healthy. See also the \"Volume expansion\" section of the documentation.","title":"Postgres instance manager"},{"location":"instance_manager/#postgres-instance-manager","text":"CloudNativePG does not rely on an external tool for failover management. It simply relies on the Kubernetes API server and a native key component called: the Postgres instance manager . The instance manager takes care of the entire lifecycle of the PostgreSQL server process (also known as postmaster ). When you create a new cluster, the operator makes a Pod per instance. The field .spec.instances specifies how many instances to create. Each Pod will start the instance manager as the parent process (PID 1) for the main container, which in turn runs the PostgreSQL instance. During the lifetime of the Pod, the instance manager acts as a backend to handle the startup, liveness and readiness probes .","title":"Postgres instance manager"},{"location":"instance_manager/#startup-liveness-and-readiness-probes","text":"CloudNativePG leverages PostgreSQL's pg_isready to implement Kubernetes startup, liveness, and readiness probes.","title":"Startup, Liveness, and Readiness Probes"},{"location":"instance_manager/#startup-probe","text":"The startup probe ensures that a PostgreSQL instance, whether a primary or standby, has fully started according to pg_isready . While the startup probe is running, the liveness and readiness probes remain disabled. Following Kubernetes standards, if the startup probe fails, the kubelet will terminate the container, which will then be restarted. The startup probe provided by CloudNativePG is configurable via the parameter .spec.startDelay , which specifies the maximum time, in seconds, allowed for the startup probe to succeed. At a minimum, the probe requires pg_isready to return 0 or 1 . By default, the startDelay is set to 3600 seconds. It is recommended to adjust this setting based on the time PostgreSQL needs to fully initialize in your specific environment. Warning Setting .spec.startDelay too low can cause the liveness probe to activate prematurely, potentially resulting in unnecessary Pod restarts if PostgreSQL hasn\u2019t fully initialized. CloudNativePG configures the startup probe with the following default parameters: failureThreshold: FAILURE_THRESHOLD periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 The failureThreshold value is automatically calculated by dividing startDelay by periodSeconds . You can customize any of the probe settings in the .spec.probes.startup section of your configuration. Warning Be sure that any custom probe settings are tailored to your cluster's operational requirements to avoid unintended disruptions. Info For more details on probe configuration, refer to the probe API documentation . If you manually specify .spec.probes.startup.failureThreshold , it will override the default behavior and disable the automatic use of startDelay . For example, the following configuration explicitly sets custom probe parameters, bypassing startDelay : # ... snip spec: probes: startup: periodSeconds: 3 timeoutSeconds: 3 failureThreshold: 10","title":"Startup Probe"},{"location":"instance_manager/#liveness-probe","text":"The liveness probe begins after the startup probe successfully completes. Its primary role is to ensure the PostgreSQL instance\u2014whether primary or standby\u2014is operating correctly. This is achieved using the pg_isready utility. Both exit codes 0 (indicating the server is accepting connections) and 1 (indicating the server is rejecting connections, such as during startup or a smart shutdown) are treated as valid outcomes. Following Kubernetes standards, if the liveness probe fails, the kubelet will terminate the container, which will then be restarted. The amount of time before a Pod is classified as not alive is configurable via the .spec.livenessProbeTimeout parameter. CloudNativePG configures the liveness probe with the following default parameters: failureThreshold: FAILURE_THRESHOLD periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 The failureThreshold value is automatically calculated by dividing livenessProbeTimeout by periodSeconds . By default, .spec.livenessProbeTimeout is set to 30 seconds. This means the liveness probe will report a failure if it detects three consecutive probe failures, with a 10-second interval between each check. You can customize any of the probe settings in the .spec.probes.liveness section of your configuration. Warning Be sure that any custom probe settings are tailored to your cluster's operational requirements to avoid unintended disruptions. Info For more details on probe configuration, refer to the probe API documentation . If you manually specify .spec.probes.liveness.failureThreshold , it will override the default behavior and disable the automatic use of livenessProbeTimeout . For example, the following configuration explicitly sets custom probe parameters, bypassing livenessProbeTimeout : # ... snip spec: probes: liveness: periodSeconds: 3 timeoutSeconds: 3 failureThreshold: 10","title":"Liveness Probe"},{"location":"instance_manager/#readiness-probe","text":"The readiness probe begins once the startup probe has successfully completed. Its purpose is to check whether the PostgreSQL instance is ready to accept traffic and serve requests. For streaming replicas, it also requires that they have connected to the source at least once. Following Kubernetes standards, if the readiness probe fails, the pod will be marked unready and will not receive traffic from any services. CloudNativePG uses the following default configuration for the readiness probe: failureThreshold: 3 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 If the default settings do not suit your requirements, you can fully customize the readiness probe by specifying parameters in the .spec.probes.readiness stanza. For example: # ... snip spec: probes: readiness: periodSeconds: 3 timeoutSeconds: 3 failureThreshold: 10 Warning Ensure that any custom probe settings are aligned with your cluster\u2019s operational requirements to prevent unintended disruptions. Info For more information on configuring probes, see the probe API .","title":"Readiness Probe"},{"location":"instance_manager/#shutdown-control","text":"When a Pod running Postgres is deleted, either manually or by Kubernetes following a node drain operation, the kubelet will send a termination signal to the instance manager, and the instance manager will take care of shutting down PostgreSQL in an appropriate way. The .spec.smartShutdownTimeout and .spec.stopDelay options, expressed in seconds, control the amount of time given to PostgreSQL to shut down. The values default to 180 and 1800 seconds, respectively. The shutdown procedure is composed of two steps: The instance manager requests a smart shut down, disallowing any new connection to PostgreSQL. This step will last for up to .spec.smartShutdownTimeout seconds. If PostgreSQL is still up, the instance manager requests a fast shut down, terminating any existing connection and exiting promptly. If the instance is archiving and/or streaming WAL files, the process will wait for up to the remaining time set in .spec.stopDelay to complete the operation and then forcibly shut down. Such a timeout needs to be at least 15 seconds. Important In order to avoid any data loss in the Postgres cluster, which impacts the database RPO , don't delete the Pod where the primary instance is running. In this case, perform a switchover to another instance first.","title":"Shutdown control"},{"location":"instance_manager/#shutdown-of-the-primary-during-a-switchover","text":"During a switchover, the shutdown procedure is slightly different from the general case. Indeed, the operator requires the former primary to issue a fast shut down before the selected new primary can be promoted, in order to ensure that all the data are available on the new primary. For this reason, the .spec.switchoverDelay , expressed in seconds, controls the time given to the former primary to shut down gracefully and archive all the WAL files. By default it is set to 3600 (1 hour). Warning The .spec.switchoverDelay option affects the RPO and RTO of your PostgreSQL database. Setting it to a low value, might favor RTO over RPO but lead to data loss at cluster level and/or backup level. On the contrary, setting it to a high value, might remove the risk of data loss while leaving the cluster without an active primary for a longer time during the switchover.","title":"Shutdown of the primary during a switchover"},{"location":"instance_manager/#failover","text":"In case of primary pod failure, the cluster will go into failover mode. Please refer to the \"Failover\" section for details.","title":"Failover"},{"location":"instance_manager/#disk-full-failure","text":"Storage exhaustion is a well known issue for PostgreSQL clusters. The PostgreSQL documentation highlights the possible failure scenarios and the importance of monitoring disk usage to prevent it from becoming full. The same applies to CloudNativePG and Kubernetes as well: the \"Monitoring\" section provides details on checking the disk space used by WAL segments and standard metrics on disk usage exported to Prometheus. Important In a production system, it is critical to monitor the database continuously. Exhausted disk storage can lead to a database server shutdown. Note The detection of exhausted storage relies on a storage class that accurately reports disk size and usage. This may not be the case in simulated Kubernetes environments like Kind or with test storage class implementations such as csi-driver-host-path . If the disk containing the WALs becomes full and no more WAL segments can be stored, PostgreSQL will stop working. CloudNativePG correctly detects this issue by verifying that there is enough space to store the next WAL segment, and avoids triggering a failover, which could complicate recovery. That allows a human administrator to address the root cause. In such a case, if supported by the storage class, the quickest course of action is currently to: Expand the storage size of the full PVC Increase the size in the Cluster resource to the same value Once the issue is resolved and there is sufficient free space for WAL segments, the Pod will restart and the cluster will become healthy. See also the \"Volume expansion\" section of the documentation.","title":"Disk Full Failure"},{"location":"kubectl-plugin/","text":"Kubectl Plugin CloudNativePG provides a plugin for kubectl to manage a cluster in Kubernetes. Install You can install the cnpg plugin using a variety of methods. Note For air-gapped systems, installation via package managers, using previously downloaded files, may be a good option. Via the installation script curl -sSfL \\ https://github.com/cloudnative-pg/cloudnative-pg/raw/main/hack/install-cnpg-plugin.sh | \\ sudo sh -s -- -b /usr/local/bin Using the Debian or RedHat packages In the releases section of the GitHub repository , you can navigate to any release of interest (pick the same or newer release than your CloudNativePG operator), and in it you will find an Assets section. In that section are pre-built packages for a variety of systems. As a result, you can follow standard practices and instructions to install them in your systems. Debian packages For example, let's install the 1.25.3 release of the plugin, for an Intel based 64 bit server. First, we download the right .deb file. wget https://github.com/cloudnative-pg/cloudnative-pg/releases/download/v1.25.3/kubectl-cnpg_1.25.3_linux_x86_64.deb \\ --output-document kube-plugin.deb Then, with superuser privileges, install from the local file using dpkg : $ sudo dpkg -i kube-plugin.deb Selecting previously unselected package cnpg. (Reading database ... 6688 files and directories currently installed.) Preparing to unpack kube-plugin.deb ... Unpacking cnpg (1.25.3) ... Setting up cnpg (1.25.3) ... RPM packages As in the example for .rpm packages, let's install the 1.25.3 release for an Intel 64 bit machine. Note the --output flag to provide a file name. curl -L https://github.com/cloudnative-pg/cloudnative-pg/releases/download/v1.25.3/kubectl-cnpg_1.25.3_linux_x86_64.rpm \\ --output kube-plugin.rpm Then, with superuser privileges, install with yum , and you're ready to use: $ sudo yum --disablerepo=* localinstall kube-plugin.rpm Failed to set locale, defaulting to C.UTF-8 Dependencies resolved. ==================================================================================================== Package Architecture Version Repository Size ==================================================================================================== Installing: cnpg x86_64 1.25.3 @commandline 20 M Transaction Summary ==================================================================================================== Install 1 Package Total size: 20 M Installed size: 78 M Is this ok [y/N]: y Using the Arch Linux User Repository (AUR) Package To install the plugin from the AUR , follow these steps: git clone https://aur.archlinux.org/kubectl-cnpg.git cd kubectl-cnpg makepkg -si Or use your favorite AUR-helper, for example paru : paru -S kubectl-cnpg Using Krew If you already have Krew installed, you can simply run: kubectl krew install cnpg When a new version of the plugin is released, you can update the existing installation with: kubectl krew update kubectl krew upgrade cnpg Using Homebrew Note Please note that the Homebrew community manages the availability of the kubectl-cnpg plugin on Homebrew . If you already have Homebrew installed, you can simply run: brew install kubectl-cnpg When a new version of the plugin is released, you can update the existing installation with: brew update brew upgrade kubectl-cnpg Note Auto-completion for the kubectl plugin is already managed by Homebrew. There's no need to create the kubectl_complete-cnpg script mentioned below. Supported Architectures CloudNativePG Plugin is currently built for the following operating system and architectures: Linux amd64 arm 5/6/7 arm64 s390x ppc64le macOS amd64 arm64 Windows 386 amd64 arm 5/6/7 arm64 Configuring auto-completion To configure auto-completion for the plugin, a helper shell script needs to be installed into your current PATH. Assuming the latter contains /usr/local/bin , this can be done with the following commands: cat > kubectl_complete-cnpg < operator.yaml The flags in the above command have the following meaning: - -n king install the CNPG operator into the king namespace - --version 1.23 install the latest patch version for minor version 1.23 - --replicas 3 install the operator with 3 replicas - --watch-namespace \"albert, bb, freddie\" have the operator watch for changes in the albert , bb and freddie namespaces only Status The status command provides an overview of the current status of your cluster, including: general information : name of the cluster, PostgreSQL's system ID, number of instances, current timeline and position in the WAL backup : point of recoverability, and WAL archiving status as returned by the pg_stat_archiver view from the primary - or designated primary in the case of a replica cluster streaming replication : information taken directly from the pg_stat_replication view on the primary instance instances : information about each Postgres instance, taken directly by each instance manager; in the case of a standby, the Current LSN field corresponds to the latest write-ahead log location that has been replayed during recovery (replay LSN). Important The status information above is taken at different times and at different locations, resulting in slightly inconsistent returned values. For example, the Current Write LSN location in the main header, might be different from the Current LSN field in the instances status as it is taken at two different time intervals. kubectl cnpg status sandbox Cluster Summary Name: default/sandbox System ID: 7423474350493388827 PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:16.4 Primary instance: sandbox-1 Primary start time: 2024-10-08 18:31:57 +0000 UTC (uptime 1m14s) Status: Cluster in healthy state Instances: 3 Ready instances: 3 Size: 126M Current Write LSN: 0/604DE38 (Timeline: 1 - WAL File: 000000010000000000000006) Continuous Backup status Not configured Streaming Replication status Replication Slots Enabled Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority Replication Slot ---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- ---------------- sandbox-2 0/604DE38 0/604DE38 0/604DE38 0/604DE38 00:00:00 00:00:00 00:00:00 streaming async 0 active sandbox-3 0/604DE38 0/604DE38 0/604DE38 0/604DE38 00:00:00 00:00:00 00:00:00 streaming async 0 active Instances status Name Current LSN Replication role Status QoS Manager Version Node ---- ----------- ---------------- ------ --- --------------- ---- sandbox-1 0/604DE38 Primary OK BestEffort 1.25.3 k8s-eu-worker sandbox-2 0/604DE38 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker2 sandbox-3 0/604DE38 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker If you require more detailed status information, use the --verbose option (or -v for short). The level of detail increases each time the flag is repeated: kubectl cnpg status sandbox --verbose Cluster Summary Name: default/sandbox System ID: 7423474350493388827 PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:16.4 Primary instance: sandbox-1 Primary start time: 2024-10-08 18:31:57 +0000 UTC (uptime 2m4s) Status: Cluster in healthy state Instances: 3 Ready instances: 3 Size: 126M Current Write LSN: 0/6053720 (Timeline: 1 - WAL File: 000000010000000000000006) Continuous Backup status Not configured Physical backups No running physical backups found Streaming Replication status Replication Slots Enabled Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority Replication Slot Slot Restart LSN Slot WAL Status Slot Safe WAL Size ---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- ---------------- ---------------- --------------- ------------------ sandbox-2 0/6053720 0/6053720 0/6053720 0/6053720 00:00:00 00:00:00 00:00:00 streaming async 0 active 0/6053720 reserved NULL sandbox-3 0/6053720 0/6053720 0/6053720 0/6053720 00:00:00 00:00:00 00:00:00 streaming async 0 active 0/6053720 reserved NULL Unmanaged Replication Slot Status No unmanaged replication slots found Managed roles status No roles managed Tablespaces status No managed tablespaces Pod Disruption Budgets status Name Role Expected Pods Current Healthy Minimum Desired Healthy Disruptions Allowed ---- ---- ------------- --------------- ----------------------- ------------------- sandbox replica 2 2 1 1 sandbox-primary primary 1 1 1 0 Instances status Name Current LSN Replication role Status QoS Manager Version Node ---- ----------- ---------------- ------ --- --------------- ---- sandbox-1 0/6053720 Primary OK BestEffort 1.25.3 k8s-eu-worker sandbox-2 0/6053720 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker2 sandbox-3 0/6053720 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker With an additional -v (e.g. kubectl cnpg status sandbox -v -v ), you can also view PostgreSQL configuration, HBA settings, and certificates. The command also supports output in yaml and json format. Promote The meaning of this command is to promote a pod in the cluster to primary, so you can start with maintenance work or test a switch-over situation in your cluster: kubectl cnpg promote CLUSTER CLUSTER-INSTANCE Or you can use the instance node number to promote: kubectl cnpg promote CLUSTER INSTANCE Certificates Clusters created using the CloudNativePG operator work with a CA to sign a TLS authentication certificate. To get a certificate, you need to provide a name for the secret to store the credentials, the cluster name, and a user for this certificate: kubectl cnpg certificate cluster-cert --cnpg-cluster CLUSTER --cnpg-user USER After the secret it's created, you can get it using kubectl : kubectl get secret cluster-cert And the content of the same in plain text using the following commands: kubectl get secret cluster-cert -o json | jq -r '.data | map(@base64d) | .[]' Restart The kubectl cnpg restart command can be used in two cases: requesting the operator to orchestrate a rollout restart for a certain cluster. This is useful to apply configuration changes to cluster dependent objects, such as ConfigMaps containing custom monitoring queries. request a single instance restart, either in-place if the instance is the cluster's primary or deleting and recreating the pod if it is a replica. # this command will restart a whole cluster in a rollout fashion kubectl cnpg restart CLUSTER # this command will restart a single instance, according to the policy above kubectl cnpg restart CLUSTER INSTANCE If the in-place restart is requested but the change cannot be applied without a switchover, the switchover will take precedence over the in-place restart. A common case for this will be a minor upgrade of PostgreSQL image. Note If you want ConfigMaps and Secrets to be automatically reloaded by instances, you can add a label with key cnpg.io/reload to it. Reload The kubectl cnpg reload command requests the operator to trigger a reconciliation loop for a certain cluster. This is useful to apply configuration changes to cluster dependent objects, such as ConfigMaps containing custom monitoring queries. The following command will reload all configurations for a given cluster: kubectl cnpg reload CLUSTER Maintenance The kubectl cnpg maintenance command helps to modify one or more clusters across namespaces and set the maintenance window values, it will change the following fields: .spec.nodeMaintenanceWindow.inProgress .spec.nodeMaintenanceWindow.reusePVC Accepts as argument set and unset using this to set the inProgress to true in case set and to false in case of unset . By default, reusePVC is always set to false unless the --reusePVC flag is passed. The plugin will ask for a confirmation with a list of the cluster to modify and their new values, if this is accepted this action will be applied to all the cluster in the list. If you want to set in maintenance all the PostgreSQL in your Kubernetes cluster, just need to write the following command: kubectl cnpg maintenance set --all-namespaces And you'll have the list of all the cluster to update The following are the new values for the clusters Namespace Cluster Name Maintenance reusePVC --------- ------------ ----------- -------- default cluster-example true false default pg-backup true false test cluster-example true false Do you want to proceed? [y/n]: y Report The kubectl cnpg report command bundles various pieces of information into a ZIP file. It aims to provide the needed context to debug problems with clusters in production. It has two sub-commands: operator and cluster . report Operator The operator sub-command requests the operator to provide information regarding the operator deployment, configuration and events. Important All confidential information in Secrets and ConfigMaps is REDACTED. The Data map will show the keys but the values will be empty. The flag -S / --stopRedaction will defeat the redaction and show the values. Use only at your own risk, this will share private data. Note By default, operator logs are not collected, but you can enable operator log collection with the --logs flag deployment information : the operator Deployment and operator Pod configuration : the Secrets and ConfigMaps in the operator namespace events : the Events in the operator namespace webhook configuration : the mutating and validating webhook configurations webhook service : the webhook service logs : logs for the operator Pod (optional, off by default) in JSON-lines format The command will generate a ZIP file containing various manifest in YAML format (by default, but settable to JSON with the -o flag). Use the -f flag to name a result file explicitly. If the -f flag is not used, a default time-stamped filename is created for the zip file. Note The report plugin obeys kubectl conventions, and will look for objects constrained by namespace. The CNPG Operator will generally not be installed in the same namespace as the clusters. E.g. the default installation namespace is cnpg-system kubectl cnpg report operator -n cnpg-system results in Successfully written report to \"report_operator_.zip\" (format: \"yaml\") With the -f flag set: kubectl cnpg report operator -n cnpg-system -f reportRedacted.zip Unzipping the file will produce a time-stamped top-level folder to keep the directory tidy: unzip reportRedacted.zip will result in: Archive: reportRedacted.zip creating: report_operator_/ creating: report_operator_/manifests/ inflating: report_operator_/manifests/deployment.yaml inflating: report_operator_/manifests/operator-pod.yaml inflating: report_operator_/manifests/events.yaml inflating: report_operator_/manifests/validating-webhook-configuration.yaml inflating: report_operator_/manifests/mutating-webhook-configuration.yaml inflating: report_operator_/manifests/webhook-service.yaml inflating: report_operator_/manifests/cnpg-ca-secret(secret).yaml inflating: report_operator_/manifests/cnpg-webhook-cert(secret).yaml If you activated the --logs option, you'd see an extra subdirectory: Archive: report_operator_.zip creating: report_operator_/operator-logs/ inflating: report_operator_/operator-logs/cnpg-controller-manager-66fb98dbc5-pxkmh-logs.jsonl Note The plugin will try to get the PREVIOUS operator's logs, which is helpful when investigating restarted operators. In all cases, it will also try to get the CURRENT operator logs. If current and previous logs are available, it will show them both. ====== Beginning of Previous Log ===== 2023-03-28T12:56:41.251711811Z {\"level\":\"info\",\"ts\":\"2023-03-28T12:56:41Z\",\"logger\":\"setup\",\"msg\":\"Starting CloudNativePG Operator\",\"version\":\"1.25.3\",\"build\":{\"Version\":\"1.25.3+dev107\",\"Commit\":\"cc9bab17\",\"Date\":\"2023-03-28\"}} 2023-03-28T12:56:41.251851909Z {\"level\":\"info\",\"ts\":\"2023-03-28T12:56:41Z\",\"logger\":\"setup\",\"msg\":\"Starting pprof HTTP server\",\"addr\":\"0.0.0.0:6060\"} ====== End of Previous Log ===== 2023-03-28T12:57:09.854306024Z {\"level\":\"info\",\"ts\":\"2023-03-28T12:57:09Z\",\"logger\":\"setup\",\"msg\":\"Starting CloudNativePG Operator\",\"version\":\"1.25.3\",\"build\":{\"Version\":\"1.25.3+dev107\",\"Commit\":\"cc9bab17\",\"Date\":\"2023-03-28\"}} 2023-03-28T12:57:09.854363943Z {\"level\":\"info\",\"ts\":\"2023-03-28T12:57:09Z\",\"logger\":\"setup\",\"msg\":\"Starting pprof HTTP server\",\"addr\":\"0.0.0.0:6060\"} If the operator hasn't been restarted, you'll still see the ====== Begin \u2026 and ====== End \u2026 guards, with no content inside. You can verify that the confidential information is REDACTED by default: cd report_operator_/manifests/ head cnpg-ca-secret\\(secret\\).yaml data: ca.crt: \"\" ca.key: \"\" metadata: creationTimestamp: \"2022-03-22T10:42:28Z\" managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: With the -S ( --stopRedaction ) option activated, secrets are shown: kubectl cnpg report operator -n cnpg-system -f reportNonRedacted.zip -S You'll get a reminder that you're about to view confidential information: WARNING: secret Redaction is OFF. Use it with caution Successfully written report to \"reportNonRedacted.zip\" (format: \"yaml\") unzip reportNonRedacted.zip head cnpg-ca-secret\\(secret\\).yaml data: ca.crt: LS0tLS1CRUdJTiBD\u2026 ca.key: LS0tLS1CRUdJTiBF\u2026 metadata: creationTimestamp: \"2022-03-22T10:42:28Z\" managedFields: - apiVersion: v1 fieldsType: FieldsV1 report Cluster The cluster sub-command gathers the following: cluster resources : the cluster information, same as kubectl get cluster -o yaml cluster pods : pods in the cluster namespace matching the cluster name cluster jobs : jobs, if any, in the cluster namespace matching the cluster name events : events in the cluster namespace pod logs : logs for the cluster Pods (optional, off by default) in JSON-lines format job logs : logs for the Pods created by jobs (optional, off by default) in JSON-lines format The cluster sub-command accepts the -f and -o flags, as the operator does. If the -f flag is not used, a default timestamped report name will be used. Note that the cluster information does not contain configuration Secrets / ConfigMaps, so the -S is disabled. Note By default, cluster logs are not collected, but you can enable cluster log collection with the --logs flag Usage: kubectl cnpg report cluster CLUSTER [flags] Note that, unlike the operator sub-command, for the cluster sub-command you need to provide the cluster name, and very likely the namespace, unless the cluster is in the default one. kubectl cnpg report cluster CLUSTER -f report.zip [-n NAMESPACE] and then: unzip report.zip Archive: report.zip creating: report_cluster_example_/ creating: report_cluster_example_/manifests/ inflating: report_cluster_example_/manifests/cluster.yaml inflating: report_cluster_example_/manifests/cluster-pods.yaml inflating: report_cluster_example_/manifests/cluster-jobs.yaml inflating: report_cluster_example_/manifests/events.yaml Remember that you can use the --logs flag to add the pod and job logs to the ZIP. kubectl cnpg report cluster CLUSTER [-n NAMESPACE] --logs will result in: Successfully written report to \"report_cluster_example_.zip\" (format: \"yaml\") unzip report_cluster_.zip Archive: report_cluster_example_.zip creating: report_cluster_example_/ creating: report_cluster_example_/manifests/ inflating: report_cluster_example_/manifests/cluster.yaml inflating: report_cluster_example_/manifests/cluster-pods.yaml inflating: report_cluster_example_/manifests/cluster-jobs.yaml inflating: report_cluster_example_/manifests/events.yaml creating: report_cluster_example_/logs/ inflating: report_cluster_example_/logs/cluster-example-full-1.jsonl creating: report_cluster_example_/job-logs/ inflating: report_cluster_example_/job-logs/cluster-example-full-1-initdb-qnnvw.jsonl inflating: report_cluster_example_/job-logs/cluster-example-full-2-join-tvj8r.jsonl Logs The kubectl cnpg logs command allows to follow the logs of a collection of pods related to CloudNativePG in a single go. It has at the moment one available sub-command: cluster . Cluster logs The cluster sub-command gathers all the pod logs for a cluster in a single stream or file. This means that you can get all the pod logs in a single terminal window, with a single invocation of the command. As in all the cnpg plugin sub-commands, you can get instructions and help with the -h flag: kubectl cnpg logs cluster -h The logs command will display logs in JSON-lines format, unless the --timestamps flag is used, in which case, a human-readable timestamp will be prepended to each line. In this case, lines will no longer be valid JSON, and tools such as jq may not work as desired. If the logs cluster sub-command is given the -f flag (aka --follow ), it will follow the cluster pod logs, and will also watch for any new pods created in the cluster after the command has been invoked. Any new pods found, including pods that have been restarted or re-created, will also have their pods followed. The logs will be displayed in the terminal's standard-out. This command will only exit when the cluster has no more pods left, or when it is interrupted by the user. If logs is called without the -f option, it will read the logs from all cluster pods until the time of invocation and display them in the terminal's standard-out, then exit. The -o or --output flag can be provided, to specify the name of the file where the logs should be saved, instead of displaying over standard-out. The --tail flag can be used to specify how many log lines will be retrieved from each pod in the cluster. By default, the logs cluster sub-command will display all the logs from each pod in the cluster. If combined with the \"follow\" flag -f , the number of logs specified by --tail will be retrieved until the current time, and from then the new logs will be followed. NOTE: unlike other cnpg plugin commands, the -f is used to denote \"follow\" rather than specify a file. This keeps with the convention of kubectl logs , which takes -f to mean the logs should be followed. Usage: kubectl cnpg logs cluster CLUSTER [flags] Using the -f option to follow: kubectl cnpg report cluster CLUSTER -f Using --tail option to display 3 lines from each pod and the -f option to follow: kubectl cnpg report cluster CLUSTER -f --tail 3 {\"level\":\"info\",\"ts\":\"2023-06-30T13:37:33Z\",\"logger\":\"postgres\",\"msg\":\"2023-06-30 13:37:33.142 UTC [26] LOG: ending log output to stderr\",\"source\":\"/controller/log/postgres\",\"logging_pod\":\"cluster-example-3\"} {\"level\":\"info\",\"ts\":\"2023-06-30T13:37:33Z\",\"logger\":\"postgres\",\"msg\":\"2023-06-30 13:37:33.142 UTC [26] HINT: Future log output will go to log destination \\\"csvlog\\\".\",\"source\":\"/controller/log/postgres\",\"logging_pod\":\"cluster-example-3\"} \u2026 \u2026 With the -o option omitted, and with --output specified: $ kubectl cnpg logs cluster CLUSTER --output my-cluster.log Successfully written logs to \"my-cluster.log\" Pretty The pretty sub-command reads a log stream from standard input, formats it into a human-readable output, and attempts to sort the entries by timestamp. It can be used in combination with kubectl cnpg logs cluster , as shown in the following example: $ kubectl cnpg logs cluster cluster-example | kubectl cnpg logs pretty 2024-10-15T17:35:00.336 INFO cluster-example-1 instance-manager Starting CloudNativePG Instance Manager 2024-10-15T17:35:00.336 INFO cluster-example-1 instance-manager Checking for free disk space for WALs before starting PostgreSQL 2024-10-15T17:35:00.347 INFO cluster-example-1 instance-manager starting tablespace manager 2024-10-15T17:35:00.347 INFO cluster-example-1 instance-manager starting external server manager [...] Alternatively, it can be used in combination with other commands that produce CNPG logs in JSON format, such as stern , or kubectl logs , as in the following example: $ kubectl logs cluster-example-1 | kubectl cnpg logs pretty 2024-10-15T17:35:00.336 INFO cluster-example-1 instance-manager Starting CloudNativePG Instance Manager 2024-10-15T17:35:00.336 INFO cluster-example-1 instance-manager Checking for free disk space for WALs before starting PostgreSQL 2024-10-15T17:35:00.347 INFO cluster-example-1 instance-manager starting tablespace manager 2024-10-15T17:35:00.347 INFO cluster-example-1 instance-manager starting external server manager [...] The pretty sub-command also supports advanced log filtering, allowing users to display logs for specific pods or loggers, or to filter logs by severity level. Here's an example: $ kubectl cnpg logs cluster cluster-example | kubectl cnpg logs pretty --pods cluster-example-1 --loggers postgres --log-level info 2024-10-15T17:35:00.509 INFO cluster-example-1 postgres 2024-10-15 17:35:00.509 UTC [29] LOG: redirecting log output to logging collector process 2024-10-15T17:35:00.509 INFO cluster-example-1 postgres 2024-10-15 17:35:00.509 UTC [29] HINT: Future log output will appear in directory \"/controller/log\"... 2024-10-15T17:35:00.510 INFO cluster-example-1 postgres 2024-10-15 17:35:00.509 UTC [29] LOG: ending log output to stderr 2024-10-15T17:35:00.510 INFO cluster-example-1 postgres ending log output to stderr [...] The pretty sub-command will try to sort the log stream, to make logs easier to reason about. In order to achieve this, it gathers the logs into groups, and within groups it sorts by timestamp. This is the only way to sort interactively, as pretty may be piped from a command in \"follow\" mode. The sub-command will add a group separator line, --- , at the end of each sorted group. The size of the grouping can be configured via the --sorting-group-size flag (default: 1000), as illustrated in the following example: $ kubectl cnpg logs cluster cluster-example | kubectl cnpg logs pretty --sorting-group-size=3 2024-10-15T17:35:20.426 INFO cluster-example-2 instance-manager Starting CloudNativePG Instance Manager 2024-10-15T17:35:20.426 INFO cluster-example-2 instance-manager Checking for free disk space for WALs before starting PostgreSQL 2024-10-15T17:35:20.438 INFO cluster-example-2 instance-manager starting tablespace manager --- 2024-10-15T17:35:20.438 INFO cluster-example-2 instance-manager starting external server manager 2024-10-15T17:35:20.438 INFO cluster-example-2 instance-manager starting controller-runtime manager 2024-10-15T17:35:20.439 INFO cluster-example-2 instance-manager Starting EventSource --- [...] To explore all available options, use the -h flag for detailed explanations of the supported flags and their usage. Info You can also increase the verbosity of the log by adding more -v options. Destroy The kubectl cnpg destroy command helps remove an instance and all the associated PVCs from a Kubernetes cluster. The optional --keep-pvc flag, if specified, allows you to keep the PVCs, while removing all metadata.ownerReferences that were set by the instance. Additionally, the cnpg.io/pvcStatus label on the PVCs will change from ready to detached to signify that they are no longer in use. Running again the command without the --keep-pvc flag will remove the detached PVCs. Usage: kubectl cnpg destroy CLUSTER INSTANCE The following example removes the cluster-example-2 pod and the associated PVCs: kubectl cnpg destroy cluster-example 2 Cluster hibernation Sometimes you may want to suspend the execution of a CloudNativePG Cluster while retaining its data, then resume its activity at a later time. We've called this feature cluster hibernation . Hibernation is only available via the kubectl cnpg hibernate [on|off] commands. Hibernating a CloudNativePG cluster means destroying all the resources generated by the cluster, except the PVCs that belong to the PostgreSQL primary instance. You can hibernate a cluster with: kubectl cnpg hibernate on CLUSTER This will: shutdown every PostgreSQL instance detach the PVCs containing the data of the primary instance, and annotate them with the latest database status and the latest cluster configuration delete the Cluster resource, including every generated resource - except the aforementioned PVCs When hibernated, a CloudNativePG cluster is represented by just a group of PVCs, in which the one containing the PGDATA is annotated with the latest available status, including content from pg_controldata . Warning A cluster having fenced instances cannot be hibernated, as fencing is part of the hibernation procedure too. In case of error the operator will not be able to revert the procedure. You can still force the operation with: kubectl cnpg hibernate on CLUSTER --force A hibernated cluster can be resumed with: kubectl cnpg hibernate off CLUSTER Once the cluster has been hibernated, it's possible to show the last configuration and the status that PostgreSQL had after it was shut down. That can be done with: kubectl cnpg hibernate status CLUSTER Benchmarking the database with pgbench Pgbench can be run against an existing PostgreSQL cluster with following command: kubectl cnpg pgbench CLUSTER -- --time 30 --client 1 --jobs 1 Refer to the Benchmarking pgbench section for more details. Benchmarking the storage with fio fio can be run on an existing storage class with following command: kubectl cnpg fio FIO_JOB_NAME [-n NAMESPACE] Refer to the Benchmarking fio section for more details. Requesting a new physical backup The kubectl cnpg backup command requests a new physical backup for an existing Postgres cluster by creating a new Backup resource. The following example requests an on-demand backup for a given cluster: kubectl cnpg backup CLUSTER or, if using volume snapshots: kubectl cnpg backup CLUSTER -m volumeSnapshot The created backup will be named after the request time: $ kubectl cnpg backup cluster-example backup/cluster-example-20230121002300 created By default, a newly created backup will use the backup target policy defined in the cluster to choose which instance to run on. However, you can override this policy with the --backup-target option. In the case of volume snapshot backups, you can also use the --online option to request an online/hot backup or an offline/cold one: additionally, you can also tune online backups by explicitly setting the --immediate-checkpoint and --wait-for-archive options. The \"Backup\" section contains more information about the configuration settings. Launching psql The kubectl cnpg psql CLUSTER command starts a new PostgreSQL interactive front-end process (psql) connected to an existing Postgres cluster, as if you were running it from the actual pod. This means that you will be using the postgres user. Important As you will be connecting as postgres user, in production environments this method should be used with extreme care, by authorized personnel only. $ kubectl cnpg psql cluster-example psql (17.5 (Debian 17.5-1.pgdg110+1)) Type \"help\" for help. postgres=# By default, the command will connect to the primary instance. The user can select to work against a replica by using the --replica option: $ kubectl cnpg psql --replica cluster-example psql (17.5 (Debian 17.5-1.pgdg110+1)) Type \"help\" for help. postgres=# select pg_is_in_recovery(); pg_is_in_recovery ------------------- t (1 row) postgres=# \\q This command will start kubectl exec , and the kubectl executable must be reachable in your PATH variable to correctly work. Snapshotting a Postgres cluster Warning The kubectl cnpg snapshot command has been removed. Please use the backup command to request backups using volume snapshots. Using pgAdmin4 for evaluation/demonstration purposes only pgAdmin stands as the most popular and feature-rich open-source administration and development platform for PostgreSQL. For more information on the project, please refer to the official documentation . Given that the pgAdmin Development Team maintains official Docker container images, you can install pgAdmin in your environment as a standard Kubernetes deployment. Important Deployment of pgAdmin in Kubernetes production environments is beyond the scope of this document and, more broadly, of the CloudNativePG project. However, for the purposes of demonstration and evaluation , CloudNativePG offers a suitable solution. The cnpg plugin implements the pgadmin4 command, providing a straightforward method to connect to a given database Cluster and navigate its content in a local environment such as kind . For example, you can install a demo deployment of pgAdmin4 for the cluster-example cluster as follows: kubectl cnpg pgadmin4 cluster-example This command will produce: ConfigMap/cluster-example-pgadmin4 created Deployment/cluster-example-pgadmin4 created Service/cluster-example-pgadmin4 created Secret/cluster-example-pgadmin4 created [...] After deploying pgAdmin, forward the port using kubectl and connect through your browser by following the on-screen instructions. As usual, you can use the --dry-run option to generate the YAML file: kubectl cnpg pgadmin4 --dry-run cluster-example pgAdmin4 can be installed in either desktop or server mode, with the default being server. In server mode, authentication is required using a randomly generated password, and users must manually specify the database to connect to. On the other hand, desktop mode initiates a pgAdmin web interface without requiring authentication. It automatically connects to the app database as the app user, making it ideal for quick demos, such as on a local deployment using kind : kubectl cnpg pgadmin4 --mode desktop cluster-example After concluding your demo, ensure the termination of the pgAdmin deployment by executing: kubectl cnpg pgadmin4 --dry-run cluster-example | kubectl delete -f - Warning Never deploy pgAdmin in production using the plugin. Logical Replication Publications The cnpg publication command group is designed to streamline the creation and removal of PostgreSQL logical replication publications . Be aware that these commands are primarily intended for assisting in the creation of logical replication publications, particularly on remote PostgreSQL databases. Warning It is crucial to have a solid understanding of both the capabilities and limitations of PostgreSQL's native logical replication system before using these commands. In particular, be mindful of the logical replication restrictions . Creating a new publication To create a logical replication publication, use the cnpg publication create command. The basic structure of this command is as follows: kubectl cnpg publication create \\ --publication PUBLICATION_NAME \\ [--external-cluster EXTERNAL_CLUSTER] LOCAL_CLUSTER [options] There are two primary use cases: With --external-cluster : Use this option to create a publication on an external cluster (i.e. defined in the externalClusters stanza). The commands will be issued from the LOCAL_CLUSTER , but the publication will be for the data in EXTERNAL_CLUSTER . Without --external-cluster : Use this option to create a publication in the LOCAL_CLUSTER PostgreSQL Cluster (by default, the app database). Warning When connecting to an external cluster, ensure that the specified user has sufficient permissions to execute the CREATE PUBLICATION command. You have several options, similar to the CREATE PUBLICATION command, to define the group of tables to replicate. Notable options include: If you specify the --all-tables option, you create a publication FOR ALL TABLES . Alternatively, you can specify multiple occurrences of: --table : Add a specific table (with an expression) to the publication. --schema : Include all tables in the specified database schema (available from PostgreSQL 15). The --dry-run option enables you to preview the SQL commands that the plugin will execute. For additional information and detailed instructions, type the following command: kubectl cnpg publication create --help Example Given a source-cluster and a destination-cluster , we would like to create a publication for the data on source-cluster . The destination-cluster has an entry in the externalClusters stanza pointing to source-cluster . We can run: kubectl cnpg publication create destination-cluster \\ --external-cluster=source-cluster --all-tables which will create a publication for all tables on source-cluster , running the SQL commands on the destination-cluster . Or instead, we can run: kubectl cnpg publication create source-cluster \\ --publication=app --all-tables which will create a publication named app for all the tables in the source-cluster , running the SQL commands on the source cluster. Info There are two sample files that have been provided for illustration and inspiration: logical-source and logical-destination . Dropping a publication The cnpg publication drop command seamlessly complements the create command by offering similar key options, including the publication name, cluster name, and an optional external cluster. You can drop a PUBLICATION with the following command structure: kubectl cnpg publication drop \\ --publication PUBLICATION_NAME \\ [--external-cluster EXTERNAL_CLUSTER] LOCAL_CLUSTER [options] To access further details and precise instructions, use the following command: kubectl cnpg publication drop --help Logical Replication Subscriptions The cnpg subscription command group is a dedicated set of commands designed to simplify the creation and removal of PostgreSQL logical replication subscriptions . These commands are specifically crafted to aid in the establishment of logical replication subscriptions, especially when dealing with remote PostgreSQL databases. Warning Before using these commands, it is essential to have a comprehensive understanding of both the capabilities and limitations of PostgreSQL's native logical replication system. In particular, be mindful of the logical replication restrictions . In addition to subscription management, we provide a helpful command for synchronizing all sequences from the source cluster. While its applicability may vary, this command can be particularly useful in scenarios involving major upgrades or data import from remote servers. Creating a new subscription To create a logical replication subscription, use the cnpg subscription create command. The basic structure of this command is as follows: kubectl cnpg subscription create \\ --subscription SUBSCRIPTION_NAME \\ --publication PUBLICATION_NAME \\ --external-cluster EXTERNAL_CLUSTER \\ LOCAL_CLUSTER [options] This command configures a subscription directed towards the specified publication in the designated external cluster, as defined in the externalClusters stanza of the LOCAL_CLUSTER . For additional information and detailed instructions, type the following command: kubectl cnpg subscription create --help Example As in the section on publications, we have a source-cluster and a destination-cluster , and we have already created a publication called app . The following command: kubectl cnpg subscription create destination-cluster \\ --external-cluster=source-cluster \\ --publication=app --subscription=app will create a subscription for app on the destination cluster. Warning Prioritize testing subscriptions in a non-production environment to ensure their effectiveness and identify any potential issues before implementing them in a production setting. Info There are two sample files that have been provided for illustration and inspiration: logical-source and logical-destination . Dropping a subscription The cnpg subscription drop command seamlessly complements the create command. You can drop a SUBSCRIPTION with the following command structure: kubectl cnpg subcription drop \\ --subscription SUBSCRIPTION_NAME \\ LOCAL_CLUSTER [options] To access further details and precise instructions, use the following command: kubectl cnpg subscription drop --help Synchronizing sequences One notable constraint of PostgreSQL logical replication, implemented through publications and subscriptions, is the lack of sequence synchronization. This becomes particularly relevant when utilizing logical replication for live database migration, especially to a higher version of PostgreSQL. A crucial step in this process involves updating sequences before transitioning applications to the new database ( cutover ). To address this limitation, the cnpg subscription sync-sequences command offers a solution. This command establishes a connection with the source database, retrieves all relevant sequences, and subsequently updates local sequences with matching identities (based on database schema and sequence name). You can use the command as shown below: kubectl cnpg subscription sync-sequences \\ --subscription SUBSCRIPTION_NAME \\ LOCAL_CLUSTER For comprehensive details and specific instructions, utilize the following command: kubectl cnpg subscription sync-sequences --help Example As in the previous sections for publication and subscription, we have a source-cluster and a destination-cluster . The publication and the subscription, both called app , are already present. The following command will synchronize the sequences involved in the app subscription, from the source cluster into the destination cluster. kubectl cnpg subscription sync-sequences destination-cluster \\ --subscription=app Warning Prioritize testing subscriptions in a non-production environment to guarantee their effectiveness and detect any potential issues before deploying them in a production setting. Integration with K9s The cnpg plugin can be easily integrated in K9s , a popular terminal-based UI to interact with Kubernetes clusters. See k9s/plugins.yml for details. Permissions required by the plugin The plugin requires a set of Kubernetes permissions that depends on the command to execute. These permissions may affect resources and sub-resources like Pods, PDBs, PVCs, and enable actions like get , delete , patch . The following table contains the full details: Command Resource Permissions backup clusters: get backups: create certificate clusters: get secrets: get,create destroy pods: get,delete jobs: delete,list PVCs: list,delete,update fencing clusters: get,patch pods: get fio PVCs: create configmaps: create deployment: create hibernate clusters: get,patch,delete pods: list,get,delete pods/exec: create jobs: list PVCs: get,list,update,patch,delete install none logs clusters: get pods: list pods/log: get maintenance clusters: get,patch,list pgadmin4 clusters: get configmaps: create deployments: create services: create secrets: create pgbench clusters: get jobs: create promote clusters: get clusters/status: patch pods: get psql pods: get,list pods/exec: create publication clusters: get pods: get,list pods/exec: create reload clusters: get,patch report cluster clusters: get pods: list pods/log: get jobs: list events: list PVCs: list report operator configmaps: get deployments: get events: list pods: list pods/log: get secrets: get services: get mutatingwebhookconfigurations: list 1 validatingwebhookconfigurations: list 1 If OLM is present on the K8s cluster, also: clusterserviceversions: list installplans: list subscriptions: list restart clusters: get,patch pods: get,delete status clusters: get pods: list pods/exec: create pods/proxy: create PDBs: list subscription clusters: get pods: get,list pods/exec: create version none The permissions are cluster scope ClusterRole resources. \u21a9 \u21a9 Additionally, assigning the list permission on the clusters will enable autocompletion for multiple commands. Role examples It is possible to create roles with restricted permissions. The following example creates a role that only has access to the cluster logs: --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: cnpg-log rules: - verbs: - get apiGroups: - postgresql.cnpg.io resources: - clusters - verbs: - list apiGroups: - '' resources: - pods - verbs: - get apiGroups: - '' resources: - pods/log The next example shows a role with the minimal permissions required to get the cluster status using the plugin's status command: apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: cnpg-status rules: - verbs: - get apiGroups: - postgresql.cnpg.io resources: - clusters - verbs: - list apiGroups: - '' resources: - pods - verbs: - create apiGroups: - '' resources: - pods/exec - verbs: - create apiGroups: - '' resources: - pods/proxy - verbs: - list apiGroups: - policy resources: - poddisruptionbudgets Important Keeping the verbs restricted per resources and per apiGroups helps to prevent inadvertently granting more than intended permissions.","title":"Kubectl Plugin"},{"location":"kubectl-plugin/#kubectl-plugin","text":"CloudNativePG provides a plugin for kubectl to manage a cluster in Kubernetes.","title":"Kubectl Plugin"},{"location":"kubectl-plugin/#install","text":"You can install the cnpg plugin using a variety of methods. Note For air-gapped systems, installation via package managers, using previously downloaded files, may be a good option.","title":"Install"},{"location":"kubectl-plugin/#via-the-installation-script","text":"curl -sSfL \\ https://github.com/cloudnative-pg/cloudnative-pg/raw/main/hack/install-cnpg-plugin.sh | \\ sudo sh -s -- -b /usr/local/bin","title":"Via the installation script"},{"location":"kubectl-plugin/#using-the-debian-or-redhat-packages","text":"In the releases section of the GitHub repository , you can navigate to any release of interest (pick the same or newer release than your CloudNativePG operator), and in it you will find an Assets section. In that section are pre-built packages for a variety of systems. As a result, you can follow standard practices and instructions to install them in your systems.","title":"Using the Debian or RedHat packages"},{"location":"kubectl-plugin/#debian-packages","text":"For example, let's install the 1.25.3 release of the plugin, for an Intel based 64 bit server. First, we download the right .deb file. wget https://github.com/cloudnative-pg/cloudnative-pg/releases/download/v1.25.3/kubectl-cnpg_1.25.3_linux_x86_64.deb \\ --output-document kube-plugin.deb Then, with superuser privileges, install from the local file using dpkg : $ sudo dpkg -i kube-plugin.deb Selecting previously unselected package cnpg. (Reading database ... 6688 files and directories currently installed.) Preparing to unpack kube-plugin.deb ... Unpacking cnpg (1.25.3) ... Setting up cnpg (1.25.3) ...","title":"Debian packages"},{"location":"kubectl-plugin/#rpm-packages","text":"As in the example for .rpm packages, let's install the 1.25.3 release for an Intel 64 bit machine. Note the --output flag to provide a file name. curl -L https://github.com/cloudnative-pg/cloudnative-pg/releases/download/v1.25.3/kubectl-cnpg_1.25.3_linux_x86_64.rpm \\ --output kube-plugin.rpm Then, with superuser privileges, install with yum , and you're ready to use: $ sudo yum --disablerepo=* localinstall kube-plugin.rpm Failed to set locale, defaulting to C.UTF-8 Dependencies resolved. ==================================================================================================== Package Architecture Version Repository Size ==================================================================================================== Installing: cnpg x86_64 1.25.3 @commandline 20 M Transaction Summary ==================================================================================================== Install 1 Package Total size: 20 M Installed size: 78 M Is this ok [y/N]: y","title":"RPM packages"},{"location":"kubectl-plugin/#using-the-arch-linux-user-repository-aur-package","text":"To install the plugin from the AUR , follow these steps: git clone https://aur.archlinux.org/kubectl-cnpg.git cd kubectl-cnpg makepkg -si Or use your favorite AUR-helper, for example paru : paru -S kubectl-cnpg","title":"Using the Arch Linux User Repository (AUR) Package"},{"location":"kubectl-plugin/#using-krew","text":"If you already have Krew installed, you can simply run: kubectl krew install cnpg When a new version of the plugin is released, you can update the existing installation with: kubectl krew update kubectl krew upgrade cnpg","title":"Using Krew"},{"location":"kubectl-plugin/#using-homebrew","text":"Note Please note that the Homebrew community manages the availability of the kubectl-cnpg plugin on Homebrew . If you already have Homebrew installed, you can simply run: brew install kubectl-cnpg When a new version of the plugin is released, you can update the existing installation with: brew update brew upgrade kubectl-cnpg Note Auto-completion for the kubectl plugin is already managed by Homebrew. There's no need to create the kubectl_complete-cnpg script mentioned below.","title":"Using Homebrew"},{"location":"kubectl-plugin/#supported-architectures","text":"CloudNativePG Plugin is currently built for the following operating system and architectures: Linux amd64 arm 5/6/7 arm64 s390x ppc64le macOS amd64 arm64 Windows 386 amd64 arm 5/6/7 arm64","title":"Supported Architectures"},{"location":"kubectl-plugin/#configuring-auto-completion","text":"To configure auto-completion for the plugin, a helper shell script needs to be installed into your current PATH. Assuming the latter contains /usr/local/bin , this can be done with the following commands: cat > kubectl_complete-cnpg < operator.yaml The flags in the above command have the following meaning: - -n king install the CNPG operator into the king namespace - --version 1.23 install the latest patch version for minor version 1.23 - --replicas 3 install the operator with 3 replicas - --watch-namespace \"albert, bb, freddie\" have the operator watch for changes in the albert , bb and freddie namespaces only","title":"Generation of installation manifests"},{"location":"kubectl-plugin/#status","text":"The status command provides an overview of the current status of your cluster, including: general information : name of the cluster, PostgreSQL's system ID, number of instances, current timeline and position in the WAL backup : point of recoverability, and WAL archiving status as returned by the pg_stat_archiver view from the primary - or designated primary in the case of a replica cluster streaming replication : information taken directly from the pg_stat_replication view on the primary instance instances : information about each Postgres instance, taken directly by each instance manager; in the case of a standby, the Current LSN field corresponds to the latest write-ahead log location that has been replayed during recovery (replay LSN). Important The status information above is taken at different times and at different locations, resulting in slightly inconsistent returned values. For example, the Current Write LSN location in the main header, might be different from the Current LSN field in the instances status as it is taken at two different time intervals. kubectl cnpg status sandbox Cluster Summary Name: default/sandbox System ID: 7423474350493388827 PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:16.4 Primary instance: sandbox-1 Primary start time: 2024-10-08 18:31:57 +0000 UTC (uptime 1m14s) Status: Cluster in healthy state Instances: 3 Ready instances: 3 Size: 126M Current Write LSN: 0/604DE38 (Timeline: 1 - WAL File: 000000010000000000000006) Continuous Backup status Not configured Streaming Replication status Replication Slots Enabled Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority Replication Slot ---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- ---------------- sandbox-2 0/604DE38 0/604DE38 0/604DE38 0/604DE38 00:00:00 00:00:00 00:00:00 streaming async 0 active sandbox-3 0/604DE38 0/604DE38 0/604DE38 0/604DE38 00:00:00 00:00:00 00:00:00 streaming async 0 active Instances status Name Current LSN Replication role Status QoS Manager Version Node ---- ----------- ---------------- ------ --- --------------- ---- sandbox-1 0/604DE38 Primary OK BestEffort 1.25.3 k8s-eu-worker sandbox-2 0/604DE38 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker2 sandbox-3 0/604DE38 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker If you require more detailed status information, use the --verbose option (or -v for short). The level of detail increases each time the flag is repeated: kubectl cnpg status sandbox --verbose Cluster Summary Name: default/sandbox System ID: 7423474350493388827 PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:16.4 Primary instance: sandbox-1 Primary start time: 2024-10-08 18:31:57 +0000 UTC (uptime 2m4s) Status: Cluster in healthy state Instances: 3 Ready instances: 3 Size: 126M Current Write LSN: 0/6053720 (Timeline: 1 - WAL File: 000000010000000000000006) Continuous Backup status Not configured Physical backups No running physical backups found Streaming Replication status Replication Slots Enabled Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority Replication Slot Slot Restart LSN Slot WAL Status Slot Safe WAL Size ---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- ---------------- ---------------- --------------- ------------------ sandbox-2 0/6053720 0/6053720 0/6053720 0/6053720 00:00:00 00:00:00 00:00:00 streaming async 0 active 0/6053720 reserved NULL sandbox-3 0/6053720 0/6053720 0/6053720 0/6053720 00:00:00 00:00:00 00:00:00 streaming async 0 active 0/6053720 reserved NULL Unmanaged Replication Slot Status No unmanaged replication slots found Managed roles status No roles managed Tablespaces status No managed tablespaces Pod Disruption Budgets status Name Role Expected Pods Current Healthy Minimum Desired Healthy Disruptions Allowed ---- ---- ------------- --------------- ----------------------- ------------------- sandbox replica 2 2 1 1 sandbox-primary primary 1 1 1 0 Instances status Name Current LSN Replication role Status QoS Manager Version Node ---- ----------- ---------------- ------ --- --------------- ---- sandbox-1 0/6053720 Primary OK BestEffort 1.25.3 k8s-eu-worker sandbox-2 0/6053720 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker2 sandbox-3 0/6053720 Standby (async) OK BestEffort 1.25.3 k8s-eu-worker With an additional -v (e.g. kubectl cnpg status sandbox -v -v ), you can also view PostgreSQL configuration, HBA settings, and certificates. The command also supports output in yaml and json format.","title":"Status"},{"location":"kubectl-plugin/#promote","text":"The meaning of this command is to promote a pod in the cluster to primary, so you can start with maintenance work or test a switch-over situation in your cluster: kubectl cnpg promote CLUSTER CLUSTER-INSTANCE Or you can use the instance node number to promote: kubectl cnpg promote CLUSTER INSTANCE","title":"Promote"},{"location":"kubectl-plugin/#certificates","text":"Clusters created using the CloudNativePG operator work with a CA to sign a TLS authentication certificate. To get a certificate, you need to provide a name for the secret to store the credentials, the cluster name, and a user for this certificate: kubectl cnpg certificate cluster-cert --cnpg-cluster CLUSTER --cnpg-user USER After the secret it's created, you can get it using kubectl : kubectl get secret cluster-cert And the content of the same in plain text using the following commands: kubectl get secret cluster-cert -o json | jq -r '.data | map(@base64d) | .[]'","title":"Certificates"},{"location":"kubectl-plugin/#restart","text":"The kubectl cnpg restart command can be used in two cases: requesting the operator to orchestrate a rollout restart for a certain cluster. This is useful to apply configuration changes to cluster dependent objects, such as ConfigMaps containing custom monitoring queries. request a single instance restart, either in-place if the instance is the cluster's primary or deleting and recreating the pod if it is a replica. # this command will restart a whole cluster in a rollout fashion kubectl cnpg restart CLUSTER # this command will restart a single instance, according to the policy above kubectl cnpg restart CLUSTER INSTANCE If the in-place restart is requested but the change cannot be applied without a switchover, the switchover will take precedence over the in-place restart. A common case for this will be a minor upgrade of PostgreSQL image. Note If you want ConfigMaps and Secrets to be automatically reloaded by instances, you can add a label with key cnpg.io/reload to it.","title":"Restart"},{"location":"kubectl-plugin/#reload","text":"The kubectl cnpg reload command requests the operator to trigger a reconciliation loop for a certain cluster. This is useful to apply configuration changes to cluster dependent objects, such as ConfigMaps containing custom monitoring queries. The following command will reload all configurations for a given cluster: kubectl cnpg reload CLUSTER","title":"Reload"},{"location":"kubectl-plugin/#maintenance","text":"The kubectl cnpg maintenance command helps to modify one or more clusters across namespaces and set the maintenance window values, it will change the following fields: .spec.nodeMaintenanceWindow.inProgress .spec.nodeMaintenanceWindow.reusePVC Accepts as argument set and unset using this to set the inProgress to true in case set and to false in case of unset . By default, reusePVC is always set to false unless the --reusePVC flag is passed. The plugin will ask for a confirmation with a list of the cluster to modify and their new values, if this is accepted this action will be applied to all the cluster in the list. If you want to set in maintenance all the PostgreSQL in your Kubernetes cluster, just need to write the following command: kubectl cnpg maintenance set --all-namespaces And you'll have the list of all the cluster to update The following are the new values for the clusters Namespace Cluster Name Maintenance reusePVC --------- ------------ ----------- -------- default cluster-example true false default pg-backup true false test cluster-example true false Do you want to proceed? [y/n]: y","title":"Maintenance"},{"location":"kubectl-plugin/#report","text":"The kubectl cnpg report command bundles various pieces of information into a ZIP file. It aims to provide the needed context to debug problems with clusters in production. It has two sub-commands: operator and cluster .","title":"Report"},{"location":"kubectl-plugin/#report-operator","text":"The operator sub-command requests the operator to provide information regarding the operator deployment, configuration and events. Important All confidential information in Secrets and ConfigMaps is REDACTED. The Data map will show the keys but the values will be empty. The flag -S / --stopRedaction will defeat the redaction and show the values. Use only at your own risk, this will share private data. Note By default, operator logs are not collected, but you can enable operator log collection with the --logs flag deployment information : the operator Deployment and operator Pod configuration : the Secrets and ConfigMaps in the operator namespace events : the Events in the operator namespace webhook configuration : the mutating and validating webhook configurations webhook service : the webhook service logs : logs for the operator Pod (optional, off by default) in JSON-lines format The command will generate a ZIP file containing various manifest in YAML format (by default, but settable to JSON with the -o flag). Use the -f flag to name a result file explicitly. If the -f flag is not used, a default time-stamped filename is created for the zip file. Note The report plugin obeys kubectl conventions, and will look for objects constrained by namespace. The CNPG Operator will generally not be installed in the same namespace as the clusters. E.g. the default installation namespace is cnpg-system kubectl cnpg report operator -n cnpg-system results in Successfully written report to \"report_operator_.zip\" (format: \"yaml\") With the -f flag set: kubectl cnpg report operator -n cnpg-system -f reportRedacted.zip Unzipping the file will produce a time-stamped top-level folder to keep the directory tidy: unzip reportRedacted.zip will result in: Archive: reportRedacted.zip creating: report_operator_/ creating: report_operator_/manifests/ inflating: report_operator_/manifests/deployment.yaml inflating: report_operator_/manifests/operator-pod.yaml inflating: report_operator_/manifests/events.yaml inflating: report_operator_/manifests/validating-webhook-configuration.yaml inflating: report_operator_/manifests/mutating-webhook-configuration.yaml inflating: report_operator_/manifests/webhook-service.yaml inflating: report_operator_/manifests/cnpg-ca-secret(secret).yaml inflating: report_operator_/manifests/cnpg-webhook-cert(secret).yaml If you activated the --logs option, you'd see an extra subdirectory: Archive: report_operator_.zip creating: report_operator_/operator-logs/ inflating: report_operator_/operator-logs/cnpg-controller-manager-66fb98dbc5-pxkmh-logs.jsonl Note The plugin will try to get the PREVIOUS operator's logs, which is helpful when investigating restarted operators. In all cases, it will also try to get the CURRENT operator logs. If current and previous logs are available, it will show them both. ====== Beginning of Previous Log ===== 2023-03-28T12:56:41.251711811Z {\"level\":\"info\",\"ts\":\"2023-03-28T12:56:41Z\",\"logger\":\"setup\",\"msg\":\"Starting CloudNativePG Operator\",\"version\":\"1.25.3\",\"build\":{\"Version\":\"1.25.3+dev107\",\"Commit\":\"cc9bab17\",\"Date\":\"2023-03-28\"}} 2023-03-28T12:56:41.251851909Z {\"level\":\"info\",\"ts\":\"2023-03-28T12:56:41Z\",\"logger\":\"setup\",\"msg\":\"Starting pprof HTTP server\",\"addr\":\"0.0.0.0:6060\"} ====== End of Previous Log ===== 2023-03-28T12:57:09.854306024Z {\"level\":\"info\",\"ts\":\"2023-03-28T12:57:09Z\",\"logger\":\"setup\",\"msg\":\"Starting CloudNativePG Operator\",\"version\":\"1.25.3\",\"build\":{\"Version\":\"1.25.3+dev107\",\"Commit\":\"cc9bab17\",\"Date\":\"2023-03-28\"}} 2023-03-28T12:57:09.854363943Z {\"level\":\"info\",\"ts\":\"2023-03-28T12:57:09Z\",\"logger\":\"setup\",\"msg\":\"Starting pprof HTTP server\",\"addr\":\"0.0.0.0:6060\"} If the operator hasn't been restarted, you'll still see the ====== Begin \u2026 and ====== End \u2026 guards, with no content inside. You can verify that the confidential information is REDACTED by default: cd report_operator_/manifests/ head cnpg-ca-secret\\(secret\\).yaml data: ca.crt: \"\" ca.key: \"\" metadata: creationTimestamp: \"2022-03-22T10:42:28Z\" managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: With the -S ( --stopRedaction ) option activated, secrets are shown: kubectl cnpg report operator -n cnpg-system -f reportNonRedacted.zip -S You'll get a reminder that you're about to view confidential information: WARNING: secret Redaction is OFF. Use it with caution Successfully written report to \"reportNonRedacted.zip\" (format: \"yaml\") unzip reportNonRedacted.zip head cnpg-ca-secret\\(secret\\).yaml data: ca.crt: LS0tLS1CRUdJTiBD\u2026 ca.key: LS0tLS1CRUdJTiBF\u2026 metadata: creationTimestamp: \"2022-03-22T10:42:28Z\" managedFields: - apiVersion: v1 fieldsType: FieldsV1","title":"report Operator"},{"location":"kubectl-plugin/#report-cluster","text":"The cluster sub-command gathers the following: cluster resources : the cluster information, same as kubectl get cluster -o yaml cluster pods : pods in the cluster namespace matching the cluster name cluster jobs : jobs, if any, in the cluster namespace matching the cluster name events : events in the cluster namespace pod logs : logs for the cluster Pods (optional, off by default) in JSON-lines format job logs : logs for the Pods created by jobs (optional, off by default) in JSON-lines format The cluster sub-command accepts the -f and -o flags, as the operator does. If the -f flag is not used, a default timestamped report name will be used. Note that the cluster information does not contain configuration Secrets / ConfigMaps, so the -S is disabled. Note By default, cluster logs are not collected, but you can enable cluster log collection with the --logs flag Usage: kubectl cnpg report cluster CLUSTER [flags] Note that, unlike the operator sub-command, for the cluster sub-command you need to provide the cluster name, and very likely the namespace, unless the cluster is in the default one. kubectl cnpg report cluster CLUSTER -f report.zip [-n NAMESPACE] and then: unzip report.zip Archive: report.zip creating: report_cluster_example_/ creating: report_cluster_example_/manifests/ inflating: report_cluster_example_/manifests/cluster.yaml inflating: report_cluster_example_/manifests/cluster-pods.yaml inflating: report_cluster_example_/manifests/cluster-jobs.yaml inflating: report_cluster_example_/manifests/events.yaml Remember that you can use the --logs flag to add the pod and job logs to the ZIP. kubectl cnpg report cluster CLUSTER [-n NAMESPACE] --logs will result in: Successfully written report to \"report_cluster_example_.zip\" (format: \"yaml\") unzip report_cluster_.zip Archive: report_cluster_example_.zip creating: report_cluster_example_/ creating: report_cluster_example_/manifests/ inflating: report_cluster_example_/manifests/cluster.yaml inflating: report_cluster_example_/manifests/cluster-pods.yaml inflating: report_cluster_example_/manifests/cluster-jobs.yaml inflating: report_cluster_example_/manifests/events.yaml creating: report_cluster_example_/logs/ inflating: report_cluster_example_/logs/cluster-example-full-1.jsonl creating: report_cluster_example_/job-logs/ inflating: report_cluster_example_/job-logs/cluster-example-full-1-initdb-qnnvw.jsonl inflating: report_cluster_example_/job-logs/cluster-example-full-2-join-tvj8r.jsonl","title":"report Cluster"},{"location":"kubectl-plugin/#logs","text":"The kubectl cnpg logs command allows to follow the logs of a collection of pods related to CloudNativePG in a single go. It has at the moment one available sub-command: cluster .","title":"Logs"},{"location":"kubectl-plugin/#cluster-logs","text":"The cluster sub-command gathers all the pod logs for a cluster in a single stream or file. This means that you can get all the pod logs in a single terminal window, with a single invocation of the command. As in all the cnpg plugin sub-commands, you can get instructions and help with the -h flag: kubectl cnpg logs cluster -h The logs command will display logs in JSON-lines format, unless the --timestamps flag is used, in which case, a human-readable timestamp will be prepended to each line. In this case, lines will no longer be valid JSON, and tools such as jq may not work as desired. If the logs cluster sub-command is given the -f flag (aka --follow ), it will follow the cluster pod logs, and will also watch for any new pods created in the cluster after the command has been invoked. Any new pods found, including pods that have been restarted or re-created, will also have their pods followed. The logs will be displayed in the terminal's standard-out. This command will only exit when the cluster has no more pods left, or when it is interrupted by the user. If logs is called without the -f option, it will read the logs from all cluster pods until the time of invocation and display them in the terminal's standard-out, then exit. The -o or --output flag can be provided, to specify the name of the file where the logs should be saved, instead of displaying over standard-out. The --tail flag can be used to specify how many log lines will be retrieved from each pod in the cluster. By default, the logs cluster sub-command will display all the logs from each pod in the cluster. If combined with the \"follow\" flag -f , the number of logs specified by --tail will be retrieved until the current time, and from then the new logs will be followed. NOTE: unlike other cnpg plugin commands, the -f is used to denote \"follow\" rather than specify a file. This keeps with the convention of kubectl logs , which takes -f to mean the logs should be followed. Usage: kubectl cnpg logs cluster CLUSTER [flags] Using the -f option to follow: kubectl cnpg report cluster CLUSTER -f Using --tail option to display 3 lines from each pod and the -f option to follow: kubectl cnpg report cluster CLUSTER -f --tail 3 {\"level\":\"info\",\"ts\":\"2023-06-30T13:37:33Z\",\"logger\":\"postgres\",\"msg\":\"2023-06-30 13:37:33.142 UTC [26] LOG: ending log output to stderr\",\"source\":\"/controller/log/postgres\",\"logging_pod\":\"cluster-example-3\"} {\"level\":\"info\",\"ts\":\"2023-06-30T13:37:33Z\",\"logger\":\"postgres\",\"msg\":\"2023-06-30 13:37:33.142 UTC [26] HINT: Future log output will go to log destination \\\"csvlog\\\".\",\"source\":\"/controller/log/postgres\",\"logging_pod\":\"cluster-example-3\"} \u2026 \u2026 With the -o option omitted, and with --output specified: $ kubectl cnpg logs cluster CLUSTER --output my-cluster.log Successfully written logs to \"my-cluster.log\"","title":"Cluster logs"},{"location":"kubectl-plugin/#pretty","text":"The pretty sub-command reads a log stream from standard input, formats it into a human-readable output, and attempts to sort the entries by timestamp. It can be used in combination with kubectl cnpg logs cluster , as shown in the following example: $ kubectl cnpg logs cluster cluster-example | kubectl cnpg logs pretty 2024-10-15T17:35:00.336 INFO cluster-example-1 instance-manager Starting CloudNativePG Instance Manager 2024-10-15T17:35:00.336 INFO cluster-example-1 instance-manager Checking for free disk space for WALs before starting PostgreSQL 2024-10-15T17:35:00.347 INFO cluster-example-1 instance-manager starting tablespace manager 2024-10-15T17:35:00.347 INFO cluster-example-1 instance-manager starting external server manager [...] Alternatively, it can be used in combination with other commands that produce CNPG logs in JSON format, such as stern , or kubectl logs , as in the following example: $ kubectl logs cluster-example-1 | kubectl cnpg logs pretty 2024-10-15T17:35:00.336 INFO cluster-example-1 instance-manager Starting CloudNativePG Instance Manager 2024-10-15T17:35:00.336 INFO cluster-example-1 instance-manager Checking for free disk space for WALs before starting PostgreSQL 2024-10-15T17:35:00.347 INFO cluster-example-1 instance-manager starting tablespace manager 2024-10-15T17:35:00.347 INFO cluster-example-1 instance-manager starting external server manager [...] The pretty sub-command also supports advanced log filtering, allowing users to display logs for specific pods or loggers, or to filter logs by severity level. Here's an example: $ kubectl cnpg logs cluster cluster-example | kubectl cnpg logs pretty --pods cluster-example-1 --loggers postgres --log-level info 2024-10-15T17:35:00.509 INFO cluster-example-1 postgres 2024-10-15 17:35:00.509 UTC [29] LOG: redirecting log output to logging collector process 2024-10-15T17:35:00.509 INFO cluster-example-1 postgres 2024-10-15 17:35:00.509 UTC [29] HINT: Future log output will appear in directory \"/controller/log\"... 2024-10-15T17:35:00.510 INFO cluster-example-1 postgres 2024-10-15 17:35:00.509 UTC [29] LOG: ending log output to stderr 2024-10-15T17:35:00.510 INFO cluster-example-1 postgres ending log output to stderr [...] The pretty sub-command will try to sort the log stream, to make logs easier to reason about. In order to achieve this, it gathers the logs into groups, and within groups it sorts by timestamp. This is the only way to sort interactively, as pretty may be piped from a command in \"follow\" mode. The sub-command will add a group separator line, --- , at the end of each sorted group. The size of the grouping can be configured via the --sorting-group-size flag (default: 1000), as illustrated in the following example: $ kubectl cnpg logs cluster cluster-example | kubectl cnpg logs pretty --sorting-group-size=3 2024-10-15T17:35:20.426 INFO cluster-example-2 instance-manager Starting CloudNativePG Instance Manager 2024-10-15T17:35:20.426 INFO cluster-example-2 instance-manager Checking for free disk space for WALs before starting PostgreSQL 2024-10-15T17:35:20.438 INFO cluster-example-2 instance-manager starting tablespace manager --- 2024-10-15T17:35:20.438 INFO cluster-example-2 instance-manager starting external server manager 2024-10-15T17:35:20.438 INFO cluster-example-2 instance-manager starting controller-runtime manager 2024-10-15T17:35:20.439 INFO cluster-example-2 instance-manager Starting EventSource --- [...] To explore all available options, use the -h flag for detailed explanations of the supported flags and their usage. Info You can also increase the verbosity of the log by adding more -v options.","title":"Pretty"},{"location":"kubectl-plugin/#destroy","text":"The kubectl cnpg destroy command helps remove an instance and all the associated PVCs from a Kubernetes cluster. The optional --keep-pvc flag, if specified, allows you to keep the PVCs, while removing all metadata.ownerReferences that were set by the instance. Additionally, the cnpg.io/pvcStatus label on the PVCs will change from ready to detached to signify that they are no longer in use. Running again the command without the --keep-pvc flag will remove the detached PVCs. Usage: kubectl cnpg destroy CLUSTER INSTANCE The following example removes the cluster-example-2 pod and the associated PVCs: kubectl cnpg destroy cluster-example 2","title":"Destroy"},{"location":"kubectl-plugin/#cluster-hibernation","text":"Sometimes you may want to suspend the execution of a CloudNativePG Cluster while retaining its data, then resume its activity at a later time. We've called this feature cluster hibernation . Hibernation is only available via the kubectl cnpg hibernate [on|off] commands. Hibernating a CloudNativePG cluster means destroying all the resources generated by the cluster, except the PVCs that belong to the PostgreSQL primary instance. You can hibernate a cluster with: kubectl cnpg hibernate on CLUSTER This will: shutdown every PostgreSQL instance detach the PVCs containing the data of the primary instance, and annotate them with the latest database status and the latest cluster configuration delete the Cluster resource, including every generated resource - except the aforementioned PVCs When hibernated, a CloudNativePG cluster is represented by just a group of PVCs, in which the one containing the PGDATA is annotated with the latest available status, including content from pg_controldata . Warning A cluster having fenced instances cannot be hibernated, as fencing is part of the hibernation procedure too. In case of error the operator will not be able to revert the procedure. You can still force the operation with: kubectl cnpg hibernate on CLUSTER --force A hibernated cluster can be resumed with: kubectl cnpg hibernate off CLUSTER Once the cluster has been hibernated, it's possible to show the last configuration and the status that PostgreSQL had after it was shut down. That can be done with: kubectl cnpg hibernate status CLUSTER","title":"Cluster hibernation"},{"location":"kubectl-plugin/#benchmarking-the-database-with-pgbench","text":"Pgbench can be run against an existing PostgreSQL cluster with following command: kubectl cnpg pgbench CLUSTER -- --time 30 --client 1 --jobs 1 Refer to the Benchmarking pgbench section for more details.","title":"Benchmarking the database with pgbench"},{"location":"kubectl-plugin/#benchmarking-the-storage-with-fio","text":"fio can be run on an existing storage class with following command: kubectl cnpg fio FIO_JOB_NAME [-n NAMESPACE] Refer to the Benchmarking fio section for more details.","title":"Benchmarking the storage with fio"},{"location":"kubectl-plugin/#requesting-a-new-physical-backup","text":"The kubectl cnpg backup command requests a new physical backup for an existing Postgres cluster by creating a new Backup resource. The following example requests an on-demand backup for a given cluster: kubectl cnpg backup CLUSTER or, if using volume snapshots: kubectl cnpg backup CLUSTER -m volumeSnapshot The created backup will be named after the request time: $ kubectl cnpg backup cluster-example backup/cluster-example-20230121002300 created By default, a newly created backup will use the backup target policy defined in the cluster to choose which instance to run on. However, you can override this policy with the --backup-target option. In the case of volume snapshot backups, you can also use the --online option to request an online/hot backup or an offline/cold one: additionally, you can also tune online backups by explicitly setting the --immediate-checkpoint and --wait-for-archive options. The \"Backup\" section contains more information about the configuration settings.","title":"Requesting a new physical backup"},{"location":"kubectl-plugin/#launching-psql","text":"The kubectl cnpg psql CLUSTER command starts a new PostgreSQL interactive front-end process (psql) connected to an existing Postgres cluster, as if you were running it from the actual pod. This means that you will be using the postgres user. Important As you will be connecting as postgres user, in production environments this method should be used with extreme care, by authorized personnel only. $ kubectl cnpg psql cluster-example psql (17.5 (Debian 17.5-1.pgdg110+1)) Type \"help\" for help. postgres=# By default, the command will connect to the primary instance. The user can select to work against a replica by using the --replica option: $ kubectl cnpg psql --replica cluster-example psql (17.5 (Debian 17.5-1.pgdg110+1)) Type \"help\" for help. postgres=# select pg_is_in_recovery(); pg_is_in_recovery ------------------- t (1 row) postgres=# \\q This command will start kubectl exec , and the kubectl executable must be reachable in your PATH variable to correctly work.","title":"Launching psql"},{"location":"kubectl-plugin/#snapshotting-a-postgres-cluster","text":"Warning The kubectl cnpg snapshot command has been removed. Please use the backup command to request backups using volume snapshots.","title":"Snapshotting a Postgres cluster"},{"location":"kubectl-plugin/#using-pgadmin4-for-evaluationdemonstration-purposes-only","text":"pgAdmin stands as the most popular and feature-rich open-source administration and development platform for PostgreSQL. For more information on the project, please refer to the official documentation . Given that the pgAdmin Development Team maintains official Docker container images, you can install pgAdmin in your environment as a standard Kubernetes deployment. Important Deployment of pgAdmin in Kubernetes production environments is beyond the scope of this document and, more broadly, of the CloudNativePG project. However, for the purposes of demonstration and evaluation , CloudNativePG offers a suitable solution. The cnpg plugin implements the pgadmin4 command, providing a straightforward method to connect to a given database Cluster and navigate its content in a local environment such as kind . For example, you can install a demo deployment of pgAdmin4 for the cluster-example cluster as follows: kubectl cnpg pgadmin4 cluster-example This command will produce: ConfigMap/cluster-example-pgadmin4 created Deployment/cluster-example-pgadmin4 created Service/cluster-example-pgadmin4 created Secret/cluster-example-pgadmin4 created [...] After deploying pgAdmin, forward the port using kubectl and connect through your browser by following the on-screen instructions. As usual, you can use the --dry-run option to generate the YAML file: kubectl cnpg pgadmin4 --dry-run cluster-example pgAdmin4 can be installed in either desktop or server mode, with the default being server. In server mode, authentication is required using a randomly generated password, and users must manually specify the database to connect to. On the other hand, desktop mode initiates a pgAdmin web interface without requiring authentication. It automatically connects to the app database as the app user, making it ideal for quick demos, such as on a local deployment using kind : kubectl cnpg pgadmin4 --mode desktop cluster-example After concluding your demo, ensure the termination of the pgAdmin deployment by executing: kubectl cnpg pgadmin4 --dry-run cluster-example | kubectl delete -f - Warning Never deploy pgAdmin in production using the plugin.","title":"Using pgAdmin4 for evaluation/demonstration purposes only"},{"location":"kubectl-plugin/#logical-replication-publications","text":"The cnpg publication command group is designed to streamline the creation and removal of PostgreSQL logical replication publications . Be aware that these commands are primarily intended for assisting in the creation of logical replication publications, particularly on remote PostgreSQL databases. Warning It is crucial to have a solid understanding of both the capabilities and limitations of PostgreSQL's native logical replication system before using these commands. In particular, be mindful of the logical replication restrictions .","title":"Logical Replication Publications"},{"location":"kubectl-plugin/#creating-a-new-publication","text":"To create a logical replication publication, use the cnpg publication create command. The basic structure of this command is as follows: kubectl cnpg publication create \\ --publication PUBLICATION_NAME \\ [--external-cluster EXTERNAL_CLUSTER] LOCAL_CLUSTER [options] There are two primary use cases: With --external-cluster : Use this option to create a publication on an external cluster (i.e. defined in the externalClusters stanza). The commands will be issued from the LOCAL_CLUSTER , but the publication will be for the data in EXTERNAL_CLUSTER . Without --external-cluster : Use this option to create a publication in the LOCAL_CLUSTER PostgreSQL Cluster (by default, the app database). Warning When connecting to an external cluster, ensure that the specified user has sufficient permissions to execute the CREATE PUBLICATION command. You have several options, similar to the CREATE PUBLICATION command, to define the group of tables to replicate. Notable options include: If you specify the --all-tables option, you create a publication FOR ALL TABLES . Alternatively, you can specify multiple occurrences of: --table : Add a specific table (with an expression) to the publication. --schema : Include all tables in the specified database schema (available from PostgreSQL 15). The --dry-run option enables you to preview the SQL commands that the plugin will execute. For additional information and detailed instructions, type the following command: kubectl cnpg publication create --help","title":"Creating a new publication"},{"location":"kubectl-plugin/#example","text":"Given a source-cluster and a destination-cluster , we would like to create a publication for the data on source-cluster . The destination-cluster has an entry in the externalClusters stanza pointing to source-cluster . We can run: kubectl cnpg publication create destination-cluster \\ --external-cluster=source-cluster --all-tables which will create a publication for all tables on source-cluster , running the SQL commands on the destination-cluster . Or instead, we can run: kubectl cnpg publication create source-cluster \\ --publication=app --all-tables which will create a publication named app for all the tables in the source-cluster , running the SQL commands on the source cluster. Info There are two sample files that have been provided for illustration and inspiration: logical-source and logical-destination .","title":"Example"},{"location":"kubectl-plugin/#dropping-a-publication","text":"The cnpg publication drop command seamlessly complements the create command by offering similar key options, including the publication name, cluster name, and an optional external cluster. You can drop a PUBLICATION with the following command structure: kubectl cnpg publication drop \\ --publication PUBLICATION_NAME \\ [--external-cluster EXTERNAL_CLUSTER] LOCAL_CLUSTER [options] To access further details and precise instructions, use the following command: kubectl cnpg publication drop --help","title":"Dropping a publication"},{"location":"kubectl-plugin/#logical-replication-subscriptions","text":"The cnpg subscription command group is a dedicated set of commands designed to simplify the creation and removal of PostgreSQL logical replication subscriptions . These commands are specifically crafted to aid in the establishment of logical replication subscriptions, especially when dealing with remote PostgreSQL databases. Warning Before using these commands, it is essential to have a comprehensive understanding of both the capabilities and limitations of PostgreSQL's native logical replication system. In particular, be mindful of the logical replication restrictions . In addition to subscription management, we provide a helpful command for synchronizing all sequences from the source cluster. While its applicability may vary, this command can be particularly useful in scenarios involving major upgrades or data import from remote servers.","title":"Logical Replication Subscriptions"},{"location":"kubectl-plugin/#creating-a-new-subscription","text":"To create a logical replication subscription, use the cnpg subscription create command. The basic structure of this command is as follows: kubectl cnpg subscription create \\ --subscription SUBSCRIPTION_NAME \\ --publication PUBLICATION_NAME \\ --external-cluster EXTERNAL_CLUSTER \\ LOCAL_CLUSTER [options] This command configures a subscription directed towards the specified publication in the designated external cluster, as defined in the externalClusters stanza of the LOCAL_CLUSTER . For additional information and detailed instructions, type the following command: kubectl cnpg subscription create --help","title":"Creating a new subscription"},{"location":"kubectl-plugin/#example_1","text":"As in the section on publications, we have a source-cluster and a destination-cluster , and we have already created a publication called app . The following command: kubectl cnpg subscription create destination-cluster \\ --external-cluster=source-cluster \\ --publication=app --subscription=app will create a subscription for app on the destination cluster. Warning Prioritize testing subscriptions in a non-production environment to ensure their effectiveness and identify any potential issues before implementing them in a production setting. Info There are two sample files that have been provided for illustration and inspiration: logical-source and logical-destination .","title":"Example"},{"location":"kubectl-plugin/#dropping-a-subscription","text":"The cnpg subscription drop command seamlessly complements the create command. You can drop a SUBSCRIPTION with the following command structure: kubectl cnpg subcription drop \\ --subscription SUBSCRIPTION_NAME \\ LOCAL_CLUSTER [options] To access further details and precise instructions, use the following command: kubectl cnpg subscription drop --help","title":"Dropping a subscription"},{"location":"kubectl-plugin/#synchronizing-sequences","text":"One notable constraint of PostgreSQL logical replication, implemented through publications and subscriptions, is the lack of sequence synchronization. This becomes particularly relevant when utilizing logical replication for live database migration, especially to a higher version of PostgreSQL. A crucial step in this process involves updating sequences before transitioning applications to the new database ( cutover ). To address this limitation, the cnpg subscription sync-sequences command offers a solution. This command establishes a connection with the source database, retrieves all relevant sequences, and subsequently updates local sequences with matching identities (based on database schema and sequence name). You can use the command as shown below: kubectl cnpg subscription sync-sequences \\ --subscription SUBSCRIPTION_NAME \\ LOCAL_CLUSTER For comprehensive details and specific instructions, utilize the following command: kubectl cnpg subscription sync-sequences --help","title":"Synchronizing sequences"},{"location":"kubectl-plugin/#example_2","text":"As in the previous sections for publication and subscription, we have a source-cluster and a destination-cluster . The publication and the subscription, both called app , are already present. The following command will synchronize the sequences involved in the app subscription, from the source cluster into the destination cluster. kubectl cnpg subscription sync-sequences destination-cluster \\ --subscription=app Warning Prioritize testing subscriptions in a non-production environment to guarantee their effectiveness and detect any potential issues before deploying them in a production setting.","title":"Example"},{"location":"kubectl-plugin/#integration-with-k9s","text":"The cnpg plugin can be easily integrated in K9s , a popular terminal-based UI to interact with Kubernetes clusters. See k9s/plugins.yml for details.","title":"Integration with K9s"},{"location":"kubectl-plugin/#permissions-required-by-the-plugin","text":"The plugin requires a set of Kubernetes permissions that depends on the command to execute. These permissions may affect resources and sub-resources like Pods, PDBs, PVCs, and enable actions like get , delete , patch . The following table contains the full details: Command Resource Permissions backup clusters: get backups: create certificate clusters: get secrets: get,create destroy pods: get,delete jobs: delete,list PVCs: list,delete,update fencing clusters: get,patch pods: get fio PVCs: create configmaps: create deployment: create hibernate clusters: get,patch,delete pods: list,get,delete pods/exec: create jobs: list PVCs: get,list,update,patch,delete install none logs clusters: get pods: list pods/log: get maintenance clusters: get,patch,list pgadmin4 clusters: get configmaps: create deployments: create services: create secrets: create pgbench clusters: get jobs: create promote clusters: get clusters/status: patch pods: get psql pods: get,list pods/exec: create publication clusters: get pods: get,list pods/exec: create reload clusters: get,patch report cluster clusters: get pods: list pods/log: get jobs: list events: list PVCs: list report operator configmaps: get deployments: get events: list pods: list pods/log: get secrets: get services: get mutatingwebhookconfigurations: list 1 validatingwebhookconfigurations: list 1 If OLM is present on the K8s cluster, also: clusterserviceversions: list installplans: list subscriptions: list restart clusters: get,patch pods: get,delete status clusters: get pods: list pods/exec: create pods/proxy: create PDBs: list subscription clusters: get pods: get,list pods/exec: create version none The permissions are cluster scope ClusterRole resources. \u21a9 \u21a9 Additionally, assigning the list permission on the clusters will enable autocompletion for multiple commands.","title":"Permissions required by the plugin"},{"location":"kubectl-plugin/#role-examples","text":"It is possible to create roles with restricted permissions. The following example creates a role that only has access to the cluster logs: --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: cnpg-log rules: - verbs: - get apiGroups: - postgresql.cnpg.io resources: - clusters - verbs: - list apiGroups: - '' resources: - pods - verbs: - get apiGroups: - '' resources: - pods/log The next example shows a role with the minimal permissions required to get the cluster status using the plugin's status command: apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: cnpg-status rules: - verbs: - get apiGroups: - postgresql.cnpg.io resources: - clusters - verbs: - list apiGroups: - '' resources: - pods - verbs: - create apiGroups: - '' resources: - pods/exec - verbs: - create apiGroups: - '' resources: - pods/proxy - verbs: - list apiGroups: - policy resources: - poddisruptionbudgets Important Keeping the verbs restricted per resources and per apiGroups helps to prevent inadvertently granting more than intended permissions.","title":"Role examples"},{"location":"kubernetes_upgrade/","text":"Kubernetes Upgrade and Maintenance Maintaining an up-to-date Kubernetes cluster is crucial for ensuring optimal performance and security, particularly for self-managed clusters, especially those running on bare metal infrastructure. Regular updates help address technical debt and mitigate business risks, despite the controlled downtimes associated with temporarily removing a node from the cluster for maintenance purposes. For further insights on embracing risk in operations, refer to the \"Embracing Risk\" chapter from the Site Reliability Engineering book. Importance of Regular Updates Updating Kubernetes involves planning and executing maintenance tasks, such as applying security updates to underlying Linux servers, replacing malfunctioning hardware components, or upgrading the cluster to the latest Kubernetes version. These activities are essential for maintaining a robust and secure infrastructure. Maintenance Operations in a Cluster Typically, maintenance operations are carried out on one node at a time, following a structured process : eviction of workloads ( drain ): workloads are gracefully moved away from the node to be updated, ensuring a smooth transition. performing the operation: the actual maintenance operation, such as a system update or hardware replacement, is executed. rejoining the node to the cluster ( uncordon ): the updated node is reintegrated into the cluster, ready to resume its responsibilities. This process requires either stopping workloads for the entire upgrade duration or migrating them to other nodes in the cluster. Temporary PostgreSQL Cluster Degradation While the standard approach ensures service reliability and leverages Kubernetes' self-healing capabilities, there are scenarios where operating with a temporarily degraded cluster may be acceptable. This is particularly relevant for PostgreSQL clusters relying on node-local storage , where the storage is local to the Kubernetes worker node running the PostgreSQL database. Node-local storage, or simply local storage , is employed to enhance performance. Note If your database files reside on shared storage accessible over the network, the default self-healing behavior of the operator can efficiently handle scenarios where volumes are reused by pods on different nodes after a drain operation. In such cases, you can skip the remaining sections of this document. Pod Disruption Budgets By default, CloudNativePG safeguards Postgres cluster operations. If a node is to be drained and contains a cluster's primary instance, a switchover happens ahead of the drain. Once the instance in the node is downgraded to replica, the draining can resume. For single-instance clusters, a switchover is not possible, so CloudNativePG will prevent draining the node where the instance is housed. Additionally, in clusters with 3 or more instances, CloudNativePG guarantees that only one replica at a time is gracefully shut down during a drain operation. Each PostgreSQL Cluster is equipped with two associated PodDisruptionBudget resources - you can easily confirm it with the kubectl get pdb command. Our recommendation is to leave pod disruption budgets enabled for every production Postgres cluster. This can be effortlessly managed by toggling the .spec.enablePDB option, as detailed in the API reference . PostgreSQL Clusters used for Development or Testing For PostgreSQL clusters used for development purposes, often consisting of a single instance, it is essential to disable pod disruption budgets. Failure to do so will prevent the node hosting that cluster from being drained. The following example illustrates how to disable pod disruption budgets for a 1-instance development cluster: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: dev spec: instances: 1 enablePDB: false storage: size: 1Gi This configuration ensures smoother maintenance procedures without restrictions on draining the node during development activities. Node Maintenance Window Important While CloudNativePG will continue supporting the node maintenance window, it is currently recommended to transition to direct control of pod disruption budgets, as explained in the previous section. This section is retained mainly for backward compatibility. Prior to release 1.23, CloudNativePG had just one declarative mechanism to manage Kubernetes upgrades when dealing with local storage: you had to temporarily put the cluster in maintenance mode through the nodeMaintenanceWindow option to avoid standard self-healing procedures to kick in, while, for example, enlarging the partition on the physical node or updating the node itself. Warning Limit the duration of the maintenance window to the shortest amount of time possible. In this phase, some of the expected behaviors of Kubernetes are either disabled or running with some limitations, including self-healing, rolling updates, and Pod disruption budget. The nodeMaintenanceWindow option of the cluster has two further settings: inProgress : Boolean value that states if the maintenance window for the nodes is currently in progress or not. By default, it is set to off . During the maintenance window, the reusePVC option below is evaluated by the operator. reusePVC : Boolean value that defines if an existing PVC is reused or not during the maintenance operation. By default, it is set to on . When enabled , Kubernetes waits for the node to come up again and then reuses the existing PVC; the PodDisruptionBudget policy is temporarily removed. When disabled , Kubernetes forces the recreation of the Pod on a different node with a new PVC by relying on PostgreSQL's physical streaming replication, then destroys the old PVC together with the Pod. This scenario is generally not recommended unless the database's size is small, and re-cloning the new PostgreSQL instance takes shorter than waiting. This behavior does not apply to clusters with only one instance and reusePVC disabled: see section below. Note When performing the kubectl drain command, you will need to add the --delete-emptydir-data option. Don't be afraid: it refers to another volume internally used by the operator - not the PostgreSQL data directory. Important PodDisruptionBudget management can be disabled by setting the .spec.enablePDB field to false . In that case, the operator won't create PodDisruptionBudgets and will delete them if they were previously created. Single instance clusters with reusePVC set to false Important We recommend to always create clusters with more than one instance in order to guarantee high availability. Deleting the only PostgreSQL instance in a single instance cluster with reusePVC set to false would imply all data being lost, therefore we prevent users from draining nodes such instances might be running on, even in maintenance mode. However, in case maintenance is required for such a node you have two options: Enable reusePVC , accepting the downtime Replicate the instance on a different node and switch over the primary As long as a database service downtime is acceptable for your environment, draining the node is as simple as setting the nodeMaintenanceWindow to inProgress: true and reusePVC: true . This will allow the instance to be deleted and recreated as soon as the original PVC is available (e.g. with node local storage, as soon as the node is back up). Otherwise you will have to scale up the cluster, creating a new instance on a different node and promoting the new instance to primary in order to shut down the original one on the node undergoing maintenance. The only downtime in this case will be the duration of the switchover. A possible approach could be: Cordon the node on which the current instance is running. Scale up the cluster to 2 instances, could take some time depending on the database size. As soon as the new instance is running, the operator will automatically perform a switchover given that the current primary is running on a cordoned node. Scale back down the cluster to a single instance, this will delete the old instance The old primary's node can now be drained successfully, while leaving the new primary running on a new node.","title":"Kubernetes Upgrade and Maintenance"},{"location":"kubernetes_upgrade/#kubernetes-upgrade-and-maintenance","text":"Maintaining an up-to-date Kubernetes cluster is crucial for ensuring optimal performance and security, particularly for self-managed clusters, especially those running on bare metal infrastructure. Regular updates help address technical debt and mitigate business risks, despite the controlled downtimes associated with temporarily removing a node from the cluster for maintenance purposes. For further insights on embracing risk in operations, refer to the \"Embracing Risk\" chapter from the Site Reliability Engineering book.","title":"Kubernetes Upgrade and Maintenance"},{"location":"kubernetes_upgrade/#importance-of-regular-updates","text":"Updating Kubernetes involves planning and executing maintenance tasks, such as applying security updates to underlying Linux servers, replacing malfunctioning hardware components, or upgrading the cluster to the latest Kubernetes version. These activities are essential for maintaining a robust and secure infrastructure.","title":"Importance of Regular Updates"},{"location":"kubernetes_upgrade/#maintenance-operations-in-a-cluster","text":"Typically, maintenance operations are carried out on one node at a time, following a structured process : eviction of workloads ( drain ): workloads are gracefully moved away from the node to be updated, ensuring a smooth transition. performing the operation: the actual maintenance operation, such as a system update or hardware replacement, is executed. rejoining the node to the cluster ( uncordon ): the updated node is reintegrated into the cluster, ready to resume its responsibilities. This process requires either stopping workloads for the entire upgrade duration or migrating them to other nodes in the cluster.","title":"Maintenance Operations in a Cluster"},{"location":"kubernetes_upgrade/#temporary-postgresql-cluster-degradation","text":"While the standard approach ensures service reliability and leverages Kubernetes' self-healing capabilities, there are scenarios where operating with a temporarily degraded cluster may be acceptable. This is particularly relevant for PostgreSQL clusters relying on node-local storage , where the storage is local to the Kubernetes worker node running the PostgreSQL database. Node-local storage, or simply local storage , is employed to enhance performance. Note If your database files reside on shared storage accessible over the network, the default self-healing behavior of the operator can efficiently handle scenarios where volumes are reused by pods on different nodes after a drain operation. In such cases, you can skip the remaining sections of this document.","title":"Temporary PostgreSQL Cluster Degradation"},{"location":"kubernetes_upgrade/#pod-disruption-budgets","text":"By default, CloudNativePG safeguards Postgres cluster operations. If a node is to be drained and contains a cluster's primary instance, a switchover happens ahead of the drain. Once the instance in the node is downgraded to replica, the draining can resume. For single-instance clusters, a switchover is not possible, so CloudNativePG will prevent draining the node where the instance is housed. Additionally, in clusters with 3 or more instances, CloudNativePG guarantees that only one replica at a time is gracefully shut down during a drain operation. Each PostgreSQL Cluster is equipped with two associated PodDisruptionBudget resources - you can easily confirm it with the kubectl get pdb command. Our recommendation is to leave pod disruption budgets enabled for every production Postgres cluster. This can be effortlessly managed by toggling the .spec.enablePDB option, as detailed in the API reference .","title":"Pod Disruption Budgets"},{"location":"kubernetes_upgrade/#postgresql-clusters-used-for-development-or-testing","text":"For PostgreSQL clusters used for development purposes, often consisting of a single instance, it is essential to disable pod disruption budgets. Failure to do so will prevent the node hosting that cluster from being drained. The following example illustrates how to disable pod disruption budgets for a 1-instance development cluster: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: dev spec: instances: 1 enablePDB: false storage: size: 1Gi This configuration ensures smoother maintenance procedures without restrictions on draining the node during development activities.","title":"PostgreSQL Clusters used for Development or Testing"},{"location":"kubernetes_upgrade/#node-maintenance-window","text":"Important While CloudNativePG will continue supporting the node maintenance window, it is currently recommended to transition to direct control of pod disruption budgets, as explained in the previous section. This section is retained mainly for backward compatibility. Prior to release 1.23, CloudNativePG had just one declarative mechanism to manage Kubernetes upgrades when dealing with local storage: you had to temporarily put the cluster in maintenance mode through the nodeMaintenanceWindow option to avoid standard self-healing procedures to kick in, while, for example, enlarging the partition on the physical node or updating the node itself. Warning Limit the duration of the maintenance window to the shortest amount of time possible. In this phase, some of the expected behaviors of Kubernetes are either disabled or running with some limitations, including self-healing, rolling updates, and Pod disruption budget. The nodeMaintenanceWindow option of the cluster has two further settings: inProgress : Boolean value that states if the maintenance window for the nodes is currently in progress or not. By default, it is set to off . During the maintenance window, the reusePVC option below is evaluated by the operator. reusePVC : Boolean value that defines if an existing PVC is reused or not during the maintenance operation. By default, it is set to on . When enabled , Kubernetes waits for the node to come up again and then reuses the existing PVC; the PodDisruptionBudget policy is temporarily removed. When disabled , Kubernetes forces the recreation of the Pod on a different node with a new PVC by relying on PostgreSQL's physical streaming replication, then destroys the old PVC together with the Pod. This scenario is generally not recommended unless the database's size is small, and re-cloning the new PostgreSQL instance takes shorter than waiting. This behavior does not apply to clusters with only one instance and reusePVC disabled: see section below. Note When performing the kubectl drain command, you will need to add the --delete-emptydir-data option. Don't be afraid: it refers to another volume internally used by the operator - not the PostgreSQL data directory. Important PodDisruptionBudget management can be disabled by setting the .spec.enablePDB field to false . In that case, the operator won't create PodDisruptionBudgets and will delete them if they were previously created.","title":"Node Maintenance Window"},{"location":"kubernetes_upgrade/#single-instance-clusters-with-reusepvc-set-to-false","text":"Important We recommend to always create clusters with more than one instance in order to guarantee high availability. Deleting the only PostgreSQL instance in a single instance cluster with reusePVC set to false would imply all data being lost, therefore we prevent users from draining nodes such instances might be running on, even in maintenance mode. However, in case maintenance is required for such a node you have two options: Enable reusePVC , accepting the downtime Replicate the instance on a different node and switch over the primary As long as a database service downtime is acceptable for your environment, draining the node is as simple as setting the nodeMaintenanceWindow to inProgress: true and reusePVC: true . This will allow the instance to be deleted and recreated as soon as the original PVC is available (e.g. with node local storage, as soon as the node is back up). Otherwise you will have to scale up the cluster, creating a new instance on a different node and promoting the new instance to primary in order to shut down the original one on the node undergoing maintenance. The only downtime in this case will be the duration of the switchover. A possible approach could be: Cordon the node on which the current instance is running. Scale up the cluster to 2 instances, could take some time depending on the database size. As soon as the new instance is running, the operator will automatically perform a switchover given that the current primary is running on a cordoned node. Scale back down the cluster to a single instance, this will delete the old instance The old primary's node can now be drained successfully, while leaving the new primary running on a new node.","title":"Single instance clusters with reusePVC set to false"},{"location":"labels_annotations/","text":"Labels and annotations Resources in Kubernetes are organized in a flat structure, with no hierarchical information or relationship between them. However, such resources and objects can be linked together and put in relationship through labels and annotations . Info For more information, see the Kubernetes documentation on annotations and labels . In brief: An annotation is used to assign additional non-identifying information to resources with the goal of facilitating integration with external tools. A label is used to group objects and query them through the Kubernetes native selector capability. You can select one or more labels or annotations to use in your CloudNativePG deployments. Then you need to configure the operator so that when you define these labels or annotations in a cluster's metadata, they're inherited by all resources created by it (including pods). Note Label and annotation inheritance is the technique adopted by CloudNativePG instead of alternative approaches such as pod templates. Predefined labels CloudNativePG manages the following predefined labels: cnpg.io/backupDate The date of the backup in ISO 8601 format ( YYYYMMDD ). This label is available only on VolumeSnapshot resources. cnpg.io/backupName Backup identifier. This label is available only on VolumeSnapshot resources. cnpg.io/backupMonth The year/month when a backup was taken. This label is available only on VolumeSnapshot resources. cnpg.io/backupTimeline The timeline of the instance when a backup was taken. This label is available only on VolumeSnapshot resources. cnpg.io/backupYear The year a backup was taken. This label is available only on VolumeSnapshot resources. cnpg.io/cluster Name of the cluster. cnpg.io/immediateBackup Applied to a Backup resource if the backup is the first one created from a ScheduledBackup object having immediate set to true . cnpg.io/instanceName Name of the PostgreSQL instance (replaces the old and deprecated postgresql label). cnpg.io/jobRole Role of the job (that is, import , initdb , join , ...) cnpg.io/onlineBackup Whether the backup is online (hot) or taken when Postgres is down (cold). This label is available only on VolumeSnapshot resources. cnpg.io/podRole Distinguishes pods dedicated to pooler deployment from those used for database instances. cnpg.io/poolerName Name of the PgBouncer pooler. cnpg.io/pvcRole Purpose of the PVC, such as PG_DATA or PG_WAL . cnpg.io/reload Available on ConfigMap and Secret resources. When set to true , a change in the resource is automatically reloaded by the operator. cnpg.io/userType Specifies the type of PostgreSQL user associated with the Secret , either superuser (Postgres superuser access) or app (application-level user in CloudNativePG terminology), and is limited to the default users created by CloudNativePG (typically postgres and app ). role - deprecated Whether the instance running in a pod is a primary or a replica . This label is deprecated, you should use cnpg.io/instanceRole instead. cnpg.io/scheduled-backup When available, name of the ScheduledBackup resource that created a given Backup object. cnpg.io/instanceRole Whether the instance running in a pod is a primary or a replica . Predefined annotations CloudNativePG manages the following predefined annotations: container.apparmor.security.beta.kubernetes.io/* Name of the AppArmor profile to apply to the named container. See AppArmor for details. cnpg.io/backupEndTime The time a backup ended. This annotation is available only on VolumeSnapshot resources. cnpg.io/backupEndWAL The WAL at the conclusion of a backup. This annotation is available only on VolumeSnapshot resources. cnpg.io/backupStartTime The time a backup started. cnpg.io/backupStartWAL The WAL at the start of a backup. This annotation is available only on VolumeSnapshot resources. cnpg.io/coredumpFilter Filter to control the coredump of Postgres processes, expressed with a bitmask. By default it's set to 0x31 to exclude shared memory segments from the dump. See PostgreSQL core dumps for more information. cnpg.io/clusterManifest Manifest of the Cluster owning this resource (such as a PVC). This label replaces the old, deprecated cnpg.io/hibernateClusterManifest label. cnpg.io/fencedInstances List of the instances that need to be fenced, expressed in JSON format. The whole cluster is fenced if the list contains the * element. cnpg.io/forceLegacyBackup Applied to a Cluster resource for testing purposes only, to simulate the behavior of barman-cloud-backup prior to version 3.4 (Jan 2023) when the --name option wasn't available. cnpg.io/hash The hash value of the resource. cnpg.io/hibernation Applied to a Cluster resource to control the declarative hibernation feature . Allowed values are on and off . cnpg.io/managedSecrets Pull secrets managed by the operator and automatically set in the ServiceAccount resources for each Postgres cluster. cnpg.io/nodeSerial On a pod resource, identifies the serial number of the instance within the Postgres cluster. cnpg.io/operatorVersion Version of the operator. cnpg.io/pgControldata Output of the pg_controldata command. This annotation replaces the old, deprecated cnpg.io/hibernatePgControlData annotation. cnpg.io/podEnvHash Deprecated, as the cnpg.io/podSpec annotation now also contains the pod environment. cnpg.io/podSpec Snapshot of the spec of the pod generated by the operator. This annotation replaces the old, deprecated cnpg.io/podEnvHash annotation. cnpg.io/poolerSpecHash Hash of the pooler resource. cnpg.io/pvcStatus Current status of the PVC: initializing , ready , or detached . cnpg.io/reconcilePodSpec Annotation can be applied to a Cluster or Pooler to prevent restarts. When set to disabled on a Cluster , the operator prevents instances from restarting due to changes in the PodSpec. This includes changes to: Topology or affinity Scheduler Volumes or containers When set to disabled on a Pooler , the operator restricts any modifications to the deployment specification, except for changes to spec.instances . cnpg.io/reconciliationLoop When set to disabled on a Cluster , the operator prevents the reconciliation loop from running. cnpg.io/reloadedAt Contains the latest cluster reload time. reload is triggered by the user through a plugin. cnpg.io/skipEmptyWalArchiveCheck When set to enabled on a Cluster resource, the operator disables the check that ensures that the WAL archive is empty before writing data. Use at your own risk. cnpg.io/skipWalArchiving When set to enabled on a Cluster resource, the operator disables WAL archiving. This will set archive_mode to off and require a restart of all PostgreSQL instances. Use at your own risk. cnpg.io/snapshotStartTime The time a snapshot started. cnpg.io/snapshotEndTime The time a snapshot was marked as ready to use. cnpg.io/validation When set to disabled on a CloudNativePG-managed custom resource, the validation webhook allows all changes without restriction. \u26a0\ufe0f WARNING: Disabling validation may permit unsafe or destructive operations. Use this setting with caution and at your own risk. cnpg.io/volumeSnapshotDeadline Applied to Backup and ScheduledBackup resources, allows you to control how long the operator should retry recoverable errors before considering the volume snapshot backup failed. In minutes, defaulting to 10. kubectl.kubernetes.io/restartedAt When available, the time of last requested restart of a Postgres cluster. Prerequisites By default, no label or annotation defined in the cluster's metadata is inherited by the associated resources. To enable label/annotation inheritance, follow the instructions provided in Operator configuration . The following continues from that example and limits it to the following: Annotations: categories Labels: app , environment , and workload Note Feel free to select the names that most suit your context for both annotations and labels. You can also use wildcards in naming and adopt strategies like using mycompany/* for all labels or setting annotations starting with mycompany/ to be inherited. Defining cluster's metadata When defining the cluster, before any resource is deployed, you can set the metadata as follows: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example annotations: categories: database labels: environment: production workload: database app: sso spec: # ... Once the cluster is deployed, you can verify, for example, that the labels were correctly set in the pods: kubectl get pods --show-labels Current limitations Currently, CloudNativePG doesn't automatically propagate labels or annotations deletions. Therefore, when an annotation or label is removed from a cluster that was previously propagated to the underlying pods, the operator doesn't remove it on the associated resources.","title":"Labels and annotations"},{"location":"labels_annotations/#labels-and-annotations","text":"Resources in Kubernetes are organized in a flat structure, with no hierarchical information or relationship between them. However, such resources and objects can be linked together and put in relationship through labels and annotations . Info For more information, see the Kubernetes documentation on annotations and labels . In brief: An annotation is used to assign additional non-identifying information to resources with the goal of facilitating integration with external tools. A label is used to group objects and query them through the Kubernetes native selector capability. You can select one or more labels or annotations to use in your CloudNativePG deployments. Then you need to configure the operator so that when you define these labels or annotations in a cluster's metadata, they're inherited by all resources created by it (including pods). Note Label and annotation inheritance is the technique adopted by CloudNativePG instead of alternative approaches such as pod templates.","title":"Labels and annotations"},{"location":"labels_annotations/#predefined-labels","text":"CloudNativePG manages the following predefined labels: cnpg.io/backupDate The date of the backup in ISO 8601 format ( YYYYMMDD ). This label is available only on VolumeSnapshot resources. cnpg.io/backupName Backup identifier. This label is available only on VolumeSnapshot resources. cnpg.io/backupMonth The year/month when a backup was taken. This label is available only on VolumeSnapshot resources. cnpg.io/backupTimeline The timeline of the instance when a backup was taken. This label is available only on VolumeSnapshot resources. cnpg.io/backupYear The year a backup was taken. This label is available only on VolumeSnapshot resources. cnpg.io/cluster Name of the cluster. cnpg.io/immediateBackup Applied to a Backup resource if the backup is the first one created from a ScheduledBackup object having immediate set to true . cnpg.io/instanceName Name of the PostgreSQL instance (replaces the old and deprecated postgresql label). cnpg.io/jobRole Role of the job (that is, import , initdb , join , ...) cnpg.io/onlineBackup Whether the backup is online (hot) or taken when Postgres is down (cold). This label is available only on VolumeSnapshot resources. cnpg.io/podRole Distinguishes pods dedicated to pooler deployment from those used for database instances. cnpg.io/poolerName Name of the PgBouncer pooler. cnpg.io/pvcRole Purpose of the PVC, such as PG_DATA or PG_WAL . cnpg.io/reload Available on ConfigMap and Secret resources. When set to true , a change in the resource is automatically reloaded by the operator. cnpg.io/userType Specifies the type of PostgreSQL user associated with the Secret , either superuser (Postgres superuser access) or app (application-level user in CloudNativePG terminology), and is limited to the default users created by CloudNativePG (typically postgres and app ). role - deprecated Whether the instance running in a pod is a primary or a replica . This label is deprecated, you should use cnpg.io/instanceRole instead. cnpg.io/scheduled-backup When available, name of the ScheduledBackup resource that created a given Backup object. cnpg.io/instanceRole Whether the instance running in a pod is a primary or a replica .","title":"Predefined labels"},{"location":"labels_annotations/#predefined-annotations","text":"CloudNativePG manages the following predefined annotations: container.apparmor.security.beta.kubernetes.io/* Name of the AppArmor profile to apply to the named container. See AppArmor for details. cnpg.io/backupEndTime The time a backup ended. This annotation is available only on VolumeSnapshot resources. cnpg.io/backupEndWAL The WAL at the conclusion of a backup. This annotation is available only on VolumeSnapshot resources. cnpg.io/backupStartTime The time a backup started. cnpg.io/backupStartWAL The WAL at the start of a backup. This annotation is available only on VolumeSnapshot resources. cnpg.io/coredumpFilter Filter to control the coredump of Postgres processes, expressed with a bitmask. By default it's set to 0x31 to exclude shared memory segments from the dump. See PostgreSQL core dumps for more information. cnpg.io/clusterManifest Manifest of the Cluster owning this resource (such as a PVC). This label replaces the old, deprecated cnpg.io/hibernateClusterManifest label. cnpg.io/fencedInstances List of the instances that need to be fenced, expressed in JSON format. The whole cluster is fenced if the list contains the * element. cnpg.io/forceLegacyBackup Applied to a Cluster resource for testing purposes only, to simulate the behavior of barman-cloud-backup prior to version 3.4 (Jan 2023) when the --name option wasn't available. cnpg.io/hash The hash value of the resource. cnpg.io/hibernation Applied to a Cluster resource to control the declarative hibernation feature . Allowed values are on and off . cnpg.io/managedSecrets Pull secrets managed by the operator and automatically set in the ServiceAccount resources for each Postgres cluster. cnpg.io/nodeSerial On a pod resource, identifies the serial number of the instance within the Postgres cluster. cnpg.io/operatorVersion Version of the operator. cnpg.io/pgControldata Output of the pg_controldata command. This annotation replaces the old, deprecated cnpg.io/hibernatePgControlData annotation. cnpg.io/podEnvHash Deprecated, as the cnpg.io/podSpec annotation now also contains the pod environment. cnpg.io/podSpec Snapshot of the spec of the pod generated by the operator. This annotation replaces the old, deprecated cnpg.io/podEnvHash annotation. cnpg.io/poolerSpecHash Hash of the pooler resource. cnpg.io/pvcStatus Current status of the PVC: initializing , ready , or detached . cnpg.io/reconcilePodSpec Annotation can be applied to a Cluster or Pooler to prevent restarts. When set to disabled on a Cluster , the operator prevents instances from restarting due to changes in the PodSpec. This includes changes to: Topology or affinity Scheduler Volumes or containers When set to disabled on a Pooler , the operator restricts any modifications to the deployment specification, except for changes to spec.instances . cnpg.io/reconciliationLoop When set to disabled on a Cluster , the operator prevents the reconciliation loop from running. cnpg.io/reloadedAt Contains the latest cluster reload time. reload is triggered by the user through a plugin. cnpg.io/skipEmptyWalArchiveCheck When set to enabled on a Cluster resource, the operator disables the check that ensures that the WAL archive is empty before writing data. Use at your own risk. cnpg.io/skipWalArchiving When set to enabled on a Cluster resource, the operator disables WAL archiving. This will set archive_mode to off and require a restart of all PostgreSQL instances. Use at your own risk. cnpg.io/snapshotStartTime The time a snapshot started. cnpg.io/snapshotEndTime The time a snapshot was marked as ready to use. cnpg.io/validation When set to disabled on a CloudNativePG-managed custom resource, the validation webhook allows all changes without restriction. \u26a0\ufe0f WARNING: Disabling validation may permit unsafe or destructive operations. Use this setting with caution and at your own risk. cnpg.io/volumeSnapshotDeadline Applied to Backup and ScheduledBackup resources, allows you to control how long the operator should retry recoverable errors before considering the volume snapshot backup failed. In minutes, defaulting to 10. kubectl.kubernetes.io/restartedAt When available, the time of last requested restart of a Postgres cluster.","title":"Predefined annotations"},{"location":"labels_annotations/#prerequisites","text":"By default, no label or annotation defined in the cluster's metadata is inherited by the associated resources. To enable label/annotation inheritance, follow the instructions provided in Operator configuration . The following continues from that example and limits it to the following: Annotations: categories Labels: app , environment , and workload Note Feel free to select the names that most suit your context for both annotations and labels. You can also use wildcards in naming and adopt strategies like using mycompany/* for all labels or setting annotations starting with mycompany/ to be inherited.","title":"Prerequisites"},{"location":"labels_annotations/#defining-clusters-metadata","text":"When defining the cluster, before any resource is deployed, you can set the metadata as follows: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example annotations: categories: database labels: environment: production workload: database app: sso spec: # ... Once the cluster is deployed, you can verify, for example, that the labels were correctly set in the pods: kubectl get pods --show-labels","title":"Defining cluster's metadata"},{"location":"labels_annotations/#current-limitations","text":"Currently, CloudNativePG doesn't automatically propagate labels or annotations deletions. Therefore, when an annotation or label is removed from a cluster that was previously propagated to the underlying pods, the operator doesn't remove it on the associated resources.","title":"Current limitations"},{"location":"logging/","text":"Logging CloudNativePG outputs logs in JSON format directly to standard output, including PostgreSQL logs, without persisting them to storage for security reasons. This design facilitates seamless integration with most Kubernetes-compatible log management tools, including command line ones like stern . Important Long-term storage and management of logs are outside the scope of the operator and should be handled at the Kubernetes infrastructure level. For more information, see the Kubernetes Logging Architecture documentation. Each log entry includes the following fields: level \u2013 The log level (e.g., info , notice ). ts \u2013 The timestamp. logger \u2013 The type of log (e.g., postgres , pg_controldata ). msg \u2013 The log message, or the keyword record if the message is in JSON format. record \u2013 The actual record, with a structure that varies depending on the logger type. logging_pod \u2013 The name of the pod where the log was generated. Info If your log ingestion system requires custom field names, you can rename the level and ts fields using the log-field-level and log-field-timestamp flags in the operator controller. This can be configured by editing the Deployment definition of the cloudnative-pg operator. Cluster Logs You can configure the log level for the instance pods in the cluster specification using the logLevel option. Available log levels are: error , warning , info (default), debug , and trace . Important Currently, the log level can only be set at the time the instance starts. Changes to the log level in the cluster specification after the cluster has started will only apply to new pods, not existing ones. Operator Logs The logs produced by the operator pod can be configured with log levels, same as instance pods: error , warning , info (default), debug , and trace . The log level for the operator can be configured by editing the Deployment definition of the operator and setting the --log-level command line argument to the desired value. PostgreSQL Logs Each PostgreSQL log entry is a JSON object with the logger key set to postgres . The structure of the log entries is as follows: { \"level\": \"info\", \"ts\": 1619781249.7188137, \"logger\": \"postgres\", \"msg\": \"record\", \"record\": { \"log_time\": \"2021-04-30 11:14:09.718 UTC\", \"user_name\": \"\", \"database_name\": \"\", \"process_id\": \"25\", \"connection_from\": \"\", \"session_id\": \"608be681.19\", \"session_line_num\": \"1\", \"command_tag\": \"\", \"session_start_time\": \"2021-04-30 11:14:09 UTC\", \"virtual_transaction_id\": \"\", \"transaction_id\": \"0\", \"error_severity\": \"LOG\", \"sql_state_code\": \"00000\", \"message\": \"database system was interrupted; last known up at 2021-04-30 11:14:07 UTC\", \"detail\": \"\", \"hint\": \"\", \"internal_query\": \"\", \"internal_query_pos\": \"\", \"context\": \"\", \"query\": \"\", \"query_pos\": \"\", \"location\": \"\", \"application_name\": \"\", \"backend_type\": \"startup\" }, \"logging_pod\": \"cluster-example-1\", } Info Internally, the operator uses PostgreSQL's CSV log format. For more details, refer to the PostgreSQL documentation on CSV log format . PGAudit Logs CloudNativePG offers seamless and native support for PGAudit on PostgreSQL clusters. To enable PGAudit, add the necessary pgaudit parameters in the postgresql section of the cluster configuration. Important The PGAudit library must be added to shared_preload_libraries . CloudNativePG automatically manages this based on the presence of pgaudit.* parameters in the PostgreSQL configuration. The operator handles both the addition and removal of the library from shared_preload_libraries . Additionally, the operator manages the creation and removal of the PGAudit extension across all databases within the cluster. Important CloudNativePG executes the CREATE EXTENSION and DROP EXTENSION commands in all databases within the cluster that accept connections. The following example demonstrates a PostgreSQL Cluster deployment with PGAudit enabled and configured: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 3 postgresql: parameters: \"pgaudit.log\": \"all, -misc\" \"pgaudit.log_catalog\": \"off\" \"pgaudit.log_parameter\": \"on\" \"pgaudit.log_relation\": \"on\" storage: size: 1Gi The audit CSV log entries generated by PGAudit are parsed and routed to standard output in JSON format, similar to all other logs: .logger is set to pgaudit . .msg is set to record . .record contains the entire parsed record as a JSON object. This structure resembles that of logging_collector logs, with the exception of .record.audit , which contains the PGAudit CSV message formatted as a JSON object. This example shows sample log entries: { \"level\": \"info\", \"ts\": 1627394507.8814096, \"logger\": \"pgaudit\", \"msg\": \"record\", \"record\": { \"log_time\": \"2021-07-27 14:01:47.881 UTC\", \"user_name\": \"postgres\", \"database_name\": \"postgres\", \"process_id\": \"203\", \"connection_from\": \"[local]\", \"session_id\": \"610011cb.cb\", \"session_line_num\": \"1\", \"command_tag\": \"SELECT\", \"session_start_time\": \"2021-07-27 14:01:47 UTC\", \"virtual_transaction_id\": \"3/336\", \"transaction_id\": \"0\", \"error_severity\": \"LOG\", \"sql_state_code\": \"00000\", \"backend_type\": \"client backend\", \"audit\": { \"audit_type\": \"SESSION\", \"statement_id\": \"1\", \"substatement_id\": \"1\", \"class\": \"READ\", \"command\": \"SELECT FOR KEY SHARE\", \"statement\": \"SELECT pg_current_wal_lsn()\", \"parameter\": \"\" } }, \"logging_pod\": \"cluster-example-1\", } See the PGAudit documentation for more details about each field in a record. Other Logs All logs generated by the operator and its instances are in JSON format, with the logger field indicating the process that produced them. The possible logger values are as follows: barman-cloud-wal-archive : logs from barman-cloud-wal-archive barman-cloud-wal-restore : logs from barman-cloud-wal-restore initdb : logs from running initdb pg_basebackup : logs from running pg_basebackup pg_controldata : logs from running pg_controldata pg_ctl : logs from running any pg_ctl subcommand pg_rewind : logs from running pg_rewind pgaudit : logs from the PGAudit extension postgres : logs from the postgres instance (with msg distinct from record ) wal-archive : logs from the wal-archive subcommand of the instance manager wal-restore : logs from the wal-restore subcommand of the instance manager instance-manager : from the PostgreSQL instance manager With the exception of postgres , which follows a specific structure, all other logger values contain the msg field with the escaped message that is logged.","title":"Logging"},{"location":"logging/#logging","text":"CloudNativePG outputs logs in JSON format directly to standard output, including PostgreSQL logs, without persisting them to storage for security reasons. This design facilitates seamless integration with most Kubernetes-compatible log management tools, including command line ones like stern . Important Long-term storage and management of logs are outside the scope of the operator and should be handled at the Kubernetes infrastructure level. For more information, see the Kubernetes Logging Architecture documentation. Each log entry includes the following fields: level \u2013 The log level (e.g., info , notice ). ts \u2013 The timestamp. logger \u2013 The type of log (e.g., postgres , pg_controldata ). msg \u2013 The log message, or the keyword record if the message is in JSON format. record \u2013 The actual record, with a structure that varies depending on the logger type. logging_pod \u2013 The name of the pod where the log was generated. Info If your log ingestion system requires custom field names, you can rename the level and ts fields using the log-field-level and log-field-timestamp flags in the operator controller. This can be configured by editing the Deployment definition of the cloudnative-pg operator.","title":"Logging"},{"location":"logging/#cluster-logs","text":"You can configure the log level for the instance pods in the cluster specification using the logLevel option. Available log levels are: error , warning , info (default), debug , and trace . Important Currently, the log level can only be set at the time the instance starts. Changes to the log level in the cluster specification after the cluster has started will only apply to new pods, not existing ones.","title":"Cluster Logs"},{"location":"logging/#operator-logs","text":"The logs produced by the operator pod can be configured with log levels, same as instance pods: error , warning , info (default), debug , and trace . The log level for the operator can be configured by editing the Deployment definition of the operator and setting the --log-level command line argument to the desired value.","title":"Operator Logs"},{"location":"logging/#postgresql-logs","text":"Each PostgreSQL log entry is a JSON object with the logger key set to postgres . The structure of the log entries is as follows: { \"level\": \"info\", \"ts\": 1619781249.7188137, \"logger\": \"postgres\", \"msg\": \"record\", \"record\": { \"log_time\": \"2021-04-30 11:14:09.718 UTC\", \"user_name\": \"\", \"database_name\": \"\", \"process_id\": \"25\", \"connection_from\": \"\", \"session_id\": \"608be681.19\", \"session_line_num\": \"1\", \"command_tag\": \"\", \"session_start_time\": \"2021-04-30 11:14:09 UTC\", \"virtual_transaction_id\": \"\", \"transaction_id\": \"0\", \"error_severity\": \"LOG\", \"sql_state_code\": \"00000\", \"message\": \"database system was interrupted; last known up at 2021-04-30 11:14:07 UTC\", \"detail\": \"\", \"hint\": \"\", \"internal_query\": \"\", \"internal_query_pos\": \"\", \"context\": \"\", \"query\": \"\", \"query_pos\": \"\", \"location\": \"\", \"application_name\": \"\", \"backend_type\": \"startup\" }, \"logging_pod\": \"cluster-example-1\", } Info Internally, the operator uses PostgreSQL's CSV log format. For more details, refer to the PostgreSQL documentation on CSV log format .","title":"PostgreSQL Logs"},{"location":"logging/#pgaudit-logs","text":"CloudNativePG offers seamless and native support for PGAudit on PostgreSQL clusters. To enable PGAudit, add the necessary pgaudit parameters in the postgresql section of the cluster configuration. Important The PGAudit library must be added to shared_preload_libraries . CloudNativePG automatically manages this based on the presence of pgaudit.* parameters in the PostgreSQL configuration. The operator handles both the addition and removal of the library from shared_preload_libraries . Additionally, the operator manages the creation and removal of the PGAudit extension across all databases within the cluster. Important CloudNativePG executes the CREATE EXTENSION and DROP EXTENSION commands in all databases within the cluster that accept connections. The following example demonstrates a PostgreSQL Cluster deployment with PGAudit enabled and configured: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example spec: instances: 3 postgresql: parameters: \"pgaudit.log\": \"all, -misc\" \"pgaudit.log_catalog\": \"off\" \"pgaudit.log_parameter\": \"on\" \"pgaudit.log_relation\": \"on\" storage: size: 1Gi The audit CSV log entries generated by PGAudit are parsed and routed to standard output in JSON format, similar to all other logs: .logger is set to pgaudit . .msg is set to record . .record contains the entire parsed record as a JSON object. This structure resembles that of logging_collector logs, with the exception of .record.audit , which contains the PGAudit CSV message formatted as a JSON object. This example shows sample log entries: { \"level\": \"info\", \"ts\": 1627394507.8814096, \"logger\": \"pgaudit\", \"msg\": \"record\", \"record\": { \"log_time\": \"2021-07-27 14:01:47.881 UTC\", \"user_name\": \"postgres\", \"database_name\": \"postgres\", \"process_id\": \"203\", \"connection_from\": \"[local]\", \"session_id\": \"610011cb.cb\", \"session_line_num\": \"1\", \"command_tag\": \"SELECT\", \"session_start_time\": \"2021-07-27 14:01:47 UTC\", \"virtual_transaction_id\": \"3/336\", \"transaction_id\": \"0\", \"error_severity\": \"LOG\", \"sql_state_code\": \"00000\", \"backend_type\": \"client backend\", \"audit\": { \"audit_type\": \"SESSION\", \"statement_id\": \"1\", \"substatement_id\": \"1\", \"class\": \"READ\", \"command\": \"SELECT FOR KEY SHARE\", \"statement\": \"SELECT pg_current_wal_lsn()\", \"parameter\": \"\" } }, \"logging_pod\": \"cluster-example-1\", } See the PGAudit documentation for more details about each field in a record.","title":"PGAudit Logs"},{"location":"logging/#other-logs","text":"All logs generated by the operator and its instances are in JSON format, with the logger field indicating the process that produced them. The possible logger values are as follows: barman-cloud-wal-archive : logs from barman-cloud-wal-archive barman-cloud-wal-restore : logs from barman-cloud-wal-restore initdb : logs from running initdb pg_basebackup : logs from running pg_basebackup pg_controldata : logs from running pg_controldata pg_ctl : logs from running any pg_ctl subcommand pg_rewind : logs from running pg_rewind pgaudit : logs from the PGAudit extension postgres : logs from the postgres instance (with msg distinct from record ) wal-archive : logs from the wal-archive subcommand of the instance manager wal-restore : logs from the wal-restore subcommand of the instance manager instance-manager : from the PostgreSQL instance manager With the exception of postgres , which follows a specific structure, all other logger values contain the msg field with the escaped message that is logged.","title":"Other Logs"},{"location":"logical_replication/","text":"Logical Replication PostgreSQL extends its replication capabilities beyond physical replication, which operates at the level of exact block addresses and byte-by-byte copying, by offering logical replication . Logical replication replicates data objects and their changes based on a defined replication identity, typically the primary key. Logical replication uses a publish-and-subscribe model, where subscribers connect to publications on a publisher node. Subscribers pull data changes from these publications and can re-publish them, enabling cascading replication and complex topologies. This flexible model is particularly useful for: Online data migrations Live PostgreSQL version upgrades Data distribution across systems Real-time analytics Integration with external applications Info For more details, examples, and limitations, please refer to the official PostgreSQL documentation on Logical Replication . CloudNativePG enhances this capability by providing declarative support for key PostgreSQL logical replication objects: Publications via the Publication resource Subscriptions via the Subscription resource Publications In PostgreSQL's publish-and-subscribe replication model, a publication is the source of data changes. It acts as a logical container for the change sets (also known as replication sets ) generated from one or more tables within a database. Publications can be defined on any PostgreSQL 10+ instance acting as the publisher , including instances managed by popular DBaaS solutions in the public cloud. Each publication is tied to a single database and provides fine-grained control over which tables and changes are replicated. For publishers outside Kubernetes, you can create publications using SQL or leverage the cnpg publication create plugin command . When managing Cluster objects with CloudNativePG , PostgreSQL publications can be defined declaratively through the Publication resource. Info Please refer to the API reference for the full list of attributes you can define for each Publication object. Suppose you have a cluster named freddie and want to replicate all tables in the app database. Here's a Publication manifest: apiVersion: postgresql.cnpg.io/v1 kind: Publication metadata: name: freddie-publisher spec: cluster: name: freddie dbname: app name: publisher target: allTables: true In the above example: The publication object is named freddie-publisher ( metadata.name ). The publication is created via the primary of the freddie cluster ( spec.cluster.name ) with name publisher ( spec.name ). It includes all tables ( spec.target.allTables: true ) from the app database ( spec.dbname ). Important While allTables simplifies configuration, PostgreSQL offers fine-grained control for replicating specific tables or targeted data changes. For advanced configurations, consult the PostgreSQL documentation . Additionally, refer to the CloudNativePG API reference for details on declaratively customizing replication targets. Required Fields in the Publication Manifest The following fields are required for a Publication object: metadata.name : Unique name for the Kubernetes Publication object. spec.cluster.name : Name of the PostgreSQL cluster. spec.dbname : Database name where the publication is created. spec.name : Publication name in PostgreSQL. spec.target : Specifies the tables or changes to include in the publication. The Publication object must reference a specific Cluster , determining where the publication will be created. It is managed by the cluster's primary instance, ensuring the publication is created or updated as needed. Reconciliation and Status After creating a Publication , CloudNativePG manages it on the primary instance of the specified cluster. Following a successful reconciliation cycle, the Publication status will reflect the following: applied: true , indicates the configuration has been successfully applied. observedGeneration matches metadata.generation , confirming the applied configuration corresponds to the most recent changes. If an error occurs during reconciliation, status.applied will be false , and an error message will be included in the status.message field. Removing a publication The publicationReclaimPolicy field controls the behavior when deleting a Publication object: retain (default): Leaves the publication in PostgreSQL for manual management. delete : Automatically removes the publication from PostgreSQL. Consider the following example: apiVersion: postgresql.cnpg.io/v1 kind: Publication metadata: name: freddie-publisher spec: cluster: name: freddie dbname: app name: publisher target: allTables: true publicationReclaimPolicy: delete In this case, deleting the Publication object also removes the publisher publication from the app database of the freddie cluster. Subscriptions In PostgreSQL's publish-and-subscribe replication model, a subscription represents the downstream component that consumes data changes. A subscription establishes the connection to a publisher's database and specifies the set of publications (one or more) it subscribes to. Subscriptions can be created on any supported PostgreSQL instance acting as the subscriber . Important Since schema definitions are not replicated, the subscriber must have the corresponding tables already defined before data replication begins. CloudNativePG simplifies subscription management by enabling you to define them declaratively using the Subscription resource. Info Please refer to the API reference for the full list of attributes you can define for each Subscription object. Suppose you want to replicate changes from the publisher publication on the app database of the freddie cluster ( publisher ) to the app database of the king cluster ( subscriber ). Here's an example of a Subscription manifest: apiVersion: postgresql.cnpg.io/v1 kind: Subscription metadata: name: freddie-to-king-subscription spec: cluster: name: king dbname: app name: subscriber externalClusterName: freddie publicationName: publisher In the above example: The subscription object is named freddie-to-king-subscriber ( metadata.name ). The subscription is created in the app database ( spec.dbname ) of the king cluster ( spec.cluster.name ), with name subscriber ( spec.name ). It connects to the publisher publication in the external freddie cluster, referenced by spec.externalClusterName . To facilitate this setup, the freddie external cluster must be defined in the king cluster's configuration. Below is an example excerpt showing how to define the external cluster in the king manifest: externalClusters: - name: freddie connectionParameters: host: freddie-rw.default.svc user: postgres dbname: app Info For more details on configuring the externalClusters section, see the \"Bootstrap\" section of the documentation. As you can see, a subscription can connect to any PostgreSQL database accessible over the network. This flexibility allows you to seamlessly migrate your data into Kubernetes with nearly zero downtime. It\u2019s an excellent option for transitioning from various environments, including popular cloud-based Database-as-a-Service (DBaaS) platforms. Required Fields in the Subscription Manifest The following fields are mandatory for defining a Subscription object: metadata.name : A unique name for the Kubernetes Subscription object within its namespace. spec.cluster.name : The name of the PostgreSQL cluster where the subscription will be created. spec.dbname : The name of the database in which the subscription will be created. spec.name : The name of the subscription as it will appear in PostgreSQL. spec.externalClusterName : The name of the external cluster, as defined in the spec.cluster.name cluster's configuration. This references the publisher database. spec.publicationName : The name of the publication in the publisher database to which the subscription will connect. The Subscription object must reference a specific Cluster , determining where the subscription will be managed. CloudNativePG ensures that the subscription is created or updated on the primary instance of the specified cluster. Reconciliation and Status After creating a Subscription , CloudNativePG manages it on the primary instance of the specified cluster. Following a successful reconciliation cycle, the Subscription status will reflect the following: applied: true , indicates the configuration has been successfully applied. observedGeneration matches metadata.generation , confirming the applied configuration corresponds to the most recent changes. If an error occurs during reconciliation, status.applied will be false , and an error message will be included in the status.message field. Removing a subscription The subscriptionReclaimPolicy field controls the behavior when deleting a Subscription object: retain (default): Leaves the subscription in PostgreSQL for manual management. delete : Automatically removes the subscription from PostgreSQL. Consider the following example: apiVersion: postgresql.cnpg.io/v1 kind: Subscription metadata: name: freddie-to-king-subscription spec: cluster: name: king dbname: app name: subscriber externalClusterName: freddie publicationName: publisher subscriptionReclaimPolicy: delete In this case, deleting the Subscription object also removes the subscriber subscription from the app database of the king cluster. Limitations Logical replication in PostgreSQL has some inherent limitations, as outlined in the official documentation . Notably, the following objects are not replicated: Database schema and DDL commands Sequence data Large objects Addressing Schema Replication The first limitation, related to schema replication, can be easily addressed using CloudNativePG's capabilities. For instance, you can leverage the import bootstrap feature to copy the schema of the tables you need to replicate. Alternatively, you can manually create the schema as you would for any PostgreSQL database. Handling Sequences While sequences are not automatically kept in sync through logical replication, CloudNativePG provides a solution to be used in live migrations. You can use the cnpg plugin to synchronize sequence values, ensuring consistency between the publisher and subscriber databases. Example of live migration and major Postgres upgrade with logical replication To highlight the powerful capabilities of logical replication, this example demonstrates how to replicate data from a publisher database ( freddie ) running PostgreSQL 16 to a subscriber database ( king ) running the latest PostgreSQL version. This setup can be deployed in your Kubernetes cluster for evaluation and hands-on learning. This example illustrates how logical replication facilitates live migrations and upgrades between PostgreSQL versions while ensuring data consistency. By combining logical replication with CloudNativePG, you can easily set up, manage, and evaluate such scenarios in a Kubernetes environment. Step 1: Setting Up the Publisher ( freddie ) The first step involves creating a freddie PostgreSQL cluster with version 16. The cluster contains a single instance and includes an app database initialized with a table, n , storing 10,000 numbers. A logical replication publication named publisher is also configured to include all tables in the database. Here\u2019s the manifest for setting up the freddie cluster and its publication resource: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: freddie spec: instances: 1 imageName: ghcr.io/cloudnative-pg/postgresql:16 storage: size: 1Gi bootstrap: initdb: postInitApplicationSQL: - CREATE TABLE n (i SERIAL PRIMARY KEY, m INTEGER) - INSERT INTO n (m) (SELECT generate_series(1, 10000)) - ALTER TABLE n OWNER TO app managed: roles: - name: app login: true replication: true --- apiVersion: postgresql.cnpg.io/v1 kind: Publication metadata: name: freddie-publisher spec: cluster: name: freddie dbname: app name: publisher target: allTables: true Step 2: Setting Up the Subscriber ( king ) Next, create the king PostgreSQL cluster, running the latest version of PostgreSQL. This cluster initializes by importing the schema from the app database on the freddie cluster using the external cluster configuration. A Subscription resource, freddie-to-king-subscription , is then configured to consume changes published by the publisher on freddie . Below is the manifest for setting up the king cluster and its subscription: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: king spec: instances: 1 storage: size: 1Gi bootstrap: initdb: import: type: microservice schemaOnly: true databases: - app source: externalCluster: freddie externalClusters: - name: freddie connectionParameters: host: freddie-rw.default.svc user: app dbname: app password: name: freddie-app key: password --- apiVersion: postgresql.cnpg.io/v1 kind: Subscription metadata: name: freddie-to-king-subscription spec: cluster: name: king dbname: app name: subscriber externalClusterName: freddie publicationName: publisher Once the king cluster is running, you can verify that the replication is working by connecting to the app database and counting the records in the n table. The following example uses the psql command provided by the cnpg plugin for simplicity: kubectl cnpg psql king -- app -qAt -c 'SELECT count(*) FROM n' 10000 This command should return 10000 , confirming that the data from the freddie cluster has been successfully replicated to the king cluster. Using the cnpg plugin, you can also synchronize existing sequences to ensure consistency between the publisher and subscriber. The example below demonstrates how to synchronize a sequence for the king cluster: kubectl cnpg subscription sync-sequences king --subscription=subscriber SELECT setval('\"public\".\"n_i_seq\"', 10000); 10000 This command updates the sequence n_i_seq in the king cluster to match the current value, ensuring it is in sync with the source database.","title":"Logical Replication"},{"location":"logical_replication/#logical-replication","text":"PostgreSQL extends its replication capabilities beyond physical replication, which operates at the level of exact block addresses and byte-by-byte copying, by offering logical replication . Logical replication replicates data objects and their changes based on a defined replication identity, typically the primary key. Logical replication uses a publish-and-subscribe model, where subscribers connect to publications on a publisher node. Subscribers pull data changes from these publications and can re-publish them, enabling cascading replication and complex topologies. This flexible model is particularly useful for: Online data migrations Live PostgreSQL version upgrades Data distribution across systems Real-time analytics Integration with external applications Info For more details, examples, and limitations, please refer to the official PostgreSQL documentation on Logical Replication . CloudNativePG enhances this capability by providing declarative support for key PostgreSQL logical replication objects: Publications via the Publication resource Subscriptions via the Subscription resource","title":"Logical Replication"},{"location":"logical_replication/#publications","text":"In PostgreSQL's publish-and-subscribe replication model, a publication is the source of data changes. It acts as a logical container for the change sets (also known as replication sets ) generated from one or more tables within a database. Publications can be defined on any PostgreSQL 10+ instance acting as the publisher , including instances managed by popular DBaaS solutions in the public cloud. Each publication is tied to a single database and provides fine-grained control over which tables and changes are replicated. For publishers outside Kubernetes, you can create publications using SQL or leverage the cnpg publication create plugin command . When managing Cluster objects with CloudNativePG , PostgreSQL publications can be defined declaratively through the Publication resource. Info Please refer to the API reference for the full list of attributes you can define for each Publication object. Suppose you have a cluster named freddie and want to replicate all tables in the app database. Here's a Publication manifest: apiVersion: postgresql.cnpg.io/v1 kind: Publication metadata: name: freddie-publisher spec: cluster: name: freddie dbname: app name: publisher target: allTables: true In the above example: The publication object is named freddie-publisher ( metadata.name ). The publication is created via the primary of the freddie cluster ( spec.cluster.name ) with name publisher ( spec.name ). It includes all tables ( spec.target.allTables: true ) from the app database ( spec.dbname ). Important While allTables simplifies configuration, PostgreSQL offers fine-grained control for replicating specific tables or targeted data changes. For advanced configurations, consult the PostgreSQL documentation . Additionally, refer to the CloudNativePG API reference for details on declaratively customizing replication targets.","title":"Publications"},{"location":"logical_replication/#required-fields-in-the-publication-manifest","text":"The following fields are required for a Publication object: metadata.name : Unique name for the Kubernetes Publication object. spec.cluster.name : Name of the PostgreSQL cluster. spec.dbname : Database name where the publication is created. spec.name : Publication name in PostgreSQL. spec.target : Specifies the tables or changes to include in the publication. The Publication object must reference a specific Cluster , determining where the publication will be created. It is managed by the cluster's primary instance, ensuring the publication is created or updated as needed.","title":"Required Fields in the Publication Manifest"},{"location":"logical_replication/#reconciliation-and-status","text":"After creating a Publication , CloudNativePG manages it on the primary instance of the specified cluster. Following a successful reconciliation cycle, the Publication status will reflect the following: applied: true , indicates the configuration has been successfully applied. observedGeneration matches metadata.generation , confirming the applied configuration corresponds to the most recent changes. If an error occurs during reconciliation, status.applied will be false , and an error message will be included in the status.message field.","title":"Reconciliation and Status"},{"location":"logical_replication/#removing-a-publication","text":"The publicationReclaimPolicy field controls the behavior when deleting a Publication object: retain (default): Leaves the publication in PostgreSQL for manual management. delete : Automatically removes the publication from PostgreSQL. Consider the following example: apiVersion: postgresql.cnpg.io/v1 kind: Publication metadata: name: freddie-publisher spec: cluster: name: freddie dbname: app name: publisher target: allTables: true publicationReclaimPolicy: delete In this case, deleting the Publication object also removes the publisher publication from the app database of the freddie cluster.","title":"Removing a publication"},{"location":"logical_replication/#subscriptions","text":"In PostgreSQL's publish-and-subscribe replication model, a subscription represents the downstream component that consumes data changes. A subscription establishes the connection to a publisher's database and specifies the set of publications (one or more) it subscribes to. Subscriptions can be created on any supported PostgreSQL instance acting as the subscriber . Important Since schema definitions are not replicated, the subscriber must have the corresponding tables already defined before data replication begins. CloudNativePG simplifies subscription management by enabling you to define them declaratively using the Subscription resource. Info Please refer to the API reference for the full list of attributes you can define for each Subscription object. Suppose you want to replicate changes from the publisher publication on the app database of the freddie cluster ( publisher ) to the app database of the king cluster ( subscriber ). Here's an example of a Subscription manifest: apiVersion: postgresql.cnpg.io/v1 kind: Subscription metadata: name: freddie-to-king-subscription spec: cluster: name: king dbname: app name: subscriber externalClusterName: freddie publicationName: publisher In the above example: The subscription object is named freddie-to-king-subscriber ( metadata.name ). The subscription is created in the app database ( spec.dbname ) of the king cluster ( spec.cluster.name ), with name subscriber ( spec.name ). It connects to the publisher publication in the external freddie cluster, referenced by spec.externalClusterName . To facilitate this setup, the freddie external cluster must be defined in the king cluster's configuration. Below is an example excerpt showing how to define the external cluster in the king manifest: externalClusters: - name: freddie connectionParameters: host: freddie-rw.default.svc user: postgres dbname: app Info For more details on configuring the externalClusters section, see the \"Bootstrap\" section of the documentation. As you can see, a subscription can connect to any PostgreSQL database accessible over the network. This flexibility allows you to seamlessly migrate your data into Kubernetes with nearly zero downtime. It\u2019s an excellent option for transitioning from various environments, including popular cloud-based Database-as-a-Service (DBaaS) platforms.","title":"Subscriptions"},{"location":"logical_replication/#required-fields-in-the-subscription-manifest","text":"The following fields are mandatory for defining a Subscription object: metadata.name : A unique name for the Kubernetes Subscription object within its namespace. spec.cluster.name : The name of the PostgreSQL cluster where the subscription will be created. spec.dbname : The name of the database in which the subscription will be created. spec.name : The name of the subscription as it will appear in PostgreSQL. spec.externalClusterName : The name of the external cluster, as defined in the spec.cluster.name cluster's configuration. This references the publisher database. spec.publicationName : The name of the publication in the publisher database to which the subscription will connect. The Subscription object must reference a specific Cluster , determining where the subscription will be managed. CloudNativePG ensures that the subscription is created or updated on the primary instance of the specified cluster.","title":"Required Fields in the Subscription Manifest"},{"location":"logical_replication/#reconciliation-and-status_1","text":"After creating a Subscription , CloudNativePG manages it on the primary instance of the specified cluster. Following a successful reconciliation cycle, the Subscription status will reflect the following: applied: true , indicates the configuration has been successfully applied. observedGeneration matches metadata.generation , confirming the applied configuration corresponds to the most recent changes. If an error occurs during reconciliation, status.applied will be false , and an error message will be included in the status.message field.","title":"Reconciliation and Status"},{"location":"logical_replication/#removing-a-subscription","text":"The subscriptionReclaimPolicy field controls the behavior when deleting a Subscription object: retain (default): Leaves the subscription in PostgreSQL for manual management. delete : Automatically removes the subscription from PostgreSQL. Consider the following example: apiVersion: postgresql.cnpg.io/v1 kind: Subscription metadata: name: freddie-to-king-subscription spec: cluster: name: king dbname: app name: subscriber externalClusterName: freddie publicationName: publisher subscriptionReclaimPolicy: delete In this case, deleting the Subscription object also removes the subscriber subscription from the app database of the king cluster.","title":"Removing a subscription"},{"location":"logical_replication/#limitations","text":"Logical replication in PostgreSQL has some inherent limitations, as outlined in the official documentation . Notably, the following objects are not replicated: Database schema and DDL commands Sequence data Large objects","title":"Limitations"},{"location":"logical_replication/#addressing-schema-replication","text":"The first limitation, related to schema replication, can be easily addressed using CloudNativePG's capabilities. For instance, you can leverage the import bootstrap feature to copy the schema of the tables you need to replicate. Alternatively, you can manually create the schema as you would for any PostgreSQL database.","title":"Addressing Schema Replication"},{"location":"logical_replication/#handling-sequences","text":"While sequences are not automatically kept in sync through logical replication, CloudNativePG provides a solution to be used in live migrations. You can use the cnpg plugin to synchronize sequence values, ensuring consistency between the publisher and subscriber databases.","title":"Handling Sequences"},{"location":"logical_replication/#example-of-live-migration-and-major-postgres-upgrade-with-logical-replication","text":"To highlight the powerful capabilities of logical replication, this example demonstrates how to replicate data from a publisher database ( freddie ) running PostgreSQL 16 to a subscriber database ( king ) running the latest PostgreSQL version. This setup can be deployed in your Kubernetes cluster for evaluation and hands-on learning. This example illustrates how logical replication facilitates live migrations and upgrades between PostgreSQL versions while ensuring data consistency. By combining logical replication with CloudNativePG, you can easily set up, manage, and evaluate such scenarios in a Kubernetes environment.","title":"Example of live migration and major Postgres upgrade with logical replication"},{"location":"logical_replication/#step-1-setting-up-the-publisher-freddie","text":"The first step involves creating a freddie PostgreSQL cluster with version 16. The cluster contains a single instance and includes an app database initialized with a table, n , storing 10,000 numbers. A logical replication publication named publisher is also configured to include all tables in the database. Here\u2019s the manifest for setting up the freddie cluster and its publication resource: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: freddie spec: instances: 1 imageName: ghcr.io/cloudnative-pg/postgresql:16 storage: size: 1Gi bootstrap: initdb: postInitApplicationSQL: - CREATE TABLE n (i SERIAL PRIMARY KEY, m INTEGER) - INSERT INTO n (m) (SELECT generate_series(1, 10000)) - ALTER TABLE n OWNER TO app managed: roles: - name: app login: true replication: true --- apiVersion: postgresql.cnpg.io/v1 kind: Publication metadata: name: freddie-publisher spec: cluster: name: freddie dbname: app name: publisher target: allTables: true","title":"Step 1: Setting Up the Publisher (freddie)"},{"location":"logical_replication/#step-2-setting-up-the-subscriber-king","text":"Next, create the king PostgreSQL cluster, running the latest version of PostgreSQL. This cluster initializes by importing the schema from the app database on the freddie cluster using the external cluster configuration. A Subscription resource, freddie-to-king-subscription , is then configured to consume changes published by the publisher on freddie . Below is the manifest for setting up the king cluster and its subscription: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: king spec: instances: 1 storage: size: 1Gi bootstrap: initdb: import: type: microservice schemaOnly: true databases: - app source: externalCluster: freddie externalClusters: - name: freddie connectionParameters: host: freddie-rw.default.svc user: app dbname: app password: name: freddie-app key: password --- apiVersion: postgresql.cnpg.io/v1 kind: Subscription metadata: name: freddie-to-king-subscription spec: cluster: name: king dbname: app name: subscriber externalClusterName: freddie publicationName: publisher Once the king cluster is running, you can verify that the replication is working by connecting to the app database and counting the records in the n table. The following example uses the psql command provided by the cnpg plugin for simplicity: kubectl cnpg psql king -- app -qAt -c 'SELECT count(*) FROM n' 10000 This command should return 10000 , confirming that the data from the freddie cluster has been successfully replicated to the king cluster. Using the cnpg plugin, you can also synchronize existing sequences to ensure consistency between the publisher and subscriber. The example below demonstrates how to synchronize a sequence for the king cluster: kubectl cnpg subscription sync-sequences king --subscription=subscriber SELECT setval('\"public\".\"n_i_seq\"', 10000); 10000 This command updates the sequence n_i_seq in the king cluster to match the current value, ensuring it is in sync with the source database.","title":"Step 2: Setting Up the Subscriber (king)"},{"location":"monitoring/","text":"Monitoring Important Installing Prometheus and Grafana is beyond the scope of this project. We assume they are correctly installed in your system. However, for experimentation we provide instructions in Part 4 of the Quickstart . Monitoring Instances For each PostgreSQL instance, the operator provides an exporter of metrics for Prometheus via HTTP or HTTPS, on port 9187, named metrics . The operator comes with a predefined set of metrics , as well as a highly configurable and customizable system to define additional queries via one or more ConfigMap or Secret resources (see the \"User defined metrics\" section below for details). Important CloudNativePG, by default, installs a set of predefined metrics in a ConfigMap named default-monitoring . Info You can inspect the exported metrics by following the instructions in the \"How to inspect the exported metrics\" section below. All monitoring queries that are performed on PostgreSQL are: atomic (one transaction per query) executed with the pg_monitor role executed with application_name set to cnpg_metrics_exporter executed as user postgres Please refer to the \"Predefined Roles\" section in PostgreSQL documentation for details on the pg_monitor role. Queries, by default, are run against the main database , as defined by the specified bootstrap method of the Cluster resource, according to the following logic: using initdb : queries will be run by default against the specified database in initdb.database , or app if not specified using recovery : queries will be run by default against the specified database in recovery.database , or postgres if not specified using pg_basebackup : queries will be run by default against the specified database in pg_basebackup.database , or postgres if not specified The default database can always be overridden for a given user-defined metric, by specifying a list of one or more databases in the target_databases option. Prometheus/Grafana If you are interested in evaluating the integration of CloudNativePG with Prometheus and Grafana, you can find a quick setup guide in Part 4 of the quickstart Monitoring with the Prometheus operator A specific PostgreSQL cluster can be monitored using the Prometheus Operator's resource PodMonitor . A PodMonitor that correctly points to the Cluster can be automatically created by the operator by setting .spec.monitoring.enablePodMonitor to true in the Cluster resource itself (default: false ). Important Any change to the PodMonitor created automatically will be overridden by the Operator at the next reconciliation cycle, in case you need to customize it, you can do so as described below. To deploy a PodMonitor for a specific Cluster manually, define it as follows and adjust as needed: apiVersion: monitoring.coreos.com/v1 kind: PodMonitor metadata: name: cluster-example spec: selector: matchLabels: \"cnpg.io/cluster\": cluster-example podMetricsEndpoints: - port: metrics Important Ensure you modify the example above with a unique name, as well as the correct cluster's namespace and labels (e.g., cluster-example ). Important The postgresql label, used in previous versions of this document, is deprecated and will be removed in the future. Please use the cnpg.io/cluster label instead to select the instances. Enabling TLS on the Metrics Port To enable TLS communication on the metrics port, configure the .spec.monitoring.tls.enabled setting to true . This setup ensures that the metrics exporter uses the same server certificate used by PostgreSQL to secure communication on port 5432. Important Changing the .spec.monitoring.tls.enabled setting will trigger a rolling restart of the Cluster. If the PodMonitor is managed by the operator ( .spec.monitoring.enablePodMonitor set to true ), it will automatically contain the necessary configurations to access the metrics via TLS. To manually deploy a PodMonitor suitable for reading metrics via TLS, define it as follows and adjust as needed: apiVersion: monitoring.coreos.com/v1 kind: PodMonitor metadata: name: cluster-example spec: selector: matchLabels: \"cnpg.io/cluster\": cluster-example podMetricsEndpoints: - port: metrics scheme: https tlsConfig: ca: secret: name: cluster-example-ca key: ca.crt serverName: cluster-example-rw Important Ensure you modify the example above with a unique name, as well as the correct Cluster's namespace and labels (e.g., cluster-example ). Important The serverName field in the metrics endpoint must match one of the names defined in the server certificate. If the default certificate is in use, the serverName value should be in the format -rw . Predefined set of metrics Every PostgreSQL instance exporter automatically exposes a set of predefined metrics, which can be classified in two major categories: PostgreSQL related metrics, starting with cnpg_collector_* , including: number of WAL files and total size on disk number of .ready and .done files in the archive status folder requested minimum and maximum number of synchronous replicas, as well as the expected and actually observed values number of distinct nodes accommodating the instances timestamps indicating last failed and last available backup, as well as the first point of recoverability for the cluster flag indicating if replica cluster mode is enabled or disabled flag indicating if a manual switchover is required flag indicating if fencing is enabled or disabled Go runtime related metrics, starting with go_* Below is a sample of the metrics returned by the localhost:9187/metrics endpoint of an instance. As you can see, the Prometheus format is self-documenting: # HELP cnpg_collector_collection_duration_seconds Collection time duration in seconds # TYPE cnpg_collector_collection_duration_seconds gauge cnpg_collector_collection_duration_seconds{collector=\"Collect.up\"} 0.0031393 # HELP cnpg_collector_collections_total Total number of times PostgreSQL was accessed for metrics. # TYPE cnpg_collector_collections_total counter cnpg_collector_collections_total 2 # HELP cnpg_collector_fencing_on 1 if the instance is fenced, 0 otherwise # TYPE cnpg_collector_fencing_on gauge cnpg_collector_fencing_on 0 # HELP cnpg_collector_nodes_used NodesUsed represents the count of distinct nodes accommodating the instances. A value of '-1' suggests that the metric is not available. A value of '1' suggests that all instances are hosted on a single node, implying the absence of High Availability (HA). Ideally this value should match the number of instances in the cluster. # TYPE cnpg_collector_nodes_used gauge cnpg_collector_nodes_used 3 # HELP cnpg_collector_last_collection_error 1 if the last collection ended with error, 0 otherwise. # TYPE cnpg_collector_last_collection_error gauge cnpg_collector_last_collection_error 0 # HELP cnpg_collector_manual_switchover_required 1 if a manual switchover is required, 0 otherwise # TYPE cnpg_collector_manual_switchover_required gauge cnpg_collector_manual_switchover_required 0 # HELP cnpg_collector_pg_wal Total size in bytes of WAL segments in the '/var/lib/postgresql/data/pgdata/pg_wal' directory computed as (wal_segment_size * count) # TYPE cnpg_collector_pg_wal gauge cnpg_collector_pg_wal{value=\"count\"} 9 cnpg_collector_pg_wal{value=\"slots_max\"} NaN cnpg_collector_pg_wal{value=\"keep\"} 32 cnpg_collector_pg_wal{value=\"max\"} 64 cnpg_collector_pg_wal{value=\"min\"} 5 cnpg_collector_pg_wal{value=\"size\"} 1.50994944e+08 cnpg_collector_pg_wal{value=\"volume_max\"} 128 cnpg_collector_pg_wal{value=\"volume_size\"} 2.147483648e+09 # HELP cnpg_collector_pg_wal_archive_status Number of WAL segments in the '/var/lib/postgresql/data/pgdata/pg_wal/archive_status' directory (ready, done) # TYPE cnpg_collector_pg_wal_archive_status gauge cnpg_collector_pg_wal_archive_status{value=\"done\"} 6 cnpg_collector_pg_wal_archive_status{value=\"ready\"} 0 # HELP cnpg_collector_replica_mode 1 if the cluster is in replica mode, 0 otherwise # TYPE cnpg_collector_replica_mode gauge cnpg_collector_replica_mode 0 # HELP cnpg_collector_sync_replicas Number of requested synchronous replicas (synchronous_standby_names) # TYPE cnpg_collector_sync_replicas gauge cnpg_collector_sync_replicas{value=\"expected\"} 0 cnpg_collector_sync_replicas{value=\"max\"} 0 cnpg_collector_sync_replicas{value=\"min\"} 0 cnpg_collector_sync_replicas{value=\"observed\"} 0 # HELP cnpg_collector_up 1 if PostgreSQL is up, 0 otherwise. # TYPE cnpg_collector_up gauge cnpg_collector_up{cluster=\"cluster-example\"} 1 # HELP cnpg_collector_postgres_version Postgres version # TYPE cnpg_collector_postgres_version gauge cnpg_collector_postgres_version{cluster=\"cluster-example\",full=\"17.5\"} 17.5 # HELP cnpg_collector_last_failed_backup_timestamp The last failed backup as a unix timestamp # TYPE cnpg_collector_last_failed_backup_timestamp gauge cnpg_collector_last_failed_backup_timestamp 0 # HELP cnpg_collector_last_available_backup_timestamp The last available backup as a unix timestamp # TYPE cnpg_collector_last_available_backup_timestamp gauge cnpg_collector_last_available_backup_timestamp 1.63238406e+09 # HELP cnpg_collector_first_recoverability_point The first point of recoverability for the cluster as a unix timestamp # TYPE cnpg_collector_first_recoverability_point gauge cnpg_collector_first_recoverability_point 1.63238406e+09 # HELP cnpg_collector_lo_pages Estimated number of pages in the pg_largeobject table # TYPE cnpg_collector_lo_pages gauge cnpg_collector_lo_pages{datname=\"app\"} 0 cnpg_collector_lo_pages{datname=\"postgres\"} 78 # HELP cnpg_collector_wal_buffers_full Number of times WAL data was written to disk because WAL buffers became full. Only available on PG 14+ # TYPE cnpg_collector_wal_buffers_full gauge cnpg_collector_wal_buffers_full{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 6472 # HELP cnpg_collector_wal_bytes Total amount of WAL generated in bytes. Only available on PG 14+ # TYPE cnpg_collector_wal_bytes gauge cnpg_collector_wal_bytes{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 1.0035147e+07 # HELP cnpg_collector_wal_fpi Total number of WAL full page images generated. Only available on PG 14+ # TYPE cnpg_collector_wal_fpi gauge cnpg_collector_wal_fpi{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 1474 # HELP cnpg_collector_wal_records Total number of WAL records generated. Only available on PG 14+ # TYPE cnpg_collector_wal_records gauge cnpg_collector_wal_records{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 26178 # HELP cnpg_collector_wal_sync Number of times WAL files were synced to disk via issue_xlog_fsync request (if fsync is on and wal_sync_method is either fdatasync, fsync or fsync_writethrough, otherwise zero). Only available on PG 14+ # TYPE cnpg_collector_wal_sync gauge cnpg_collector_wal_sync{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 37 # HELP cnpg_collector_wal_sync_time Total amount of time spent syncing WAL files to disk via issue_xlog_fsync request, in milliseconds (if track_wal_io_timing is enabled, fsync is on, and wal_sync_method is either fdatasync, fsync or fsync_writethrough, otherwise zero). Only available on PG 14+ # TYPE cnpg_collector_wal_sync_time gauge cnpg_collector_wal_sync_time{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 0 # HELP cnpg_collector_wal_write Number of times WAL buffers were written out to disk via XLogWrite request. Only available on PG 14+ # TYPE cnpg_collector_wal_write gauge cnpg_collector_wal_write{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 7243 # HELP cnpg_collector_wal_write_time Total amount of time spent writing WAL buffers to disk via XLogWrite request, in milliseconds (if track_wal_io_timing is enabled, otherwise zero). This includes the sync time when wal_sync_method is either open_datasync or open_sync. Only available on PG 14+ # TYPE cnpg_collector_wal_write_time gauge cnpg_collector_wal_write_time{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 0 # HELP cnpg_last_error 1 if the last collection ended with error, 0 otherwise. # TYPE cnpg_last_error gauge cnpg_last_error 0 # HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles. # TYPE go_gc_duration_seconds summary go_gc_duration_seconds{quantile=\"0\"} 5.01e-05 go_gc_duration_seconds{quantile=\"0.25\"} 7.27e-05 go_gc_duration_seconds{quantile=\"0.5\"} 0.0001748 go_gc_duration_seconds{quantile=\"0.75\"} 0.0002959 go_gc_duration_seconds{quantile=\"1\"} 0.0012776 go_gc_duration_seconds_sum 0.0035741 go_gc_duration_seconds_count 13 # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge go_goroutines 25 # HELP go_info Information about the Go environment. # TYPE go_info gauge go_info{version=\"go1.20.5\"} 1 # HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. # TYPE go_memstats_alloc_bytes gauge go_memstats_alloc_bytes 4.493744e+06 # HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed. # TYPE go_memstats_alloc_bytes_total counter go_memstats_alloc_bytes_total 2.1698216e+07 # HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. # TYPE go_memstats_buck_hash_sys_bytes gauge go_memstats_buck_hash_sys_bytes 1.456234e+06 # HELP go_memstats_frees_total Total number of frees. # TYPE go_memstats_frees_total counter go_memstats_frees_total 172118 # HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started. # TYPE go_memstats_gc_cpu_fraction gauge go_memstats_gc_cpu_fraction 1.0749468700447189e-05 # HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. # TYPE go_memstats_gc_sys_bytes gauge go_memstats_gc_sys_bytes 5.530048e+06 # HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use. # TYPE go_memstats_heap_alloc_bytes gauge go_memstats_heap_alloc_bytes 4.493744e+06 # HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used. # TYPE go_memstats_heap_idle_bytes gauge go_memstats_heap_idle_bytes 5.8236928e+07 # HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use. # TYPE go_memstats_heap_inuse_bytes gauge go_memstats_heap_inuse_bytes 7.528448e+06 # HELP go_memstats_heap_objects Number of allocated objects. # TYPE go_memstats_heap_objects gauge go_memstats_heap_objects 26306 # HELP go_memstats_heap_released_bytes Number of heap bytes released to OS. # TYPE go_memstats_heap_released_bytes gauge go_memstats_heap_released_bytes 5.7401344e+07 # HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system. # TYPE go_memstats_heap_sys_bytes gauge go_memstats_heap_sys_bytes 6.5765376e+07 # HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection. # TYPE go_memstats_last_gc_time_seconds gauge go_memstats_last_gc_time_seconds 1.6311727586032727e+09 # HELP go_memstats_lookups_total Total number of pointer lookups. # TYPE go_memstats_lookups_total counter go_memstats_lookups_total 0 # HELP go_memstats_mallocs_total Total number of mallocs. # TYPE go_memstats_mallocs_total counter go_memstats_mallocs_total 198424 # HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures. # TYPE go_memstats_mcache_inuse_bytes gauge go_memstats_mcache_inuse_bytes 14400 # HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system. # TYPE go_memstats_mcache_sys_bytes gauge go_memstats_mcache_sys_bytes 16384 # HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures. # TYPE go_memstats_mspan_inuse_bytes gauge go_memstats_mspan_inuse_bytes 191896 # HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system. # TYPE go_memstats_mspan_sys_bytes gauge go_memstats_mspan_sys_bytes 212992 # HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place. # TYPE go_memstats_next_gc_bytes gauge go_memstats_next_gc_bytes 8.689632e+06 # HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations. # TYPE go_memstats_other_sys_bytes gauge go_memstats_other_sys_bytes 2.566622e+06 # HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator. # TYPE go_memstats_stack_inuse_bytes gauge go_memstats_stack_inuse_bytes 1.343488e+06 # HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator. # TYPE go_memstats_stack_sys_bytes gauge go_memstats_stack_sys_bytes 1.343488e+06 # HELP go_memstats_sys_bytes Number of bytes obtained from system. # TYPE go_memstats_sys_bytes gauge go_memstats_sys_bytes 7.6891144e+07 # HELP go_threads Number of OS threads created. # TYPE go_threads gauge go_threads 18 Note cnpg_collector_postgres_version is a GaugeVec metric containing the Major.Minor version of PostgreSQL. The full semantic version Major.Minor.Patch can be found inside one of its label field named full . Note cnpg_collector_first_recoverability_point and cnpg_collector_last_available_backup_timestamp will be zero until your first backup to the object store. This is separate from the WAL archival. User defined metrics This feature is currently in beta state and the format is inspired by the queries.yaml file (release 0.12) of the PostgreSQL Prometheus Exporter. Custom metrics can be defined by users by referring to the created Configmap / Secret in a Cluster definition under the .spec.monitoring.customQueriesConfigMap or customQueriesSecret section as in the following example: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example namespace: test spec: instances: 3 storage: size: 1Gi monitoring: customQueriesConfigMap: - name: example-monitoring key: custom-queries The customQueriesConfigMap / customQueriesSecret sections contain a list of ConfigMap / Secret references specifying the key in which the custom queries are defined. Take care that the referred resources have to be created in the same namespace as the Cluster resource. Note If you want ConfigMaps and Secrets to be automatically reloaded by instances, you can add a label with key cnpg.io/reload to it, otherwise you will have to reload the instances using the kubectl cnpg reload subcommand. Important When a user defined metric overwrites an already existing metric the instance manager prints a json warning log, containing the message: Query with the same name already found. Overwriting the existing one. and a key queryName containing the overwritten query name. Example of a user defined metric Here you can see an example of a ConfigMap containing a single custom query, referenced by the Cluster example above: apiVersion: v1 kind: ConfigMap metadata: name: example-monitoring namespace: test labels: cnpg.io/reload: \"\" data: custom-queries: | pg_replication: query: \"SELECT CASE WHEN NOT pg_is_in_recovery() THEN 0 ELSE GREATEST (0, EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))) END AS lag, pg_is_in_recovery() AS in_recovery, EXISTS (TABLE pg_stat_wal_receiver) AS is_wal_receiver_up, (SELECT count(*) FROM pg_stat_replication) AS streaming_replicas\" metrics: - lag: usage: \"GAUGE\" description: \"Replication lag behind primary in seconds\" - in_recovery: usage: \"GAUGE\" description: \"Whether the instance is in recovery\" - is_wal_receiver_up: usage: \"GAUGE\" description: \"Whether the instance wal_receiver is up\" - streaming_replicas: usage: \"GAUGE\" description: \"Number of streaming replicas connected to the instance\" A list of basic monitoring queries can be found in the default-monitoring.yaml file that is already installed in your CloudNativePG deployment (see \"Default set of metrics\" ). Example of a user defined metric with predicate query The predicate_query option allows the user to execute the query to collect the metrics only under the specified conditions. To do so the user needs to provide a predicate query that returns at most one row with a single boolean column. The predicate query is executed in the same transaction as the main query and against the same databases. some_query: | predicate_query: | SELECT some_bool as predicate FROM some_table query: | SELECT count(*) as rows FROM some_table metrics: - rows: usage: \"GAUGE\" description: \"number of rows\" Example of a user defined metric running on multiple databases If the target_databases option lists more than one database the metric is collected from each of them. Database auto-discovery can be enabled for a specific query by specifying a shell-like pattern (i.e., containing * , ? or [] ) in the list of target_databases . If provided, the operator will expand the list of target databases by adding all the databases returned by the execution of SELECT datname FROM pg_database WHERE datallowconn AND NOT datistemplate and matching the pattern according to path.Match() rules. Note The * character has a special meaning in yaml, so you need to quote ( \"*\" ) the target_databases value when it includes such a pattern. It is recommended that you always include the name of the database in the returned labels, for example using the current_database() function as in the following example: some_query: | query: | SELECT current_database() as datname, count(*) as rows FROM some_table metrics: - datname: usage: \"LABEL\" description: \"Name of current database\" - rows: usage: \"GAUGE\" description: \"number of rows\" target_databases: - albert - bb - freddie This will produce in the following metric being exposed: cnpg_some_query_rows{datname=\"albert\"} 2 cnpg_some_query_rows{datname=\"bb\"} 5 cnpg_some_query_rows{datname=\"freddie\"} 10 Here is an example of a query with auto-discovery enabled which also runs on the template1 database (otherwise not returned by the aforementioned query): some_query: | query: | SELECT current_database() as datname, count(*) as rows FROM some_table metrics: - datname: usage: \"LABEL\" description: \"Name of current database\" - rows: usage: \"GAUGE\" description: \"number of rows\" target_databases: - \"*\" - \"template1\" The above example will produce the following metrics (provided the databases exist): cnpg_some_query_rows{datname=\"albert\"} 2 cnpg_some_query_rows{datname=\"bb\"} 5 cnpg_some_query_rows{datname=\"freddie\"} 10 cnpg_some_query_rows{datname=\"template1\"} 7 cnpg_some_query_rows{datname=\"postgres\"} 42 Structure of a user defined metric Every custom query has the following basic structure: : query: \"\" metrics: - : usage: \"\" description: \"\" Here is a short description of all the available fields: : the name of the Prometheus metric name : override , if defined query : the SQL query to run on the target database to generate the metrics primary : whether to run the query only on the primary instance master : same as primary (for compatibility with the Prometheus PostgreSQL exporter's syntax - deprecated) runonserver : a semantic version range to limit the versions of PostgreSQL the query should run on (e.g. \">=11.0.0\" or \">=12.0.0 <=15.0.0\" ) target_databases : a list of databases to run the query against, or a shell-like pattern to enable auto discovery. Overwrites the default database if provided. predicate_query : a SQL query that returns at most one row and one boolean column to run on the target database. The system evaluates the predicate and if true executes the query . metrics : section containing a list of all exported columns, defined as follows: : the name of the column returned by the query name : override the ColumnName of the column in the metric, if defined usage : one of the values described below description : the metric's description metrics_mapping : the optional column mapping when usage is set to MAPPEDMETRIC The possible values for usage are: Column Usage Label Description DISCARD this column should be ignored LABEL use this column as a label COUNTER use this column as a counter GAUGE use this column as a gauge MAPPEDMETRIC use this column with the supplied mapping of text values DURATION use this column as a text duration (in milliseconds) HISTOGRAM use this column as a histogram Please visit the \"Metric Types\" page from the Prometheus documentation for more information. Output of a user defined metric Custom defined metrics are returned by the Prometheus exporter endpoint ( :9187/metrics ) with the following format: cnpg__{= ... } Note LabelColumnName are metrics with usage set to LABEL and their Value Considering the pg_replication example above, the exporter's endpoint would return the following output when invoked: # HELP cnpg_pg_replication_in_recovery Whether the instance is in recovery # TYPE cnpg_pg_replication_in_recovery gauge cnpg_pg_replication_in_recovery 0 # HELP cnpg_pg_replication_lag Replication lag behind primary in seconds # TYPE cnpg_pg_replication_lag gauge cnpg_pg_replication_lag 0 # HELP cnpg_pg_replication_streaming_replicas Number of streaming replicas connected to the instance # TYPE cnpg_pg_replication_streaming_replicas gauge cnpg_pg_replication_streaming_replicas 2 # HELP cnpg_pg_replication_is_wal_receiver_up Whether the instance wal_receiver is up # TYPE cnpg_pg_replication_is_wal_receiver_up gauge cnpg_pg_replication_is_wal_receiver_up 0 Default set of metrics The operator can be configured to automatically inject in a Cluster a set of monitoring queries defined in a ConfigMap or a Secret, inside the operator's namespace. You have to set the MONITORING_QUERIES_CONFIGMAP or MONITORING_QUERIES_SECRET key in the \"operator configuration\" , respectively to the name of the ConfigMap or the Secret; the operator will then use the content of the queries key. Any change to the queries content will be immediately reflected on all the deployed Clusters using it. The operator installation manifests come with a predefined ConfigMap, called cnpg-default-monitoring , to be used by all Clusters. MONITORING_QUERIES_CONFIGMAP is by default set to cnpg-default-monitoring in the operator configuration. If you want to disable the default set of metrics, you can: disable it at operator level: set the MONITORING_QUERIES_CONFIGMAP / MONITORING_QUERIES_SECRET key to \"\" (empty string), in the operator ConfigMap. Changes to operator ConfigMap require an operator restart. disable it for a specific Cluster: set .spec.monitoring.disableDefaultQueries to true in the Cluster. Important The ConfigMap or Secret specified via MONITORING_QUERIES_CONFIGMAP / MONITORING_QUERIES_SECRET will always be copied to the Cluster's namespace with a fixed name: cnpg-default-monitoring . So that, if you intend to have default metrics, you should not create a ConfigMap with this name in the cluster's namespace. Differences with the Prometheus Postgres exporter CloudNativePG is inspired by the PostgreSQL Prometheus Exporter, but presents some differences. In particular, the cache_seconds field is not implemented in CloudNativePG's exporter. Monitoring the CloudNativePG operator The operator internally exposes Prometheus metrics via HTTP on port 8080, named metrics . Info You can inspect the exported metrics by following the instructions in the \"How to inspect the exported metrics\" section below. Currently, the operator exposes default kubebuilder metrics. See kubebuilder documentation for more details. Monitoring the operator with Prometheus The operator can be monitored using the Prometheus Operator by defining a PodMonitor pointing to the operator pod(s), as follows (note it's applied in the same namespace as the operator): kubectl -n cnpg-system apply -f - < 8080:8080 With port forwarding active, the metrics are easily viewable on a browser at localhost:8080/metrics . Using curl Create the curl pod with the following command: kubectl apply -f - <:9187/metrics For example, if your PostgreSQL cluster is called cluster-example and you want to retrieve the exported metrics of the first pod in the cluster, you can run the following command to programmatically get the IP of that pod: POD_IP=$(kubectl get pod cluster-example-1 --template '{{.status.podIP}}') And then run: kubectl exec -ti curl -- curl -s ${POD_IP}:9187/metrics If you enabled TLS metrics, run instead: kubectl exec -ti curl -- curl -sk https://${POD_IP}:9187/metrics To access the metrics of the operator, you need to point to the pod where the operator is running, and use TCP port 8080 as target. When you're done inspecting metrics, please remember to delete the curl pod: kubectl delete -f curl.yaml Auxiliary resources Important These resources are provided for illustration and experimentation, and do not represent any kind of recommendation for your production system In the doc/src/samples/monitoring/ directory you will find a series of sample files for observability. Please refer to Part 4 of the quickstart section for context: kube-stack-config.yaml : a configuration file for the kube-stack helm chart installation. It ensures that Prometheus listens for all PodMonitor resources. prometheusrule.yaml : a PrometheusRule with alerts for CloudNativePG. NOTE: this does not include inter-operation with notification services. Please refer to the Prometheus documentation . podmonitor.yaml : a PodMonitor for the CloudNativePG Operator deployment. In addition, we provide the \"raw\" sources for the Prometheus alert rules in the alerts.yaml file. A Grafana dashboard for CloudNativePG clusters and operator, is kept in the dedicated repository cloudnative-pg/grafana-dashboards as a dashboard JSON configuration: grafana-dashboard.json . The file can be downloaded, and imported into Grafana (menus: Dashboard > New > Import). For a general reference on the settings available on kube-prometheus-stack , you can execute helm show values prometheus-community/kube-prometheus-stack . Please refer to the kube-prometheus-stack page for more detail.","title":"Monitoring"},{"location":"monitoring/#monitoring","text":"Important Installing Prometheus and Grafana is beyond the scope of this project. We assume they are correctly installed in your system. However, for experimentation we provide instructions in Part 4 of the Quickstart .","title":"Monitoring"},{"location":"monitoring/#monitoring-instances","text":"For each PostgreSQL instance, the operator provides an exporter of metrics for Prometheus via HTTP or HTTPS, on port 9187, named metrics . The operator comes with a predefined set of metrics , as well as a highly configurable and customizable system to define additional queries via one or more ConfigMap or Secret resources (see the \"User defined metrics\" section below for details). Important CloudNativePG, by default, installs a set of predefined metrics in a ConfigMap named default-monitoring . Info You can inspect the exported metrics by following the instructions in the \"How to inspect the exported metrics\" section below. All monitoring queries that are performed on PostgreSQL are: atomic (one transaction per query) executed with the pg_monitor role executed with application_name set to cnpg_metrics_exporter executed as user postgres Please refer to the \"Predefined Roles\" section in PostgreSQL documentation for details on the pg_monitor role. Queries, by default, are run against the main database , as defined by the specified bootstrap method of the Cluster resource, according to the following logic: using initdb : queries will be run by default against the specified database in initdb.database , or app if not specified using recovery : queries will be run by default against the specified database in recovery.database , or postgres if not specified using pg_basebackup : queries will be run by default against the specified database in pg_basebackup.database , or postgres if not specified The default database can always be overridden for a given user-defined metric, by specifying a list of one or more databases in the target_databases option. Prometheus/Grafana If you are interested in evaluating the integration of CloudNativePG with Prometheus and Grafana, you can find a quick setup guide in Part 4 of the quickstart","title":"Monitoring Instances"},{"location":"monitoring/#monitoring-with-the-prometheus-operator","text":"A specific PostgreSQL cluster can be monitored using the Prometheus Operator's resource PodMonitor . A PodMonitor that correctly points to the Cluster can be automatically created by the operator by setting .spec.monitoring.enablePodMonitor to true in the Cluster resource itself (default: false ). Important Any change to the PodMonitor created automatically will be overridden by the Operator at the next reconciliation cycle, in case you need to customize it, you can do so as described below. To deploy a PodMonitor for a specific Cluster manually, define it as follows and adjust as needed: apiVersion: monitoring.coreos.com/v1 kind: PodMonitor metadata: name: cluster-example spec: selector: matchLabels: \"cnpg.io/cluster\": cluster-example podMetricsEndpoints: - port: metrics Important Ensure you modify the example above with a unique name, as well as the correct cluster's namespace and labels (e.g., cluster-example ). Important The postgresql label, used in previous versions of this document, is deprecated and will be removed in the future. Please use the cnpg.io/cluster label instead to select the instances.","title":"Monitoring with the Prometheus operator"},{"location":"monitoring/#enabling-tls-on-the-metrics-port","text":"To enable TLS communication on the metrics port, configure the .spec.monitoring.tls.enabled setting to true . This setup ensures that the metrics exporter uses the same server certificate used by PostgreSQL to secure communication on port 5432. Important Changing the .spec.monitoring.tls.enabled setting will trigger a rolling restart of the Cluster. If the PodMonitor is managed by the operator ( .spec.monitoring.enablePodMonitor set to true ), it will automatically contain the necessary configurations to access the metrics via TLS. To manually deploy a PodMonitor suitable for reading metrics via TLS, define it as follows and adjust as needed: apiVersion: monitoring.coreos.com/v1 kind: PodMonitor metadata: name: cluster-example spec: selector: matchLabels: \"cnpg.io/cluster\": cluster-example podMetricsEndpoints: - port: metrics scheme: https tlsConfig: ca: secret: name: cluster-example-ca key: ca.crt serverName: cluster-example-rw Important Ensure you modify the example above with a unique name, as well as the correct Cluster's namespace and labels (e.g., cluster-example ). Important The serverName field in the metrics endpoint must match one of the names defined in the server certificate. If the default certificate is in use, the serverName value should be in the format -rw .","title":"Enabling TLS on the Metrics Port"},{"location":"monitoring/#predefined-set-of-metrics","text":"Every PostgreSQL instance exporter automatically exposes a set of predefined metrics, which can be classified in two major categories: PostgreSQL related metrics, starting with cnpg_collector_* , including: number of WAL files and total size on disk number of .ready and .done files in the archive status folder requested minimum and maximum number of synchronous replicas, as well as the expected and actually observed values number of distinct nodes accommodating the instances timestamps indicating last failed and last available backup, as well as the first point of recoverability for the cluster flag indicating if replica cluster mode is enabled or disabled flag indicating if a manual switchover is required flag indicating if fencing is enabled or disabled Go runtime related metrics, starting with go_* Below is a sample of the metrics returned by the localhost:9187/metrics endpoint of an instance. As you can see, the Prometheus format is self-documenting: # HELP cnpg_collector_collection_duration_seconds Collection time duration in seconds # TYPE cnpg_collector_collection_duration_seconds gauge cnpg_collector_collection_duration_seconds{collector=\"Collect.up\"} 0.0031393 # HELP cnpg_collector_collections_total Total number of times PostgreSQL was accessed for metrics. # TYPE cnpg_collector_collections_total counter cnpg_collector_collections_total 2 # HELP cnpg_collector_fencing_on 1 if the instance is fenced, 0 otherwise # TYPE cnpg_collector_fencing_on gauge cnpg_collector_fencing_on 0 # HELP cnpg_collector_nodes_used NodesUsed represents the count of distinct nodes accommodating the instances. A value of '-1' suggests that the metric is not available. A value of '1' suggests that all instances are hosted on a single node, implying the absence of High Availability (HA). Ideally this value should match the number of instances in the cluster. # TYPE cnpg_collector_nodes_used gauge cnpg_collector_nodes_used 3 # HELP cnpg_collector_last_collection_error 1 if the last collection ended with error, 0 otherwise. # TYPE cnpg_collector_last_collection_error gauge cnpg_collector_last_collection_error 0 # HELP cnpg_collector_manual_switchover_required 1 if a manual switchover is required, 0 otherwise # TYPE cnpg_collector_manual_switchover_required gauge cnpg_collector_manual_switchover_required 0 # HELP cnpg_collector_pg_wal Total size in bytes of WAL segments in the '/var/lib/postgresql/data/pgdata/pg_wal' directory computed as (wal_segment_size * count) # TYPE cnpg_collector_pg_wal gauge cnpg_collector_pg_wal{value=\"count\"} 9 cnpg_collector_pg_wal{value=\"slots_max\"} NaN cnpg_collector_pg_wal{value=\"keep\"} 32 cnpg_collector_pg_wal{value=\"max\"} 64 cnpg_collector_pg_wal{value=\"min\"} 5 cnpg_collector_pg_wal{value=\"size\"} 1.50994944e+08 cnpg_collector_pg_wal{value=\"volume_max\"} 128 cnpg_collector_pg_wal{value=\"volume_size\"} 2.147483648e+09 # HELP cnpg_collector_pg_wal_archive_status Number of WAL segments in the '/var/lib/postgresql/data/pgdata/pg_wal/archive_status' directory (ready, done) # TYPE cnpg_collector_pg_wal_archive_status gauge cnpg_collector_pg_wal_archive_status{value=\"done\"} 6 cnpg_collector_pg_wal_archive_status{value=\"ready\"} 0 # HELP cnpg_collector_replica_mode 1 if the cluster is in replica mode, 0 otherwise # TYPE cnpg_collector_replica_mode gauge cnpg_collector_replica_mode 0 # HELP cnpg_collector_sync_replicas Number of requested synchronous replicas (synchronous_standby_names) # TYPE cnpg_collector_sync_replicas gauge cnpg_collector_sync_replicas{value=\"expected\"} 0 cnpg_collector_sync_replicas{value=\"max\"} 0 cnpg_collector_sync_replicas{value=\"min\"} 0 cnpg_collector_sync_replicas{value=\"observed\"} 0 # HELP cnpg_collector_up 1 if PostgreSQL is up, 0 otherwise. # TYPE cnpg_collector_up gauge cnpg_collector_up{cluster=\"cluster-example\"} 1 # HELP cnpg_collector_postgres_version Postgres version # TYPE cnpg_collector_postgres_version gauge cnpg_collector_postgres_version{cluster=\"cluster-example\",full=\"17.5\"} 17.5 # HELP cnpg_collector_last_failed_backup_timestamp The last failed backup as a unix timestamp # TYPE cnpg_collector_last_failed_backup_timestamp gauge cnpg_collector_last_failed_backup_timestamp 0 # HELP cnpg_collector_last_available_backup_timestamp The last available backup as a unix timestamp # TYPE cnpg_collector_last_available_backup_timestamp gauge cnpg_collector_last_available_backup_timestamp 1.63238406e+09 # HELP cnpg_collector_first_recoverability_point The first point of recoverability for the cluster as a unix timestamp # TYPE cnpg_collector_first_recoverability_point gauge cnpg_collector_first_recoverability_point 1.63238406e+09 # HELP cnpg_collector_lo_pages Estimated number of pages in the pg_largeobject table # TYPE cnpg_collector_lo_pages gauge cnpg_collector_lo_pages{datname=\"app\"} 0 cnpg_collector_lo_pages{datname=\"postgres\"} 78 # HELP cnpg_collector_wal_buffers_full Number of times WAL data was written to disk because WAL buffers became full. Only available on PG 14+ # TYPE cnpg_collector_wal_buffers_full gauge cnpg_collector_wal_buffers_full{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 6472 # HELP cnpg_collector_wal_bytes Total amount of WAL generated in bytes. Only available on PG 14+ # TYPE cnpg_collector_wal_bytes gauge cnpg_collector_wal_bytes{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 1.0035147e+07 # HELP cnpg_collector_wal_fpi Total number of WAL full page images generated. Only available on PG 14+ # TYPE cnpg_collector_wal_fpi gauge cnpg_collector_wal_fpi{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 1474 # HELP cnpg_collector_wal_records Total number of WAL records generated. Only available on PG 14+ # TYPE cnpg_collector_wal_records gauge cnpg_collector_wal_records{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 26178 # HELP cnpg_collector_wal_sync Number of times WAL files were synced to disk via issue_xlog_fsync request (if fsync is on and wal_sync_method is either fdatasync, fsync or fsync_writethrough, otherwise zero). Only available on PG 14+ # TYPE cnpg_collector_wal_sync gauge cnpg_collector_wal_sync{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 37 # HELP cnpg_collector_wal_sync_time Total amount of time spent syncing WAL files to disk via issue_xlog_fsync request, in milliseconds (if track_wal_io_timing is enabled, fsync is on, and wal_sync_method is either fdatasync, fsync or fsync_writethrough, otherwise zero). Only available on PG 14+ # TYPE cnpg_collector_wal_sync_time gauge cnpg_collector_wal_sync_time{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 0 # HELP cnpg_collector_wal_write Number of times WAL buffers were written out to disk via XLogWrite request. Only available on PG 14+ # TYPE cnpg_collector_wal_write gauge cnpg_collector_wal_write{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 7243 # HELP cnpg_collector_wal_write_time Total amount of time spent writing WAL buffers to disk via XLogWrite request, in milliseconds (if track_wal_io_timing is enabled, otherwise zero). This includes the sync time when wal_sync_method is either open_datasync or open_sync. Only available on PG 14+ # TYPE cnpg_collector_wal_write_time gauge cnpg_collector_wal_write_time{stats_reset=\"2023-06-19T10:51:27.473259Z\"} 0 # HELP cnpg_last_error 1 if the last collection ended with error, 0 otherwise. # TYPE cnpg_last_error gauge cnpg_last_error 0 # HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles. # TYPE go_gc_duration_seconds summary go_gc_duration_seconds{quantile=\"0\"} 5.01e-05 go_gc_duration_seconds{quantile=\"0.25\"} 7.27e-05 go_gc_duration_seconds{quantile=\"0.5\"} 0.0001748 go_gc_duration_seconds{quantile=\"0.75\"} 0.0002959 go_gc_duration_seconds{quantile=\"1\"} 0.0012776 go_gc_duration_seconds_sum 0.0035741 go_gc_duration_seconds_count 13 # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge go_goroutines 25 # HELP go_info Information about the Go environment. # TYPE go_info gauge go_info{version=\"go1.20.5\"} 1 # HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. # TYPE go_memstats_alloc_bytes gauge go_memstats_alloc_bytes 4.493744e+06 # HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed. # TYPE go_memstats_alloc_bytes_total counter go_memstats_alloc_bytes_total 2.1698216e+07 # HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. # TYPE go_memstats_buck_hash_sys_bytes gauge go_memstats_buck_hash_sys_bytes 1.456234e+06 # HELP go_memstats_frees_total Total number of frees. # TYPE go_memstats_frees_total counter go_memstats_frees_total 172118 # HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started. # TYPE go_memstats_gc_cpu_fraction gauge go_memstats_gc_cpu_fraction 1.0749468700447189e-05 # HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. # TYPE go_memstats_gc_sys_bytes gauge go_memstats_gc_sys_bytes 5.530048e+06 # HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use. # TYPE go_memstats_heap_alloc_bytes gauge go_memstats_heap_alloc_bytes 4.493744e+06 # HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used. # TYPE go_memstats_heap_idle_bytes gauge go_memstats_heap_idle_bytes 5.8236928e+07 # HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use. # TYPE go_memstats_heap_inuse_bytes gauge go_memstats_heap_inuse_bytes 7.528448e+06 # HELP go_memstats_heap_objects Number of allocated objects. # TYPE go_memstats_heap_objects gauge go_memstats_heap_objects 26306 # HELP go_memstats_heap_released_bytes Number of heap bytes released to OS. # TYPE go_memstats_heap_released_bytes gauge go_memstats_heap_released_bytes 5.7401344e+07 # HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system. # TYPE go_memstats_heap_sys_bytes gauge go_memstats_heap_sys_bytes 6.5765376e+07 # HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection. # TYPE go_memstats_last_gc_time_seconds gauge go_memstats_last_gc_time_seconds 1.6311727586032727e+09 # HELP go_memstats_lookups_total Total number of pointer lookups. # TYPE go_memstats_lookups_total counter go_memstats_lookups_total 0 # HELP go_memstats_mallocs_total Total number of mallocs. # TYPE go_memstats_mallocs_total counter go_memstats_mallocs_total 198424 # HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures. # TYPE go_memstats_mcache_inuse_bytes gauge go_memstats_mcache_inuse_bytes 14400 # HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system. # TYPE go_memstats_mcache_sys_bytes gauge go_memstats_mcache_sys_bytes 16384 # HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures. # TYPE go_memstats_mspan_inuse_bytes gauge go_memstats_mspan_inuse_bytes 191896 # HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system. # TYPE go_memstats_mspan_sys_bytes gauge go_memstats_mspan_sys_bytes 212992 # HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place. # TYPE go_memstats_next_gc_bytes gauge go_memstats_next_gc_bytes 8.689632e+06 # HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations. # TYPE go_memstats_other_sys_bytes gauge go_memstats_other_sys_bytes 2.566622e+06 # HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator. # TYPE go_memstats_stack_inuse_bytes gauge go_memstats_stack_inuse_bytes 1.343488e+06 # HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator. # TYPE go_memstats_stack_sys_bytes gauge go_memstats_stack_sys_bytes 1.343488e+06 # HELP go_memstats_sys_bytes Number of bytes obtained from system. # TYPE go_memstats_sys_bytes gauge go_memstats_sys_bytes 7.6891144e+07 # HELP go_threads Number of OS threads created. # TYPE go_threads gauge go_threads 18 Note cnpg_collector_postgres_version is a GaugeVec metric containing the Major.Minor version of PostgreSQL. The full semantic version Major.Minor.Patch can be found inside one of its label field named full . Note cnpg_collector_first_recoverability_point and cnpg_collector_last_available_backup_timestamp will be zero until your first backup to the object store. This is separate from the WAL archival.","title":"Predefined set of metrics"},{"location":"monitoring/#user-defined-metrics","text":"This feature is currently in beta state and the format is inspired by the queries.yaml file (release 0.12) of the PostgreSQL Prometheus Exporter. Custom metrics can be defined by users by referring to the created Configmap / Secret in a Cluster definition under the .spec.monitoring.customQueriesConfigMap or customQueriesSecret section as in the following example: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: cluster-example namespace: test spec: instances: 3 storage: size: 1Gi monitoring: customQueriesConfigMap: - name: example-monitoring key: custom-queries The customQueriesConfigMap / customQueriesSecret sections contain a list of ConfigMap / Secret references specifying the key in which the custom queries are defined. Take care that the referred resources have to be created in the same namespace as the Cluster resource. Note If you want ConfigMaps and Secrets to be automatically reloaded by instances, you can add a label with key cnpg.io/reload to it, otherwise you will have to reload the instances using the kubectl cnpg reload subcommand. Important When a user defined metric overwrites an already existing metric the instance manager prints a json warning log, containing the message: Query with the same name already found. Overwriting the existing one. and a key queryName containing the overwritten query name.","title":"User defined metrics"},{"location":"monitoring/#example-of-a-user-defined-metric","text":"Here you can see an example of a ConfigMap containing a single custom query, referenced by the Cluster example above: apiVersion: v1 kind: ConfigMap metadata: name: example-monitoring namespace: test labels: cnpg.io/reload: \"\" data: custom-queries: | pg_replication: query: \"SELECT CASE WHEN NOT pg_is_in_recovery() THEN 0 ELSE GREATEST (0, EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))) END AS lag, pg_is_in_recovery() AS in_recovery, EXISTS (TABLE pg_stat_wal_receiver) AS is_wal_receiver_up, (SELECT count(*) FROM pg_stat_replication) AS streaming_replicas\" metrics: - lag: usage: \"GAUGE\" description: \"Replication lag behind primary in seconds\" - in_recovery: usage: \"GAUGE\" description: \"Whether the instance is in recovery\" - is_wal_receiver_up: usage: \"GAUGE\" description: \"Whether the instance wal_receiver is up\" - streaming_replicas: usage: \"GAUGE\" description: \"Number of streaming replicas connected to the instance\" A list of basic monitoring queries can be found in the default-monitoring.yaml file that is already installed in your CloudNativePG deployment (see \"Default set of metrics\" ).","title":"Example of a user defined metric"},{"location":"monitoring/#example-of-a-user-defined-metric-with-predicate-query","text":"The predicate_query option allows the user to execute the query to collect the metrics only under the specified conditions. To do so the user needs to provide a predicate query that returns at most one row with a single boolean column. The predicate query is executed in the same transaction as the main query and against the same databases. some_query: | predicate_query: | SELECT some_bool as predicate FROM some_table query: | SELECT count(*) as rows FROM some_table metrics: - rows: usage: \"GAUGE\" description: \"number of rows\"","title":"Example of a user defined metric with predicate query"},{"location":"monitoring/#example-of-a-user-defined-metric-running-on-multiple-databases","text":"If the target_databases option lists more than one database the metric is collected from each of them. Database auto-discovery can be enabled for a specific query by specifying a shell-like pattern (i.e., containing * , ? or [] ) in the list of target_databases . If provided, the operator will expand the list of target databases by adding all the databases returned by the execution of SELECT datname FROM pg_database WHERE datallowconn AND NOT datistemplate and matching the pattern according to path.Match() rules. Note The * character has a special meaning in yaml, so you need to quote ( \"*\" ) the target_databases value when it includes such a pattern. It is recommended that you always include the name of the database in the returned labels, for example using the current_database() function as in the following example: some_query: | query: | SELECT current_database() as datname, count(*) as rows FROM some_table metrics: - datname: usage: \"LABEL\" description: \"Name of current database\" - rows: usage: \"GAUGE\" description: \"number of rows\" target_databases: - albert - bb - freddie This will produce in the following metric being exposed: cnpg_some_query_rows{datname=\"albert\"} 2 cnpg_some_query_rows{datname=\"bb\"} 5 cnpg_some_query_rows{datname=\"freddie\"} 10 Here is an example of a query with auto-discovery enabled which also runs on the template1 database (otherwise not returned by the aforementioned query): some_query: | query: | SELECT current_database() as datname, count(*) as rows FROM some_table metrics: - datname: usage: \"LABEL\" description: \"Name of current database\" - rows: usage: \"GAUGE\" description: \"number of rows\" target_databases: - \"*\" - \"template1\" The above example will produce the following metrics (provided the databases exist): cnpg_some_query_rows{datname=\"albert\"} 2 cnpg_some_query_rows{datname=\"bb\"} 5 cnpg_some_query_rows{datname=\"freddie\"} 10 cnpg_some_query_rows{datname=\"template1\"} 7 cnpg_some_query_rows{datname=\"postgres\"} 42","title":"Example of a user defined metric running on multiple databases"},{"location":"monitoring/#structure-of-a-user-defined-metric","text":"Every custom query has the following basic structure: : query: \"\" metrics: - : usage: \"\" description: \"\" Here is a short description of all the available fields: : the name of the Prometheus metric name : override , if defined query : the SQL query to run on the target database to generate the metrics primary : whether to run the query only on the primary instance master : same as primary (for compatibility with the Prometheus PostgreSQL exporter's syntax - deprecated) runonserver : a semantic version range to limit the versions of PostgreSQL the query should run on (e.g. \">=11.0.0\" or \">=12.0.0 <=15.0.0\" ) target_databases : a list of databases to run the query against, or a shell-like pattern to enable auto discovery. Overwrites the default database if provided. predicate_query : a SQL query that returns at most one row and one boolean column to run on the target database. The system evaluates the predicate and if true executes the query . metrics : section containing a list of all exported columns, defined as follows: : the name of the column returned by the query name : override the ColumnName of the column in the metric, if defined usage : one of the values described below description : the metric's description metrics_mapping : the optional column mapping when usage is set to MAPPEDMETRIC The possible values for usage are: Column Usage Label Description DISCARD this column should be ignored LABEL use this column as a label COUNTER use this column as a counter GAUGE use this column as a gauge MAPPEDMETRIC use this column with the supplied mapping of text values DURATION use this column as a text duration (in milliseconds) HISTOGRAM use this column as a histogram Please visit the \"Metric Types\" page from the Prometheus documentation for more information.","title":"Structure of a user defined metric"},{"location":"monitoring/#output-of-a-user-defined-metric","text":"Custom defined metrics are returned by the Prometheus exporter endpoint ( :9187/metrics ) with the following format: cnpg__{= ... } Note LabelColumnName are metrics with usage set to LABEL and their Value Considering the pg_replication example above, the exporter's endpoint would return the following output when invoked: # HELP cnpg_pg_replication_in_recovery Whether the instance is in recovery # TYPE cnpg_pg_replication_in_recovery gauge cnpg_pg_replication_in_recovery 0 # HELP cnpg_pg_replication_lag Replication lag behind primary in seconds # TYPE cnpg_pg_replication_lag gauge cnpg_pg_replication_lag 0 # HELP cnpg_pg_replication_streaming_replicas Number of streaming replicas connected to the instance # TYPE cnpg_pg_replication_streaming_replicas gauge cnpg_pg_replication_streaming_replicas 2 # HELP cnpg_pg_replication_is_wal_receiver_up Whether the instance wal_receiver is up # TYPE cnpg_pg_replication_is_wal_receiver_up gauge cnpg_pg_replication_is_wal_receiver_up 0","title":"Output of a user defined metric"},{"location":"monitoring/#default-set-of-metrics","text":"The operator can be configured to automatically inject in a Cluster a set of monitoring queries defined in a ConfigMap or a Secret, inside the operator's namespace. You have to set the MONITORING_QUERIES_CONFIGMAP or MONITORING_QUERIES_SECRET key in the \"operator configuration\" , respectively to the name of the ConfigMap or the Secret; the operator will then use the content of the queries key. Any change to the queries content will be immediately reflected on all the deployed Clusters using it. The operator installation manifests come with a predefined ConfigMap, called cnpg-default-monitoring , to be used by all Clusters. MONITORING_QUERIES_CONFIGMAP is by default set to cnpg-default-monitoring in the operator configuration. If you want to disable the default set of metrics, you can: disable it at operator level: set the MONITORING_QUERIES_CONFIGMAP / MONITORING_QUERIES_SECRET key to \"\" (empty string), in the operator ConfigMap. Changes to operator ConfigMap require an operator restart. disable it for a specific Cluster: set .spec.monitoring.disableDefaultQueries to true in the Cluster. Important The ConfigMap or Secret specified via MONITORING_QUERIES_CONFIGMAP / MONITORING_QUERIES_SECRET will always be copied to the Cluster's namespace with a fixed name: cnpg-default-monitoring . So that, if you intend to have default metrics, you should not create a ConfigMap with this name in the cluster's namespace.","title":"Default set of metrics"},{"location":"monitoring/#differences-with-the-prometheus-postgres-exporter","text":"CloudNativePG is inspired by the PostgreSQL Prometheus Exporter, but presents some differences. In particular, the cache_seconds field is not implemented in CloudNativePG's exporter.","title":"Differences with the Prometheus Postgres exporter"},{"location":"monitoring/#monitoring-the-cloudnativepg-operator","text":"The operator internally exposes Prometheus metrics via HTTP on port 8080, named metrics . Info You can inspect the exported metrics by following the instructions in the \"How to inspect the exported metrics\" section below. Currently, the operator exposes default kubebuilder metrics. See kubebuilder documentation for more details.","title":"Monitoring the CloudNativePG operator"},{"location":"monitoring/#monitoring-the-operator-with-prometheus","text":"The operator can be monitored using the Prometheus Operator by defining a PodMonitor pointing to the operator pod(s), as follows (note it's applied in the same namespace as the operator): kubectl -n cnpg-system apply -f - < 8080:8080 With port forwarding active, the metrics are easily viewable on a browser at localhost:8080/metrics .","title":"Using port forwarding"},{"location":"monitoring/#using-curl","text":"Create the curl pod with the following command: kubectl apply -f - <:9187/metrics For example, if your PostgreSQL cluster is called cluster-example and you want to retrieve the exported metrics of the first pod in the cluster, you can run the following command to programmatically get the IP of that pod: POD_IP=$(kubectl get pod cluster-example-1 --template '{{.status.podIP}}') And then run: kubectl exec -ti curl -- curl -s ${POD_IP}:9187/metrics If you enabled TLS metrics, run instead: kubectl exec -ti curl -- curl -sk https://${POD_IP}:9187/metrics To access the metrics of the operator, you need to point to the pod where the operator is running, and use TCP port 8080 as target. When you're done inspecting metrics, please remember to delete the curl pod: kubectl delete -f curl.yaml","title":"Using curl"},{"location":"monitoring/#auxiliary-resources","text":"Important These resources are provided for illustration and experimentation, and do not represent any kind of recommendation for your production system In the doc/src/samples/monitoring/ directory you will find a series of sample files for observability. Please refer to Part 4 of the quickstart section for context: kube-stack-config.yaml : a configuration file for the kube-stack helm chart installation. It ensures that Prometheus listens for all PodMonitor resources. prometheusrule.yaml : a PrometheusRule with alerts for CloudNativePG. NOTE: this does not include inter-operation with notification services. Please refer to the Prometheus documentation . podmonitor.yaml : a PodMonitor for the CloudNativePG Operator deployment. In addition, we provide the \"raw\" sources for the Prometheus alert rules in the alerts.yaml file. A Grafana dashboard for CloudNativePG clusters and operator, is kept in the dedicated repository cloudnative-pg/grafana-dashboards as a dashboard JSON configuration: grafana-dashboard.json . The file can be downloaded, and imported into Grafana (menus: Dashboard > New > Import). For a general reference on the settings available on kube-prometheus-stack , you can execute helm show values prometheus-community/kube-prometheus-stack . Please refer to the kube-prometheus-stack page for more detail.","title":"Auxiliary resources"},{"location":"networking/","text":"Networking CloudNativePG assumes the underlying Kubernetes cluster has the required connectivity already set up. Networking on Kubernetes is an important and extended topic; please refer to the Kubernetes documentation for further information. If you're following the quickstart guide to install CloudNativePG on a local KinD or K3d cluster, you should not encounter any networking issues as neither platform will add any networking restrictions by default. However, when deploying CloudNativePG on existing infrastructure, networking restrictions might be in place that could impair the communication of the operator with PostgreSQL clusters. Specifically, existing Network Policies might restrict certain types of traffic. Or, you might be interested in adding network policies in your environment for increased security. As mentioned in the security document , please ensure the operator can reach every cluster pod on ports 8000 and 5432, and that pods can connect to each other. Cross-namespace network policy for the operator Following the quickstart guide or using helm chart for deployment will install the operator in a dedicated namespace ( cnpg-system by default). We recommend that you create clusters in a different namespace. The operator must be able to connect to cluster pods. This might be precluded if there is a NetworkPolicy restricting cross-namespace traffic. For example, the kubernetes guide on network policies contains an example policy denying all ingress traffic by default. If your local kubernetes setup has this kind of restrictive network policy, you will need to create a NetworkPolicy to explicitly allow connection from the operator namespace and pod to the cluster namespace and pods. You can find an example in the networkpolicy-example.yaml file in this repository. Please note, you'll need to adjust the cluster name and cluster namespace to match your specific setup, and also the operator namespace if it is not the default namespace. Cross-cluster networking While bootstrapping from another cluster or when using the externalClusters section, ensure connectivity among all clusters, object stores, and namespaces involved. Again, we refer you to the Kubernetes documentation for setup information.","title":"Networking"},{"location":"networking/#networking","text":"CloudNativePG assumes the underlying Kubernetes cluster has the required connectivity already set up. Networking on Kubernetes is an important and extended topic; please refer to the Kubernetes documentation for further information. If you're following the quickstart guide to install CloudNativePG on a local KinD or K3d cluster, you should not encounter any networking issues as neither platform will add any networking restrictions by default. However, when deploying CloudNativePG on existing infrastructure, networking restrictions might be in place that could impair the communication of the operator with PostgreSQL clusters. Specifically, existing Network Policies might restrict certain types of traffic. Or, you might be interested in adding network policies in your environment for increased security. As mentioned in the security document , please ensure the operator can reach every cluster pod on ports 8000 and 5432, and that pods can connect to each other.","title":"Networking"},{"location":"networking/#cross-namespace-network-policy-for-the-operator","text":"Following the quickstart guide or using helm chart for deployment will install the operator in a dedicated namespace ( cnpg-system by default). We recommend that you create clusters in a different namespace. The operator must be able to connect to cluster pods. This might be precluded if there is a NetworkPolicy restricting cross-namespace traffic. For example, the kubernetes guide on network policies contains an example policy denying all ingress traffic by default. If your local kubernetes setup has this kind of restrictive network policy, you will need to create a NetworkPolicy to explicitly allow connection from the operator namespace and pod to the cluster namespace and pods. You can find an example in the networkpolicy-example.yaml file in this repository. Please note, you'll need to adjust the cluster name and cluster namespace to match your specific setup, and also the operator namespace if it is not the default namespace.","title":"Cross-namespace network policy for the operator"},{"location":"networking/#cross-cluster-networking","text":"While bootstrapping from another cluster or when using the externalClusters section, ensure connectivity among all clusters, object stores, and namespaces involved. Again, we refer you to the Kubernetes documentation for setup information.","title":"Cross-cluster networking"},{"location":"operator_capability_levels/","text":"Operator capability levels These capabilities were implemented by CloudNativePG, classified using the Operator SDK definition of Capability Levels framework. Important Based on the Operator Capability Levels model , you can expect a \"Level V - Auto Pilot\" set of capabilities from the CloudNativePG operator. Each capability level is associated with a certain set of management features the operator offers: Basic install Seamless upgrades Full lifecycle Deep insights Auto pilot Note We consider this framework as a guide for future work and implementations in the operator. Level 1: Basic install Capability level 1 involves installing and configuring the operator. This category includes usability and user experience enhancements, such as improvements in how you interact with the operator and a PostgreSQL cluster configuration. Important We consider information security part of this level. Operator deployment via declarative configuration The operator is installed in a declarative way using a Kubernetes manifest that defines four major CustomResourceDefinition objects: Cluster , Pooler , Backup , and ScheduledBackup . PostgreSQL cluster deployment via declarative configuration You define a PostgreSQL cluster (operand) using the Cluster custom resource in a fully declarative way. The PostgreSQL version is determined by the operand container image defined in the CR, which is automatically fetched from the requested registry. When deploying an operand, the operator also creates the following resources: Pod , Service , Secret , ConfigMap , PersistentVolumeClaim , PodDisruptionBudget , ServiceAccount , RoleBinding , and Role . Override of operand images through the CRD The operator is designed to support any operand container image with PostgreSQL inside. By default, the operator uses the latest available minor version of the latest stable major version supported by the PostgreSQL community and published on ghcr.io. You can use any compatible image of PostgreSQL supporting the primary/standby architecture directly by setting the imageName attribute in the CR. The operator also supports imagePullSecrets to access private container registries, and it supports digests and tags for finer control of container image immutability. If you prefer not to specify an image name, you can leverage image catalogs by simply referencing the PostgreSQL major version. Moreover, image catalogs enable you to effortlessly create custom catalogs, directing to images based on your specific requirements. Labels and annotations You can configure the operator to support inheriting labels and annotations that are defined in a cluster's metadata. The goal is to improve the organization of the CloudNativePG deployment in your Kubernetes infrastructure. Self-contained instance manager Instead of relying on an external tool to coordinate PostgreSQL instances in the Kubernetes cluster pods, such as Patroni or Stolon, the operator injects the operator executable inside each pod, in a file named /controller/manager . The application is used to control the underlying PostgreSQL instance and to reconcile the pod status with the instance based on the PostgreSQL cluster topology. The instance manager also starts a web server that's invoked by the kubelet for probes. Unix signals invoked by the kubelet are filtered by the instance manager. Where appropriate, they're forwarded to the postgres process for fast and controlled reactions to external events. The instance manager is written in Go and has no external dependencies. Storage configuration Storage is a critical component in a database workload. Taking advantage of the Kubernetes native capabilities and resources in terms of storage, the operator gives you enough flexibility to choose the right storage for your workload requirements, based on what the underlying Kubernetes environment can offer. This implies choosing a particular storage class in a public cloud environment or fine-tuning the generated PVC through a PVC template in the CR's storage parameter. For better performance and finer control, you can also choose to host your cluster's write-ahead log (WAL, also known as pg_wal ) on a separate volume, preferably on different storage. The \"Benchmarking\" section of the documentation provides detailed instructions on benchmarking both storage and the database before production. It relies on the cnpg plugin to ensure optimal performance and reliability. Replica configuration The operator detects replicas in a cluster through a single parameter, called instances . If set to 1 , the cluster comprises a single primary PostgreSQL instance with no replica. If higher than 1 , the operator manages instances -1 replicas, including high availability (HA) through automated failover and rolling updates through switchover operations. CloudNativePG manages replication slots for all the replicas in the HA cluster. The implementation is inspired by the previously proposed patch for PostgreSQL, called failover slots , and also supports user defined physical replication slots on the primary. Service Configuration By default, CloudNativePG creates three Kubernetes services for applications to access the cluster via the network: One pointing to the primary for read/write operations. One pointing to replicas for read-only queries. A generic one pointing to any instance for read operations. You can disable the read-only and read services via configuration. Additionally, you can leverage the service template capability to create custom service resources, including load balancers, to access PostgreSQL outside Kubernetes. This is particularly useful for DBaaS purposes. Database configuration The operator is designed to bootstrap a PostgreSQL cluster with a single database. The operator transparently manages network access to the cluster through three Kubernetes services provisioned and managed for read-write, read, and read-only workloads. Using the convention-over-configuration approach, the operator creates a database called app , by default owned by a regular Postgres user with the same name. You can specify both the database name and the user name, if required, as part of the bootstrap. Additional databases can be created or managed via declarative database management using the Database CRD. Although no configuration is required to run the cluster, you can customize both PostgreSQL runtime configuration and PostgreSQL host-based authentication rules in the postgresql section of the CR. Configuration of Postgres roles, users, and groups CloudNativePG supports management of PostgreSQL roles, users, and groups through declarative configuration using the .spec.managed.roles stanza. Pod security policies For InfoSec requirements, the operator doesn't require privileged mode for any container. It enforces a read-only root filesystem to guarantee containers immutability for both the operator and the operand pods. It also explicitly sets the required security contexts. Affinity The cluster's affinity section enables fine-tuning of how pods and related resources, such as persistent volumes, are scheduled across the nodes of a Kubernetes cluster. In particular, the operator supports: Pod affinity and anti-affinity Node selector Taints and tolerations Topology spread constraints The cluster's topologySpreadConstraints section enables additional control of scheduling pods across topologies, enhancing what affinity and anti-affinity can offer. Command-line interface CloudNativePG doesn't have its own command-line interface. It relies on the best command-line interface for Kubernetes, kubectl, by providing a plugin called cnpg . This plugin enhances and simplifies your PostgreSQL cluster management experience. Current status of the cluster The operator continuously updates the status section of the CR with the observed status of the cluster. The entire PostgreSQL cluster status is continuously monitored by the instance manager running in each pod. The instance manager is responsible for applying the required changes to the controlled PostgreSQL instance to converge to the required status of the cluster. (For example, if the cluster status reports that pod -1 is the primary, pod -1 needs to promote itself while the other pods need to follow pod -1 .) The same status is used by the cnpg plugin for kubectl to provide details. Operator's certification authority The operator creates a certification authority for itself. It creates and signs with the operator certification authority a leaf certificate for the webhook server to use. This certificate ensures safe communication between the Kubernetes API server and the operator. Cluster's certification authority The operator creates a certification authority for every PostgreSQL cluster. This certification authority is used to issue and renew TLS certificates for clients' authentication, including streaming replication standby servers (instead of passwords). Support for a custom certification authority for client certificates is available through secrets, which also includes integration with cert-manager. Certificates can be issued with the cnpg plugin for kubectl. TLS connections The operator transparently and natively supports TLS/SSL connections to encrypt client/server communications for increased security using the cluster's certification authority. Support for custom server certificates is available through secrets, which also includes integration with cert-manager. Certificate authentication for streaming replication To authorize streaming replication connections from the standby servers, the operator relies on TLS client certificate authentication. This method is used instead of relying on a password (and therefore a secret). Continuous configuration management The operator enables you to apply changes to the Cluster resource YAML section of the PostgreSQL configuration. Depending on the configuration option, it also makes sure that all instances are properly reloaded or restarted. Note Changes with ALTER SYSTEM aren't detected, meaning that the cluster state isn't enforced. Import of existing PostgreSQL databases The operator provides a declarative way to import existing Postgres databases in a new CloudNativePG cluster in Kubernetes, using offline migrations. The same feature also covers offline major upgrades of PostgreSQL databases. Offline means that applications must stop their write operations at the source until the database is imported. The feature extends the initdb bootstrap method to create a new PostgreSQL cluster using a logical snapshot of the data available in another PostgreSQL database. This data can be accessed by way of the network through a superuser connection. Import is from any supported version of Postgres. It relies on pg_dump and pg_restore being executed from the new cluster primary for all databases that are part of the operation and, if requested, for roles. PostGIS clusters CloudNativePG supports the installation of clusters with the PostGIS open source extension for geographical databases. This extension is one of the most popular extensions for PostgreSQL. Basic LDAP authentication for PostgreSQL The operator allows you to configure LDAP authentication for your PostgreSQL clients, using either the simple bind or search+bind mode, as described in the LDAP authentication section of the PostgreSQL documentation . Multiple installation methods The operator can be installed through a Kubernetes manifest by way of kubectl apply , to be used in a traditional Kubernetes installation in public and private cloud environments. CloudNativePG also supports installation by way of a Helm chart or OLM bundle from OperatorHub.io. Convention over configuration The operator supports the convention-over-configuration paradigm, deciding standard default values while allowing you to override them and customize them. You can specify a deployment of a PostgreSQL cluster using the Cluster CRD in a couple of lines of YAML code. Level 2: Seamless upgrades Capability level 2 is about enabling updates of the operator and the actual workload, in this case PostgreSQL servers. This includes PostgreSQL minor release updates (security and bug fixes normally) as well as major online upgrades. Operator Upgrade Upgrading the operator is seamless and can be done as a new deployment. After upgrading the controller, a rolling update of all deployed PostgreSQL clusters is initiated. You can choose to update all clusters simultaneously or distribute their upgrades over time. Thanks to the instance manager's injection, upgrading the operator does not require changes to the operand, allowing the operator to manage older versions of it. Additionally, CloudNativePG supports in-place updates of the instance manager following an operator upgrade. In-place updates do not require a rolling update or a subsequent switchover of the cluster. Upgrade of the managed workload The operand can be upgraded using a declarative configuration approach as part of changing the CR and, in particular, the imageName parameter. The operator prevents major upgrades of PostgreSQL while making it possible to go in both directions in terms of minor PostgreSQL releases within a major version, enabling updates and rollbacks. In the presence of standby servers, the operator performs rolling updates starting from the replicas. It does this by dropping the existing pod and creating a new one with the new requested operand image that reuses the underlying storage. Depending on the value of the primaryUpdateStrategy , the operator proceeds with a switchover before updating the former primary ( unsupervised ). Or, it waits for the user to manually issue the switchover procedure ( supervised ) by way of the cnpg plugin for kubectl. The setting to use depends on the business requirements, as the operation might generate some downtime for the applications. This downtime can range from a few seconds to minutes, based on the actual database workload. Display cluster availability status during upgrade At any time, convey the cluster's high availability status, for example, Setting up primary , Creating a new replica , Cluster in healthy state , Switchover in progress , Failing over , and Upgrading cluster . Level 3: Full lifecycle Capability level 3 requires the operator to manage aspects of business continuity and scalability. Disaster recovery is a business continuity component that requires that both backup and recovery of a database work correctly. While as a starting point, the goal is to achieve RPO < 5 minutes, the long-term goal is to implement RPO=0 backup solutions. High availability is the other important component of business continuity. Through PostgreSQL native physical replication and hot standby replicas, it allows the operator to perform failover and switchover operations. This area includes enhancements in: Control of PostgreSQL physical replication, such as synchronous replication, (cascading) replication clusters, and so on Connection pooling, to improve performance and control through a connection pooling layer with pgBouncer PostgreSQL WAL archive The operator supports PostgreSQL continuous archiving of WAL files to an object store (AWS S3 and S3-compatible, Azure Blob Storage, Google Cloud Storage, and gateways like MinIO). WAL archiving is defined at the cluster level, declaratively, through the backup parameter in the cluster definition. This is done by specifying an S3 protocol destination URL (for example, to point to a specific folder in an AWS S3 bucket) and, optionally, a generic endpoint URL. WAL archiving, a prerequisite for continuous backup, doesn't require any further user action. The operator transparently sets the archive_command to rely on barman-cloud-wal-archive to ship WAL files to the defined endpoint. You can decide the compression algorithm, as well as the number of parallel jobs to concurrently upload WAL files in the archive. In addition, Instance Manager checks the correctness of the archive destination by performing the barman-cloud-check-wal-archive command before beginning to ship the first set of WAL files. PostgreSQL backups The operator was designed to provide application-level backups using PostgreSQL\u2019s native continuous hot backup technology based on physical base backups and continuous WAL archiving. Base backups can be saved on: Kubernetes volume snapshots Object stores (AWS S3 and S3-compatible, Azure Blob Storage, Google Cloud Storage, and gateways like MinIO) Base backups are defined at the cluster level, declaratively, through the backup parameter in the cluster definition. You can define base backups in two ways: On-demand, through the Backup custom resource definition Scheduled, through the ScheduledBackup custom resource definition, using a cron-like syntax Volume snapshots rely directly on the Kubernetes API, which delegates this capability to the underlying storage classes and CSI drivers. Volume snapshot backups are suitable for very large database (VLDB) contexts. Object store backups rely on barman-cloud-backup for the job (distributed as part of the application container image) to relay backups in the same endpoint, alongside WAL files. Both barman-cloud-wal-restore and barman-cloud-backup are distributed in the application container image under GNU GPL 3 terms. Object store backups and volume snapshot backups are taken while PostgreSQL is up and running (hot backups). Volume snapshots also support taking consistent database snapshots with cold backups. Backups from a standby The operator supports offloading base backups onto a standby without impacting the RPO of the database. This allows resources to be preserved on the primary, in particular I/O, for standard database operations. Full restore from a backup The operator enables you to bootstrap a new cluster (with its settings) starting from an existing and accessible backup, either on a volume snapshot or in an object store. Once the bootstrap process is completed, the operator initiates the instance in recovery mode. It replays all available WAL files from the specified archive, exiting recovery and starting as a primary. Subsequently, the operator clones the requested number of standby instances from the primary. CloudNativePG supports parallel WAL fetching from the archive. Point-in-time recovery (PITR) from a backup The operator enables you to create a new PostgreSQL cluster by recovering an existing backup to a specific point in time, defined with a timestamp, a label, or a transaction ID. This capability is built on top of the full restore one and supports all the options available in PostgreSQL for PITR . Zero-Data-Loss Clusters Through Synchronous Replication Achieve zero data loss (RPO=0) in your local high-availability CloudNativePG cluster with support for both quorum-based and priority-based synchronous replication. The operator offers a flexible way to define the number of expected synchronous standby replicas available at any time, and allows customization of the synchronous_standby_names option as needed. Replica clusters Establish a robust cross-Kubernetes cluster topology for PostgreSQL clusters, harnessing the power of native streaming and cascading replication. With the replica option, you can configure an autonomous cluster to consistently replicate data from another PostgreSQL source of the same major version. This source can be located anywhere, provided you have access to a WAL archive for fetching WAL files or a direct streaming connection via TLS between the two endpoints. Notably, the source PostgreSQL instance can exist outside the Kubernetes environment, whether in a physical or virtual setting. Replica clusters can be instantiated through various methods, including volume snapshots, a recovery object store (using the Barman Cloud backup format), or streaming using pg_basebackup . Both WAL file shipping and WAL streaming are supported. The deployment of replica clusters significantly elevates the business continuity posture of PostgreSQL databases within Kubernetes, extending across multiple data centers and facilitating hybrid and multi-cloud setups. (While anticipating Kubernetes federation native capabilities, manual switchover across data centers remains necessary.) Additionally, the flexibility extends to creating delayed replica clusters intentionally lagging behind the primary cluster. This intentional lag aims to minimize the Recovery Time Objective ( RTO ) in the event of unintended errors, such as incorrect DELETE or UPDATE SQL operations. Distributed Database Topologies Leverage replica clusters to define distributed database topologies for PostgreSQL that span across various Kubernetes clusters, facilitating hybrid and multi-cloud deployments. With CloudNativePG, you gain powerful capabilities, including: Declarative Primary Control : Easily specify which PostgreSQL cluster acts as the primary. Seamless Primary Switchover : Effortlessly demote the current primary and promote another PostgreSQL cluster, typically located in a different region, without needing to re-clone the former primary. This setup can efficiently operate across two or more regions, can rely entirely on object stores for replication, and guarantees a maximum RPO (Recovery Point Objective) of 5 minutes. This advanced feature is uniquely provided by CloudNativePG, ensuring robust data integrity and continuity across diverse environments. Tablespace support CloudNativePG seamlessly integrates robust support for PostgreSQL tablespaces by facilitating the declarative definition of individual persistent volumes. This innovative feature empowers you to efficiently distribute I/O operations across a diverse array of storage devices. Through the transparent orchestration of tablespaces, CloudNativePG enhances the performance and scalability of PostgreSQL databases, ensuring a streamlined and optimized experience for managing large scale data storage in cloud-native environments. Support for temporary tablespaces is also included. Startup, Liveness, and Readiness Probes CloudNativePG configures startup, liveness, and readiness probes for PostgreSQL containers, which are managed by the Kubernetes kubelet. These probes interact with the /healthz and /readyz endpoints exposed by the instance manager's web server to monitor the Pod's health and readiness. The startup and liveness probes use the pg_isready utility. A Pod is considered healthy if pg_isready returns an exit code of 0 (indicating the server is accepting connections) or 1 (indicating the server is rejecting connections, such as during startup). The readiness probe executes a simple SQL query ( ; ) to verify that the PostgreSQL server is ready to accept client connections. All probes are configured with default settings but can be fully customized to meet specific needs, allowing for fine-tuning to align with your environment and workloads. Rolling deployments The operator supports rolling deployments to minimize the downtime. If a PostgreSQL cluster is exposed publicly, the service load-balances the read-only traffic only to available pods during the initialization or the update. Scale up and down of replicas The operator allows you to scale up and down the number of instances in a PostgreSQL cluster. New replicas are started up from the primary server and participate in the cluster's HA infrastructure. The CRD declares a \"scale\" subresource that allows you to use the kubectl scale command. Maintenance window and PodDisruptionBudget for Kubernetes nodes The operator creates a PodDisruptionBudget resource to limit the number of concurrent disruptions to one primary instance. This configuration prevents the maintenance operation from deleting all the pods in a cluster, allowing the specified number of instances to be created. The PodDisruptionBudget is applied during the node-draining operation, preventing any disruption of the cluster service. While this strategy is correct for Kubernetes clusters where storage is shared among all the worker nodes, it might not be the best solution for clusters using local storage or for clusters installed in a private cloud. The operator allows you to specify a maintenance window and configure the reaction to any underlying node eviction. The ReusePVC option in the maintenance window section enables to specify the strategy to use. Allocate new storage in a different PVC for the evicted instance, or wait for the underlying node to be available again. Fencing Fencing is the process of protecting the data in one, more, or even all instances of a PostgreSQL cluster when they appear to be malfunctioning. When an instance is fenced, the PostgreSQL server process is guaranteed to be shut down, while the pod is kept running. This ensures that, until the fence is lifted, data on the pod isn't modified by PostgreSQL and that you can investigate file system for debugging and troubleshooting purposes. Hibernation (declarative) CloudNativePG supports hibernation of a running PostgreSQL cluster in a declarative manner, through the cnpg.io/hibernation annotation. Hibernation enables saving CPU power by removing the database pods while keeping the database PVCs. This feature simulates scaling to 0 instances. Hibernation (imperative) CloudNativePG supports hibernation of a running PostgreSQL cluster by way of the cnpg plugin. Hibernation shuts down all Postgres instances in the high-availability cluster and keeps a static copy of the PVC group of the primary. The copy contains PGDATA and WALs. The plugin enables you to exit the hibernation phase by resuming the primary and then recreating all the replicas, if they exist. Reuse of persistent volumes storage in pods When the operator needs to create a pod that was deleted by the user or was evicted by a Kubernetes maintenance operation, it reuses the PersistentVolumeClaim , if available. This ability avoids the need to clone the data from the primary again. CPU and memory requests and limits The operator allows administrators to control and manage resource usage by the cluster's pods in the resources section of the manifest. In particular, you can set requests and limits values for both CPU and RAM. Connection pooling with PgBouncer CloudNativePG provides native support for connection pooling with PgBouncer , one of the most popular open source connection poolers for PostgreSQL. From an architectural point of view, the native implementation of a PgBouncer connection pooler introduces a new layer to access the database. This optimizes the query flow toward the instances and makes the use of the underlying PostgreSQL resources more efficient. Instead of connecting directly to a PostgreSQL service, applications can now connect to the PgBouncer service and start reusing any existing connection. Logical Replication CloudNativePG supports PostgreSQL's logical replication in a declarative manner using Publication and Subscription custom resource definitions. Logical replication is particularly useful together with the import facility for online data migrations (even from public DBaaS solutions) and major PostgreSQL upgrades. Level 4: Deep insights Capability level 4 is about observability : monitoring, alerting, trending, and log processing. This might involve the use of external tools, such as Prometheus, Grafana, and Fluent Bit, as well as extensions in the PostgreSQL engine for the output of error logs directly in JSON format. CloudNativePG was designed to provide everything needed to easily integrate with industry-standard and community-accepted tools for flexible monitoring and logging. Prometheus exporter with configurable queries The instance manager provides a pluggable framework. By way of its own web server listening on the metrics port (9187), it exposes an endpoint to export metrics for the Prometheus monitoring and alerting tool. The operator supports custom monitoring queries defined as ConfigMap or Secret objects using a syntax that's compatible with postgres_exporter for Prometheus . CloudNativePG provides a set of basic monitoring queries for PostgreSQL that can be integrated and adapted to your context. Grafana dashboard CloudNativePG comes with a Grafana dashboard that you can use as a base to monitor all critical aspects of a PostgreSQL cluster, and customize. Standard output logging of PostgreSQL error messages in JSON format Every log message is delivered to standard output in JSON format. The first level is the definition of the timestamp, the log level, and the type of log entry, such as postgres for the canonical PostgreSQL error message channel. As a result, every pod managed by CloudNativePG can be easily and directly integrated with any downstream log processing stack that supports JSON as source data type. Real-time query monitoring CloudNativePG transparently and natively supports: The essential pg_stat_statements extension , which enables tracking of planning and execution statistics of all SQL statements executed by a PostgreSQL server The auto_explain extension , which provides a means for logging execution plans of slow statements automatically, without having to manually run EXPLAIN (helpful for tracking down un-optimized queries) The pg_failover_slots extension , which makes logical replication slots usable across a physical failover, ensuring resilience in change data capture (CDC) contexts based on PostgreSQL's native logical replication Audit CloudNativePG allows database and security administrators, auditors, and operators to track and analyze database activities using PGAudit for PostgreSQL. Such activities flow directly in the JSON log and can be properly routed to the correct downstream target using common log brokers like Fluentd. Kubernetes events Record major events as expected by the Kubernetes API, such as creating resources, removing nodes, and upgrading. Events can be displayed by using the kubectl describe and kubectl get events commands. Level 5: Auto pilot Capability level 5 is focused on automated scaling, healing, and tuning through the discovery of anomalies and insights that emerged from the observability layer. Automated failover for self-healing In case of detected failure on the primary, the operator changes the status of the cluster by setting the most aligned replica as the new target primary. As a consequence, the instance manager in each alive pod initiates the required procedures to align itself with the requested status of the cluster. It does this by either becoming the new primary or by following it. In case the former primary comes back up, the same mechanism avoids a split-brain by preventing applications from reaching it, running pg_rewind on the server and restarting it as a standby. Automated recreation of a standby If the pod hosting a standby is removed, the operator initiates the procedure to re-create a standby server.","title":"Operator capability levels"},{"location":"operator_capability_levels/#operator-capability-levels","text":"These capabilities were implemented by CloudNativePG, classified using the Operator SDK definition of Capability Levels framework. Important Based on the Operator Capability Levels model , you can expect a \"Level V - Auto Pilot\" set of capabilities from the CloudNativePG operator. Each capability level is associated with a certain set of management features the operator offers: Basic install Seamless upgrades Full lifecycle Deep insights Auto pilot Note We consider this framework as a guide for future work and implementations in the operator.","title":"Operator capability levels"},{"location":"operator_capability_levels/#level-1-basic-install","text":"Capability level 1 involves installing and configuring the operator. This category includes usability and user experience enhancements, such as improvements in how you interact with the operator and a PostgreSQL cluster configuration. Important We consider information security part of this level.","title":"Level 1: Basic install"},{"location":"operator_capability_levels/#operator-deployment-via-declarative-configuration","text":"The operator is installed in a declarative way using a Kubernetes manifest that defines four major CustomResourceDefinition objects: Cluster , Pooler , Backup , and ScheduledBackup .","title":"Operator deployment via declarative configuration"},{"location":"operator_capability_levels/#postgresql-cluster-deployment-via-declarative-configuration","text":"You define a PostgreSQL cluster (operand) using the Cluster custom resource in a fully declarative way. The PostgreSQL version is determined by the operand container image defined in the CR, which is automatically fetched from the requested registry. When deploying an operand, the operator also creates the following resources: Pod , Service , Secret , ConfigMap , PersistentVolumeClaim , PodDisruptionBudget , ServiceAccount , RoleBinding , and Role .","title":"PostgreSQL cluster deployment via declarative configuration"},{"location":"operator_capability_levels/#override-of-operand-images-through-the-crd","text":"The operator is designed to support any operand container image with PostgreSQL inside. By default, the operator uses the latest available minor version of the latest stable major version supported by the PostgreSQL community and published on ghcr.io. You can use any compatible image of PostgreSQL supporting the primary/standby architecture directly by setting the imageName attribute in the CR. The operator also supports imagePullSecrets to access private container registries, and it supports digests and tags for finer control of container image immutability. If you prefer not to specify an image name, you can leverage image catalogs by simply referencing the PostgreSQL major version. Moreover, image catalogs enable you to effortlessly create custom catalogs, directing to images based on your specific requirements.","title":"Override of operand images through the CRD"},{"location":"operator_capability_levels/#labels-and-annotations","text":"You can configure the operator to support inheriting labels and annotations that are defined in a cluster's metadata. The goal is to improve the organization of the CloudNativePG deployment in your Kubernetes infrastructure.","title":"Labels and annotations"},{"location":"operator_capability_levels/#self-contained-instance-manager","text":"Instead of relying on an external tool to coordinate PostgreSQL instances in the Kubernetes cluster pods, such as Patroni or Stolon, the operator injects the operator executable inside each pod, in a file named /controller/manager . The application is used to control the underlying PostgreSQL instance and to reconcile the pod status with the instance based on the PostgreSQL cluster topology. The instance manager also starts a web server that's invoked by the kubelet for probes. Unix signals invoked by the kubelet are filtered by the instance manager. Where appropriate, they're forwarded to the postgres process for fast and controlled reactions to external events. The instance manager is written in Go and has no external dependencies.","title":"Self-contained instance manager"},{"location":"operator_capability_levels/#storage-configuration","text":"Storage is a critical component in a database workload. Taking advantage of the Kubernetes native capabilities and resources in terms of storage, the operator gives you enough flexibility to choose the right storage for your workload requirements, based on what the underlying Kubernetes environment can offer. This implies choosing a particular storage class in a public cloud environment or fine-tuning the generated PVC through a PVC template in the CR's storage parameter. For better performance and finer control, you can also choose to host your cluster's write-ahead log (WAL, also known as pg_wal ) on a separate volume, preferably on different storage. The \"Benchmarking\" section of the documentation provides detailed instructions on benchmarking both storage and the database before production. It relies on the cnpg plugin to ensure optimal performance and reliability.","title":"Storage configuration"},{"location":"operator_capability_levels/#replica-configuration","text":"The operator detects replicas in a cluster through a single parameter, called instances . If set to 1 , the cluster comprises a single primary PostgreSQL instance with no replica. If higher than 1 , the operator manages instances -1 replicas, including high availability (HA) through automated failover and rolling updates through switchover operations. CloudNativePG manages replication slots for all the replicas in the HA cluster. The implementation is inspired by the previously proposed patch for PostgreSQL, called failover slots , and also supports user defined physical replication slots on the primary.","title":"Replica configuration"},{"location":"operator_capability_levels/#service-configuration","text":"By default, CloudNativePG creates three Kubernetes services for applications to access the cluster via the network: One pointing to the primary for read/write operations. One pointing to replicas for read-only queries. A generic one pointing to any instance for read operations. You can disable the read-only and read services via configuration. Additionally, you can leverage the service template capability to create custom service resources, including load balancers, to access PostgreSQL outside Kubernetes. This is particularly useful for DBaaS purposes.","title":"Service Configuration"},{"location":"operator_capability_levels/#database-configuration","text":"The operator is designed to bootstrap a PostgreSQL cluster with a single database. The operator transparently manages network access to the cluster through three Kubernetes services provisioned and managed for read-write, read, and read-only workloads. Using the convention-over-configuration approach, the operator creates a database called app , by default owned by a regular Postgres user with the same name. You can specify both the database name and the user name, if required, as part of the bootstrap. Additional databases can be created or managed via declarative database management using the Database CRD. Although no configuration is required to run the cluster, you can customize both PostgreSQL runtime configuration and PostgreSQL host-based authentication rules in the postgresql section of the CR.","title":"Database configuration"},{"location":"operator_capability_levels/#configuration-of-postgres-roles-users-and-groups","text":"CloudNativePG supports management of PostgreSQL roles, users, and groups through declarative configuration using the .spec.managed.roles stanza.","title":"Configuration of Postgres roles, users, and groups"},{"location":"operator_capability_levels/#pod-security-policies","text":"For InfoSec requirements, the operator doesn't require privileged mode for any container. It enforces a read-only root filesystem to guarantee containers immutability for both the operator and the operand pods. It also explicitly sets the required security contexts.","title":"Pod security policies"},{"location":"operator_capability_levels/#affinity","text":"The cluster's affinity section enables fine-tuning of how pods and related resources, such as persistent volumes, are scheduled across the nodes of a Kubernetes cluster. In particular, the operator supports: Pod affinity and anti-affinity Node selector Taints and tolerations","title":"Affinity"},{"location":"operator_capability_levels/#topology-spread-constraints","text":"The cluster's topologySpreadConstraints section enables additional control of scheduling pods across topologies, enhancing what affinity and anti-affinity can offer.","title":"Topology spread constraints"},{"location":"operator_capability_levels/#command-line-interface","text":"CloudNativePG doesn't have its own command-line interface. It relies on the best command-line interface for Kubernetes, kubectl, by providing a plugin called cnpg . This plugin enhances and simplifies your PostgreSQL cluster management experience.","title":"Command-line interface"},{"location":"operator_capability_levels/#current-status-of-the-cluster","text":"The operator continuously updates the status section of the CR with the observed status of the cluster. The entire PostgreSQL cluster status is continuously monitored by the instance manager running in each pod. The instance manager is responsible for applying the required changes to the controlled PostgreSQL instance to converge to the required status of the cluster. (For example, if the cluster status reports that pod -1 is the primary, pod -1 needs to promote itself while the other pods need to follow pod -1 .) The same status is used by the cnpg plugin for kubectl to provide details.","title":"Current status of the cluster"},{"location":"operator_capability_levels/#operators-certification-authority","text":"The operator creates a certification authority for itself. It creates and signs with the operator certification authority a leaf certificate for the webhook server to use. This certificate ensures safe communication between the Kubernetes API server and the operator.","title":"Operator's certification authority"},{"location":"operator_capability_levels/#clusters-certification-authority","text":"The operator creates a certification authority for every PostgreSQL cluster. This certification authority is used to issue and renew TLS certificates for clients' authentication, including streaming replication standby servers (instead of passwords). Support for a custom certification authority for client certificates is available through secrets, which also includes integration with cert-manager. Certificates can be issued with the cnpg plugin for kubectl.","title":"Cluster's certification authority"},{"location":"operator_capability_levels/#tls-connections","text":"The operator transparently and natively supports TLS/SSL connections to encrypt client/server communications for increased security using the cluster's certification authority. Support for custom server certificates is available through secrets, which also includes integration with cert-manager.","title":"TLS connections"},{"location":"operator_capability_levels/#certificate-authentication-for-streaming-replication","text":"To authorize streaming replication connections from the standby servers, the operator relies on TLS client certificate authentication. This method is used instead of relying on a password (and therefore a secret).","title":"Certificate authentication for streaming replication"},{"location":"operator_capability_levels/#continuous-configuration-management","text":"The operator enables you to apply changes to the Cluster resource YAML section of the PostgreSQL configuration. Depending on the configuration option, it also makes sure that all instances are properly reloaded or restarted. Note Changes with ALTER SYSTEM aren't detected, meaning that the cluster state isn't enforced.","title":"Continuous configuration management"},{"location":"operator_capability_levels/#import-of-existing-postgresql-databases","text":"The operator provides a declarative way to import existing Postgres databases in a new CloudNativePG cluster in Kubernetes, using offline migrations. The same feature also covers offline major upgrades of PostgreSQL databases. Offline means that applications must stop their write operations at the source until the database is imported. The feature extends the initdb bootstrap method to create a new PostgreSQL cluster using a logical snapshot of the data available in another PostgreSQL database. This data can be accessed by way of the network through a superuser connection. Import is from any supported version of Postgres. It relies on pg_dump and pg_restore being executed from the new cluster primary for all databases that are part of the operation and, if requested, for roles.","title":"Import of existing PostgreSQL databases"},{"location":"operator_capability_levels/#postgis-clusters","text":"CloudNativePG supports the installation of clusters with the PostGIS open source extension for geographical databases. This extension is one of the most popular extensions for PostgreSQL.","title":"PostGIS clusters"},{"location":"operator_capability_levels/#basic-ldap-authentication-for-postgresql","text":"The operator allows you to configure LDAP authentication for your PostgreSQL clients, using either the simple bind or search+bind mode, as described in the LDAP authentication section of the PostgreSQL documentation .","title":"Basic LDAP authentication for PostgreSQL"},{"location":"operator_capability_levels/#multiple-installation-methods","text":"The operator can be installed through a Kubernetes manifest by way of kubectl apply , to be used in a traditional Kubernetes installation in public and private cloud environments. CloudNativePG also supports installation by way of a Helm chart or OLM bundle from OperatorHub.io.","title":"Multiple installation methods"},{"location":"operator_capability_levels/#convention-over-configuration","text":"The operator supports the convention-over-configuration paradigm, deciding standard default values while allowing you to override them and customize them. You can specify a deployment of a PostgreSQL cluster using the Cluster CRD in a couple of lines of YAML code.","title":"Convention over configuration"},{"location":"operator_capability_levels/#level-2-seamless-upgrades","text":"Capability level 2 is about enabling updates of the operator and the actual workload, in this case PostgreSQL servers. This includes PostgreSQL minor release updates (security and bug fixes normally) as well as major online upgrades.","title":"Level 2: Seamless upgrades"},{"location":"operator_capability_levels/#operator-upgrade","text":"Upgrading the operator is seamless and can be done as a new deployment. After upgrading the controller, a rolling update of all deployed PostgreSQL clusters is initiated. You can choose to update all clusters simultaneously or distribute their upgrades over time. Thanks to the instance manager's injection, upgrading the operator does not require changes to the operand, allowing the operator to manage older versions of it. Additionally, CloudNativePG supports in-place updates of the instance manager following an operator upgrade. In-place updates do not require a rolling update or a subsequent switchover of the cluster.","title":"Operator Upgrade"},{"location":"operator_capability_levels/#upgrade-of-the-managed-workload","text":"The operand can be upgraded using a declarative configuration approach as part of changing the CR and, in particular, the imageName parameter. The operator prevents major upgrades of PostgreSQL while making it possible to go in both directions in terms of minor PostgreSQL releases within a major version, enabling updates and rollbacks. In the presence of standby servers, the operator performs rolling updates starting from the replicas. It does this by dropping the existing pod and creating a new one with the new requested operand image that reuses the underlying storage. Depending on the value of the primaryUpdateStrategy , the operator proceeds with a switchover before updating the former primary ( unsupervised ). Or, it waits for the user to manually issue the switchover procedure ( supervised ) by way of the cnpg plugin for kubectl. The setting to use depends on the business requirements, as the operation might generate some downtime for the applications. This downtime can range from a few seconds to minutes, based on the actual database workload.","title":"Upgrade of the managed workload"},{"location":"operator_capability_levels/#display-cluster-availability-status-during-upgrade","text":"At any time, convey the cluster's high availability status, for example, Setting up primary , Creating a new replica , Cluster in healthy state , Switchover in progress , Failing over , and Upgrading cluster .","title":"Display cluster availability status during upgrade"},{"location":"operator_capability_levels/#level-3-full-lifecycle","text":"Capability level 3 requires the operator to manage aspects of business continuity and scalability. Disaster recovery is a business continuity component that requires that both backup and recovery of a database work correctly. While as a starting point, the goal is to achieve RPO < 5 minutes, the long-term goal is to implement RPO=0 backup solutions. High availability is the other important component of business continuity. Through PostgreSQL native physical replication and hot standby replicas, it allows the operator to perform failover and switchover operations. This area includes enhancements in: Control of PostgreSQL physical replication, such as synchronous replication, (cascading) replication clusters, and so on Connection pooling, to improve performance and control through a connection pooling layer with pgBouncer","title":"Level 3: Full lifecycle"},{"location":"operator_capability_levels/#postgresql-wal-archive","text":"The operator supports PostgreSQL continuous archiving of WAL files to an object store (AWS S3 and S3-compatible, Azure Blob Storage, Google Cloud Storage, and gateways like MinIO). WAL archiving is defined at the cluster level, declaratively, through the backup parameter in the cluster definition. This is done by specifying an S3 protocol destination URL (for example, to point to a specific folder in an AWS S3 bucket) and, optionally, a generic endpoint URL. WAL archiving, a prerequisite for continuous backup, doesn't require any further user action. The operator transparently sets the archive_command to rely on barman-cloud-wal-archive to ship WAL files to the defined endpoint. You can decide the compression algorithm, as well as the number of parallel jobs to concurrently upload WAL files in the archive. In addition, Instance Manager checks the correctness of the archive destination by performing the barman-cloud-check-wal-archive command before beginning to ship the first set of WAL files.","title":"PostgreSQL WAL archive"},{"location":"operator_capability_levels/#postgresql-backups","text":"The operator was designed to provide application-level backups using PostgreSQL\u2019s native continuous hot backup technology based on physical base backups and continuous WAL archiving. Base backups can be saved on: Kubernetes volume snapshots Object stores (AWS S3 and S3-compatible, Azure Blob Storage, Google Cloud Storage, and gateways like MinIO) Base backups are defined at the cluster level, declaratively, through the backup parameter in the cluster definition. You can define base backups in two ways: On-demand, through the Backup custom resource definition Scheduled, through the ScheduledBackup custom resource definition, using a cron-like syntax Volume snapshots rely directly on the Kubernetes API, which delegates this capability to the underlying storage classes and CSI drivers. Volume snapshot backups are suitable for very large database (VLDB) contexts. Object store backups rely on barman-cloud-backup for the job (distributed as part of the application container image) to relay backups in the same endpoint, alongside WAL files. Both barman-cloud-wal-restore and barman-cloud-backup are distributed in the application container image under GNU GPL 3 terms. Object store backups and volume snapshot backups are taken while PostgreSQL is up and running (hot backups). Volume snapshots also support taking consistent database snapshots with cold backups.","title":"PostgreSQL backups"},{"location":"operator_capability_levels/#backups-from-a-standby","text":"The operator supports offloading base backups onto a standby without impacting the RPO of the database. This allows resources to be preserved on the primary, in particular I/O, for standard database operations.","title":"Backups from a standby"},{"location":"operator_capability_levels/#full-restore-from-a-backup","text":"The operator enables you to bootstrap a new cluster (with its settings) starting from an existing and accessible backup, either on a volume snapshot or in an object store. Once the bootstrap process is completed, the operator initiates the instance in recovery mode. It replays all available WAL files from the specified archive, exiting recovery and starting as a primary. Subsequently, the operator clones the requested number of standby instances from the primary. CloudNativePG supports parallel WAL fetching from the archive.","title":"Full restore from a backup"},{"location":"operator_capability_levels/#point-in-time-recovery-pitr-from-a-backup","text":"The operator enables you to create a new PostgreSQL cluster by recovering an existing backup to a specific point in time, defined with a timestamp, a label, or a transaction ID. This capability is built on top of the full restore one and supports all the options available in PostgreSQL for PITR .","title":"Point-in-time recovery (PITR) from a backup"},{"location":"operator_capability_levels/#zero-data-loss-clusters-through-synchronous-replication","text":"Achieve zero data loss (RPO=0) in your local high-availability CloudNativePG cluster with support for both quorum-based and priority-based synchronous replication. The operator offers a flexible way to define the number of expected synchronous standby replicas available at any time, and allows customization of the synchronous_standby_names option as needed.","title":"Zero-Data-Loss Clusters Through Synchronous Replication"},{"location":"operator_capability_levels/#replica-clusters","text":"Establish a robust cross-Kubernetes cluster topology for PostgreSQL clusters, harnessing the power of native streaming and cascading replication. With the replica option, you can configure an autonomous cluster to consistently replicate data from another PostgreSQL source of the same major version. This source can be located anywhere, provided you have access to a WAL archive for fetching WAL files or a direct streaming connection via TLS between the two endpoints. Notably, the source PostgreSQL instance can exist outside the Kubernetes environment, whether in a physical or virtual setting. Replica clusters can be instantiated through various methods, including volume snapshots, a recovery object store (using the Barman Cloud backup format), or streaming using pg_basebackup . Both WAL file shipping and WAL streaming are supported. The deployment of replica clusters significantly elevates the business continuity posture of PostgreSQL databases within Kubernetes, extending across multiple data centers and facilitating hybrid and multi-cloud setups. (While anticipating Kubernetes federation native capabilities, manual switchover across data centers remains necessary.) Additionally, the flexibility extends to creating delayed replica clusters intentionally lagging behind the primary cluster. This intentional lag aims to minimize the Recovery Time Objective ( RTO ) in the event of unintended errors, such as incorrect DELETE or UPDATE SQL operations.","title":"Replica clusters"},{"location":"operator_capability_levels/#distributed-database-topologies","text":"Leverage replica clusters to define distributed database topologies for PostgreSQL that span across various Kubernetes clusters, facilitating hybrid and multi-cloud deployments. With CloudNativePG, you gain powerful capabilities, including: Declarative Primary Control : Easily specify which PostgreSQL cluster acts as the primary. Seamless Primary Switchover : Effortlessly demote the current primary and promote another PostgreSQL cluster, typically located in a different region, without needing to re-clone the former primary. This setup can efficiently operate across two or more regions, can rely entirely on object stores for replication, and guarantees a maximum RPO (Recovery Point Objective) of 5 minutes. This advanced feature is uniquely provided by CloudNativePG, ensuring robust data integrity and continuity across diverse environments.","title":"Distributed Database Topologies"},{"location":"operator_capability_levels/#tablespace-support","text":"CloudNativePG seamlessly integrates robust support for PostgreSQL tablespaces by facilitating the declarative definition of individual persistent volumes. This innovative feature empowers you to efficiently distribute I/O operations across a diverse array of storage devices. Through the transparent orchestration of tablespaces, CloudNativePG enhances the performance and scalability of PostgreSQL databases, ensuring a streamlined and optimized experience for managing large scale data storage in cloud-native environments. Support for temporary tablespaces is also included.","title":"Tablespace support"},{"location":"operator_capability_levels/#startup-liveness-and-readiness-probes","text":"CloudNativePG configures startup, liveness, and readiness probes for PostgreSQL containers, which are managed by the Kubernetes kubelet. These probes interact with the /healthz and /readyz endpoints exposed by the instance manager's web server to monitor the Pod's health and readiness. The startup and liveness probes use the pg_isready utility. A Pod is considered healthy if pg_isready returns an exit code of 0 (indicating the server is accepting connections) or 1 (indicating the server is rejecting connections, such as during startup). The readiness probe executes a simple SQL query ( ; ) to verify that the PostgreSQL server is ready to accept client connections. All probes are configured with default settings but can be fully customized to meet specific needs, allowing for fine-tuning to align with your environment and workloads.","title":"Startup, Liveness, and Readiness Probes"},{"location":"operator_capability_levels/#rolling-deployments","text":"The operator supports rolling deployments to minimize the downtime. If a PostgreSQL cluster is exposed publicly, the service load-balances the read-only traffic only to available pods during the initialization or the update.","title":"Rolling deployments"},{"location":"operator_capability_levels/#scale-up-and-down-of-replicas","text":"The operator allows you to scale up and down the number of instances in a PostgreSQL cluster. New replicas are started up from the primary server and participate in the cluster's HA infrastructure. The CRD declares a \"scale\" subresource that allows you to use the kubectl scale command.","title":"Scale up and down of replicas"},{"location":"operator_capability_levels/#maintenance-window-and-poddisruptionbudget-for-kubernetes-nodes","text":"The operator creates a PodDisruptionBudget resource to limit the number of concurrent disruptions to one primary instance. This configuration prevents the maintenance operation from deleting all the pods in a cluster, allowing the specified number of instances to be created. The PodDisruptionBudget is applied during the node-draining operation, preventing any disruption of the cluster service. While this strategy is correct for Kubernetes clusters where storage is shared among all the worker nodes, it might not be the best solution for clusters using local storage or for clusters installed in a private cloud. The operator allows you to specify a maintenance window and configure the reaction to any underlying node eviction. The ReusePVC option in the maintenance window section enables to specify the strategy to use. Allocate new storage in a different PVC for the evicted instance, or wait for the underlying node to be available again.","title":"Maintenance window and PodDisruptionBudget for Kubernetes nodes"},{"location":"operator_capability_levels/#fencing","text":"Fencing is the process of protecting the data in one, more, or even all instances of a PostgreSQL cluster when they appear to be malfunctioning. When an instance is fenced, the PostgreSQL server process is guaranteed to be shut down, while the pod is kept running. This ensures that, until the fence is lifted, data on the pod isn't modified by PostgreSQL and that you can investigate file system for debugging and troubleshooting purposes.","title":"Fencing"},{"location":"operator_capability_levels/#hibernation-declarative","text":"CloudNativePG supports hibernation of a running PostgreSQL cluster in a declarative manner, through the cnpg.io/hibernation annotation. Hibernation enables saving CPU power by removing the database pods while keeping the database PVCs. This feature simulates scaling to 0 instances.","title":"Hibernation (declarative)"},{"location":"operator_capability_levels/#hibernation-imperative","text":"CloudNativePG supports hibernation of a running PostgreSQL cluster by way of the cnpg plugin. Hibernation shuts down all Postgres instances in the high-availability cluster and keeps a static copy of the PVC group of the primary. The copy contains PGDATA and WALs. The plugin enables you to exit the hibernation phase by resuming the primary and then recreating all the replicas, if they exist.","title":"Hibernation (imperative)"},{"location":"operator_capability_levels/#reuse-of-persistent-volumes-storage-in-pods","text":"When the operator needs to create a pod that was deleted by the user or was evicted by a Kubernetes maintenance operation, it reuses the PersistentVolumeClaim , if available. This ability avoids the need to clone the data from the primary again.","title":"Reuse of persistent volumes storage in pods"},{"location":"operator_capability_levels/#cpu-and-memory-requests-and-limits","text":"The operator allows administrators to control and manage resource usage by the cluster's pods in the resources section of the manifest. In particular, you can set requests and limits values for both CPU and RAM.","title":"CPU and memory requests and limits"},{"location":"operator_capability_levels/#connection-pooling-with-pgbouncer","text":"CloudNativePG provides native support for connection pooling with PgBouncer , one of the most popular open source connection poolers for PostgreSQL. From an architectural point of view, the native implementation of a PgBouncer connection pooler introduces a new layer to access the database. This optimizes the query flow toward the instances and makes the use of the underlying PostgreSQL resources more efficient. Instead of connecting directly to a PostgreSQL service, applications can now connect to the PgBouncer service and start reusing any existing connection.","title":"Connection pooling with PgBouncer"},{"location":"operator_capability_levels/#logical-replication","text":"CloudNativePG supports PostgreSQL's logical replication in a declarative manner using Publication and Subscription custom resource definitions. Logical replication is particularly useful together with the import facility for online data migrations (even from public DBaaS solutions) and major PostgreSQL upgrades.","title":"Logical Replication"},{"location":"operator_capability_levels/#level-4-deep-insights","text":"Capability level 4 is about observability : monitoring, alerting, trending, and log processing. This might involve the use of external tools, such as Prometheus, Grafana, and Fluent Bit, as well as extensions in the PostgreSQL engine for the output of error logs directly in JSON format. CloudNativePG was designed to provide everything needed to easily integrate with industry-standard and community-accepted tools for flexible monitoring and logging.","title":"Level 4: Deep insights"},{"location":"operator_capability_levels/#prometheus-exporter-with-configurable-queries","text":"The instance manager provides a pluggable framework. By way of its own web server listening on the metrics port (9187), it exposes an endpoint to export metrics for the Prometheus monitoring and alerting tool. The operator supports custom monitoring queries defined as ConfigMap or Secret objects using a syntax that's compatible with postgres_exporter for Prometheus . CloudNativePG provides a set of basic monitoring queries for PostgreSQL that can be integrated and adapted to your context.","title":"Prometheus exporter with configurable queries"},{"location":"operator_capability_levels/#grafana-dashboard","text":"CloudNativePG comes with a Grafana dashboard that you can use as a base to monitor all critical aspects of a PostgreSQL cluster, and customize.","title":"Grafana dashboard"},{"location":"operator_capability_levels/#standard-output-logging-of-postgresql-error-messages-in-json-format","text":"Every log message is delivered to standard output in JSON format. The first level is the definition of the timestamp, the log level, and the type of log entry, such as postgres for the canonical PostgreSQL error message channel. As a result, every pod managed by CloudNativePG can be easily and directly integrated with any downstream log processing stack that supports JSON as source data type.","title":"Standard output logging of PostgreSQL error messages in JSON format"},{"location":"operator_capability_levels/#real-time-query-monitoring","text":"CloudNativePG transparently and natively supports: The essential pg_stat_statements extension , which enables tracking of planning and execution statistics of all SQL statements executed by a PostgreSQL server The auto_explain extension , which provides a means for logging execution plans of slow statements automatically, without having to manually run EXPLAIN (helpful for tracking down un-optimized queries) The pg_failover_slots extension , which makes logical replication slots usable across a physical failover, ensuring resilience in change data capture (CDC) contexts based on PostgreSQL's native logical replication","title":"Real-time query monitoring"},{"location":"operator_capability_levels/#audit","text":"CloudNativePG allows database and security administrators, auditors, and operators to track and analyze database activities using PGAudit for PostgreSQL. Such activities flow directly in the JSON log and can be properly routed to the correct downstream target using common log brokers like Fluentd.","title":"Audit"},{"location":"operator_capability_levels/#kubernetes-events","text":"Record major events as expected by the Kubernetes API, such as creating resources, removing nodes, and upgrading. Events can be displayed by using the kubectl describe and kubectl get events commands.","title":"Kubernetes events"},{"location":"operator_capability_levels/#level-5-auto-pilot","text":"Capability level 5 is focused on automated scaling, healing, and tuning through the discovery of anomalies and insights that emerged from the observability layer.","title":"Level 5: Auto pilot"},{"location":"operator_capability_levels/#automated-failover-for-self-healing","text":"In case of detected failure on the primary, the operator changes the status of the cluster by setting the most aligned replica as the new target primary. As a consequence, the instance manager in each alive pod initiates the required procedures to align itself with the requested status of the cluster. It does this by either becoming the new primary or by following it. In case the former primary comes back up, the same mechanism avoids a split-brain by preventing applications from reaching it, running pg_rewind on the server and restarting it as a standby.","title":"Automated failover for self-healing"},{"location":"operator_capability_levels/#automated-recreation-of-a-standby","text":"If the pod hosting a standby is removed, the operator initiates the procedure to re-create a standby server.","title":"Automated recreation of a standby"},{"location":"operator_conf/","text":"Operator configuration The operator for CloudNativePG is installed from a standard deployment manifest and follows the convention over configuration paradigm. While this is fine in most cases, there are some scenarios where you want to change the default behavior, such as: defining annotations and labels to be inherited by all resources created by the operator and that are set in the cluster resource defining a different default image for PostgreSQL or an additional pull secret By default, the operator is installed in the cnpg-system namespace as a Kubernetes Deployment called cnpg-controller-manager . Note In the examples below we assume the default name and namespace for the operator deployment. The behavior of the operator can be customized through a ConfigMap / Secret that is located in the same namespace of the operator deployment and with cnpg-controller-manager-config as the name. Important Any change to the config's ConfigMap / Secret will not be automatically detected by the operator, - and as such, it needs to be reloaded (see below). Moreover, changes only apply to the resources created after the configuration is reloaded. Important The operator first processes the ConfigMap values and then the Secret\u2019s, in this order. As a result, if a parameter is defined in both places, the one in the Secret will be used. Available options The operator looks for the following environment variables to be defined in the ConfigMap / Secret : Name Description CERTIFICATE_DURATION Determines the lifetime of the generated certificates in days. Default is 90. CLUSTERS_ROLLOUT_DELAY The duration (in seconds) to wait between the roll-outs of different clusters during an operator upgrade. This setting controls the timing of upgrades across clusters, spreading them out to reduce system impact. The default value is 0 which means no delay between PostgreSQL cluster upgrades. CREATE_ANY_SERVICE When set to true , will create -any service for the cluster. Default is false ENABLE_AZURE_PVC_UPDATES Enables to delete Postgres pod if its PVC is stuck in Resizing condition. This feature is mainly for the Azure environment (default false ) ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES When set to true , enables in-place updates of the instance manager after an update of the operator, avoiding rolling updates of the cluster (default false ) EXPIRING_CHECK_THRESHOLD Determines the threshold, in days, for identifying a certificate as expiring. Default is 7. INCLUDE_PLUGINS A comma-separated list of plugins to be always included in the Cluster's reconciliation. INHERITED_ANNOTATIONS List of annotation names that, when defined in a Cluster metadata, will be inherited by all the generated resources, including pods INHERITED_LABELS List of label names that, when defined in a Cluster metadata, will be inherited by all the generated resources, including pods INSTANCES_ROLLOUT_DELAY The duration (in seconds) to wait between roll-outs of individual PostgreSQL instances within the same cluster during an operator upgrade. The default value is 0 , meaning no delay between upgrades of instances in the same PostgreSQL cluster. KUBERNETES_CLUSTER_DOMAIN Defines the domain suffix for service FQDNs within the Kubernetes cluster. If left unset, it defaults to \"cluster.local\". MONITORING_QUERIES_CONFIGMAP The name of a ConfigMap in the operator's namespace with a set of default queries (to be specified under the key queries ) to be applied to all created Clusters MONITORING_QUERIES_SECRET The name of a Secret in the operator's namespace with a set of default queries (to be specified under the key queries ) to be applied to all created Clusters OPERATOR_IMAGE_NAME The name of the operator image used to bootstrap Pods. Defaults to the image specified during installation. POSTGRES_IMAGE_NAME The name of the PostgreSQL image used by default for new clusters. Defaults to the version specified in the operator. PULL_SECRET_NAME Name of an additional pull secret to be defined in the operator's namespace and to be used to download images Values in INHERITED_ANNOTATIONS and INHERITED_LABELS support path-like wildcards. For example, the value example.com/* will match both the value example.com/one and example.com/two . When you specify an additional pull secret name using the PULL_SECRET_NAME parameter, the operator will use that secret to create a pull secret for every created PostgreSQL cluster. That secret will be named -pull . The namespace where the operator looks for the PULL_SECRET_NAME secret is where you installed the operator. If the operator is not able to find that secret, it will ignore the configuration parameter. Defining an operator config map The example below customizes the behavior of the operator, by defining the label/annotation names to be inherited by the resources created by any Cluster object that is deployed at a later time, by enabling in-place updates for the instance manager , and by spreading upgrades. apiVersion: v1 kind: ConfigMap metadata: name: cnpg-controller-manager-config namespace: cnpg-system data: CLUSTERS_ROLLOUT_DELAY: '60' ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES: 'true' INHERITED_ANNOTATIONS: categories INHERITED_LABELS: environment, workload, app INSTANCES_ROLLOUT_DELAY: '10' Defining an operator secret The example below customizes the behavior of the operator, by defining the label/annotation names to be inherited by the resources created by any Cluster object that is deployed at a later time, and by enabling in-place updates for the instance manager , and by spreading upgrades. apiVersion: v1 kind: Secret metadata: name: cnpg-controller-manager-config namespace: cnpg-system type: Opaque stringData: CLUSTERS_ROLLOUT_DELAY: '60' ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES: 'true' INHERITED_ANNOTATIONS: categories INHERITED_LABELS: environment, workload, app INSTANCES_ROLLOUT_DELAY: '10' Restarting the operator to reload configs For the change to be effective, you need to recreate the operator pods to reload the config map. If you have installed the operator on Kubernetes using the manifest you can do that by issuing: kubectl rollout restart deployment \\ -n cnpg-system \\ cnpg-controller-manager In general, given a specific namespace, you can delete the operator pods with the following command: kubectl delete pods -n [NAMESPACE_NAME_HERE] \\ -l app.kubernetes.io/name=cloudnative-pg Warning Customizations will be applied only to Cluster resources created after the reload of the operator deployment. Following the above example, if the Cluster definition contains a categories annotation and any of the environment , workload , or app labels, these will be inherited by all the resources generated by the deployment. pprof HTTP Server The operator can expose a PPROF HTTP server with the following endpoints on localhost:6060 : /debug/pprof/ . Responds to a request for \"/debug/pprof/\" with an HTML page listing the available profiles /debug/pprof/cmdline . Responds with the running program's command line, with arguments separated by NULL bytes. /debug/pprof/profile . Responds with the pprof-formatted cpu profile. Profiling lasts for duration specified in seconds GET parameter, or for 30 seconds if not specified. /debug/pprof/symbol . Looks up the program counters listed in the request, responding with a table mapping program counters to function names. /debug/pprof/trace . Responds with the execution trace in binary form. Tracing lasts for duration specified in seconds GET parameter, or for 1 second if not specified. To enable the operator you need to edit the operator deployment add the flag --pprof-server=true . You can do this by executing these commands: kubectl edit deployment -n cnpg-system cnpg-controller-manager Then on the edit page scroll down the container args and add --pprof-server=true , as in this example: containers: - args: - controller - --enable-leader-election - --config-map-name=cnpg-controller-manager-config - --secret-name=cnpg-controller-manager-config - --log-level=info - --pprof-server=true # relevant line command: - /manager Save the changes; the deployment now will execute a roll-out, and the new pod will have the PPROF server enabled. Once the pod is running you can exec inside the container by doing: kubectl exec -ti -n cnpg-system -- bash Once inside execute: curl localhost:6060/debug/pprof/","title":"Operator configuration"},{"location":"operator_conf/#operator-configuration","text":"The operator for CloudNativePG is installed from a standard deployment manifest and follows the convention over configuration paradigm. While this is fine in most cases, there are some scenarios where you want to change the default behavior, such as: defining annotations and labels to be inherited by all resources created by the operator and that are set in the cluster resource defining a different default image for PostgreSQL or an additional pull secret By default, the operator is installed in the cnpg-system namespace as a Kubernetes Deployment called cnpg-controller-manager . Note In the examples below we assume the default name and namespace for the operator deployment. The behavior of the operator can be customized through a ConfigMap / Secret that is located in the same namespace of the operator deployment and with cnpg-controller-manager-config as the name. Important Any change to the config's ConfigMap / Secret will not be automatically detected by the operator, - and as such, it needs to be reloaded (see below). Moreover, changes only apply to the resources created after the configuration is reloaded. Important The operator first processes the ConfigMap values and then the Secret\u2019s, in this order. As a result, if a parameter is defined in both places, the one in the Secret will be used.","title":"Operator configuration"},{"location":"operator_conf/#available-options","text":"The operator looks for the following environment variables to be defined in the ConfigMap / Secret : Name Description CERTIFICATE_DURATION Determines the lifetime of the generated certificates in days. Default is 90. CLUSTERS_ROLLOUT_DELAY The duration (in seconds) to wait between the roll-outs of different clusters during an operator upgrade. This setting controls the timing of upgrades across clusters, spreading them out to reduce system impact. The default value is 0 which means no delay between PostgreSQL cluster upgrades. CREATE_ANY_SERVICE When set to true , will create -any service for the cluster. Default is false ENABLE_AZURE_PVC_UPDATES Enables to delete Postgres pod if its PVC is stuck in Resizing condition. This feature is mainly for the Azure environment (default false ) ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES When set to true , enables in-place updates of the instance manager after an update of the operator, avoiding rolling updates of the cluster (default false ) EXPIRING_CHECK_THRESHOLD Determines the threshold, in days, for identifying a certificate as expiring. Default is 7. INCLUDE_PLUGINS A comma-separated list of plugins to be always included in the Cluster's reconciliation. INHERITED_ANNOTATIONS List of annotation names that, when defined in a Cluster metadata, will be inherited by all the generated resources, including pods INHERITED_LABELS List of label names that, when defined in a Cluster metadata, will be inherited by all the generated resources, including pods INSTANCES_ROLLOUT_DELAY The duration (in seconds) to wait between roll-outs of individual PostgreSQL instances within the same cluster during an operator upgrade. The default value is 0 , meaning no delay between upgrades of instances in the same PostgreSQL cluster. KUBERNETES_CLUSTER_DOMAIN Defines the domain suffix for service FQDNs within the Kubernetes cluster. If left unset, it defaults to \"cluster.local\". MONITORING_QUERIES_CONFIGMAP The name of a ConfigMap in the operator's namespace with a set of default queries (to be specified under the key queries ) to be applied to all created Clusters MONITORING_QUERIES_SECRET The name of a Secret in the operator's namespace with a set of default queries (to be specified under the key queries ) to be applied to all created Clusters OPERATOR_IMAGE_NAME The name of the operator image used to bootstrap Pods. Defaults to the image specified during installation. POSTGRES_IMAGE_NAME The name of the PostgreSQL image used by default for new clusters. Defaults to the version specified in the operator. PULL_SECRET_NAME Name of an additional pull secret to be defined in the operator's namespace and to be used to download images Values in INHERITED_ANNOTATIONS and INHERITED_LABELS support path-like wildcards. For example, the value example.com/* will match both the value example.com/one and example.com/two . When you specify an additional pull secret name using the PULL_SECRET_NAME parameter, the operator will use that secret to create a pull secret for every created PostgreSQL cluster. That secret will be named -pull . The namespace where the operator looks for the PULL_SECRET_NAME secret is where you installed the operator. If the operator is not able to find that secret, it will ignore the configuration parameter.","title":"Available options"},{"location":"operator_conf/#defining-an-operator-config-map","text":"The example below customizes the behavior of the operator, by defining the label/annotation names to be inherited by the resources created by any Cluster object that is deployed at a later time, by enabling in-place updates for the instance manager , and by spreading upgrades. apiVersion: v1 kind: ConfigMap metadata: name: cnpg-controller-manager-config namespace: cnpg-system data: CLUSTERS_ROLLOUT_DELAY: '60' ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES: 'true' INHERITED_ANNOTATIONS: categories INHERITED_LABELS: environment, workload, app INSTANCES_ROLLOUT_DELAY: '10'","title":"Defining an operator config map"},{"location":"operator_conf/#defining-an-operator-secret","text":"The example below customizes the behavior of the operator, by defining the label/annotation names to be inherited by the resources created by any Cluster object that is deployed at a later time, and by enabling in-place updates for the instance manager , and by spreading upgrades. apiVersion: v1 kind: Secret metadata: name: cnpg-controller-manager-config namespace: cnpg-system type: Opaque stringData: CLUSTERS_ROLLOUT_DELAY: '60' ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES: 'true' INHERITED_ANNOTATIONS: categories INHERITED_LABELS: environment, workload, app INSTANCES_ROLLOUT_DELAY: '10'","title":"Defining an operator secret"},{"location":"operator_conf/#restarting-the-operator-to-reload-configs","text":"For the change to be effective, you need to recreate the operator pods to reload the config map. If you have installed the operator on Kubernetes using the manifest you can do that by issuing: kubectl rollout restart deployment \\ -n cnpg-system \\ cnpg-controller-manager In general, given a specific namespace, you can delete the operator pods with the following command: kubectl delete pods -n [NAMESPACE_NAME_HERE] \\ -l app.kubernetes.io/name=cloudnative-pg Warning Customizations will be applied only to Cluster resources created after the reload of the operator deployment. Following the above example, if the Cluster definition contains a categories annotation and any of the environment , workload , or app labels, these will be inherited by all the resources generated by the deployment.","title":"Restarting the operator to reload configs"},{"location":"operator_conf/#pprof-http-server","text":"The operator can expose a PPROF HTTP server with the following endpoints on localhost:6060 : /debug/pprof/ . Responds to a request for \"/debug/pprof/\" with an HTML page listing the available profiles /debug/pprof/cmdline . Responds with the running program's command line, with arguments separated by NULL bytes. /debug/pprof/profile . Responds with the pprof-formatted cpu profile. Profiling lasts for duration specified in seconds GET parameter, or for 30 seconds if not specified. /debug/pprof/symbol . Looks up the program counters listed in the request, responding with a table mapping program counters to function names. /debug/pprof/trace . Responds with the execution trace in binary form. Tracing lasts for duration specified in seconds GET parameter, or for 1 second if not specified. To enable the operator you need to edit the operator deployment add the flag --pprof-server=true . You can do this by executing these commands: kubectl edit deployment -n cnpg-system cnpg-controller-manager Then on the edit page scroll down the container args and add --pprof-server=true , as in this example: containers: - args: - controller - --enable-leader-election - --config-map-name=cnpg-controller-manager-config - --secret-name=cnpg-controller-manager-config - --log-level=info - --pprof-server=true # relevant line command: - /manager Save the changes; the deployment now will execute a roll-out, and the new pod will have the PPROF server enabled. Once the pod is running you can exec inside the container by doing: kubectl exec -ti -n cnpg-system -- bash Once inside execute: curl localhost:6060/debug/pprof/","title":"pprof HTTP Server"},{"location":"postgis/","text":"PostGIS PostGIS is a very popular open source extension for PostgreSQL that introduces support for storing GIS (Geographic Information Systems) objects in the database and be queried via SQL. Important This section assumes you are familiar with PostGIS and provides some basic information about how to create a new PostgreSQL cluster with a PostGIS database in Kubernetes via CloudNativePG. The CloudNativePG Community maintains container images that are built on top of the official PostGIS images hosted on DockerHub . For more information please visit: The postgis-containers project in GitHub The postgis-containers Container Registry in GitHub Basic concepts about a PostGIS cluster Conceptually, a PostGIS-based PostgreSQL cluster (or simply a PostGIS cluster) is like any other PostgreSQL cluster. The only differences are: the presence in the system of PostGIS and related libraries the presence in the database(s) of the PostGIS extension Since CloudNativePG is based on Immutable Application Containers, the only way to provision PostGIS is to add it to the container image that you use for the operand. The \"Container Image Requirements\" section provides detailed instructions on how this is achieved. More simply, you can just use the PostGIS container images from the Community, as in the examples below. The second step is to install the extension in the PostgreSQL database. You can do this in two ways: install it in the application database, which is the main and supposedly only database you host in the cluster according to the microservice architecture, or install it in the template1 database so as to make it available for all the databases you end up creating in the cluster, in case you adopt the monolith architecture where the instance is shared by multiple databases Info For more information on the microservice vs monolith architecture in the database please refer to the \"How many databases should be hosted in a single PostgreSQL instance?\" FAQ or the \"Database import\" section . Create a new PostgreSQL cluster with PostGIS Let's suppose you want to create a new PostgreSQL 14 cluster with PostGIS 3.2. The first step is to ensure you use the right PostGIS container image for the operand, and properly set the .spec.imageName option in the Cluster resource. The postgis-example.yaml manifest below provides some guidance on how the creation of a PostGIS cluster can be done. Warning Please consider that, although convention over configuration applies in CloudNativePG, you should spend time configuring and tuning your system for production. Also the imageName in the example below deliberately points to the latest available image for PostgreSQL 14 - you should use a specific image name or, preferably, the SHA256 digest for true immutability. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: postgis-example spec: instances: 3 imageName: ghcr.io/cloudnative-pg/postgis:14 bootstrap: initdb: postInitTemplateSQL: - CREATE EXTENSION postgis; - CREATE EXTENSION postgis_topology; - CREATE EXTENSION fuzzystrmatch; - CREATE EXTENSION postgis_tiger_geocoder; storage: size: 1Gi The example relies on the postInitTemplateSQL option which executes a list of queries against the template1 database, before the actual creation of the application database (called app ). This means that, once you have applied the manifest and the cluster is up, you will have the above extensions installed in both the template database and the application database, ready for use. Info Take some time and look at the available options in .spec.bootstrap.initdb from the API reference , such as postInitApplicationSQL . You can easily verify the available version of PostGIS that is in the container, by connecting to the app database (you might obtain different values from the ones in this document): $ kubectl exec -ti postgis-example-1 -- psql app Defaulted container \"postgres\" out of: postgres, bootstrap-controller (init) psql (17.5 (Debian 17.5-1.pgdg110+1)) Type \"help\" for help. app=# SELECT * FROM pg_available_extensions WHERE name ~ '^postgis' ORDER BY 1; name | default_version | installed_version | comment --------------------------+-----------------+-------------------+------------------------------------------------------------ postgis | 3.2.2 | 3.2.2 | PostGIS geometry and geography spatial types and functions postgis-3 | 3.2.2 | | PostGIS geometry and geography spatial types and functions postgis_raster | 3.2.2 | | PostGIS raster types and functions postgis_raster-3 | 3.2.2 | | PostGIS raster types and functions postgis_sfcgal | 3.2.2 | | PostGIS SFCGAL functions postgis_sfcgal-3 | 3.2.2 | | PostGIS SFCGAL functions postgis_tiger_geocoder | 3.2.2 | 3.2.2 | PostGIS tiger geocoder and reverse geocoder postgis_tiger_geocoder-3 | 3.2.2 | | PostGIS tiger geocoder and reverse geocoder postgis_topology | 3.2.2 | 3.2.2 | PostGIS topology spatial types and functions postgis_topology-3 | 3.2.2 | | PostGIS topology spatial types and functions (10 rows) The next step is to verify that the extensions listed in the postInitTemplateSQL section have been correctly installed in the app database. app=# \\dx List of installed extensions Name | Version | Schema | Description ------------------------+---------+------------+------------------------------------------------------------ fuzzystrmatch | 1.1 | public | determine similarities and distance between strings plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language postgis | 3.2.2 | public | PostGIS geometry and geography spatial types and functions postgis_tiger_geocoder | 3.2.2 | tiger | PostGIS tiger geocoder and reverse geocoder postgis_topology | 3.2.2 | topology | PostGIS topology spatial types and functions (5 rows) Finally: app=# SELECT postgis_full_version(); postgis_full_version ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- POSTGIS=\"3.2.2 628da50\" [EXTENSION] PGSQL=\"140\" GEOS=\"3.9.0-CAPI-1.16.2\" PROJ=\"7.2.1\" LIBXML=\"2.9.10\" LIBJSON=\"0.15\" LIBPROTOBUF=\"1.3.3\" WAGYU=\"0.5.0 (Internal)\" TOPOLOGY (1 row)","title":"PostGIS"},{"location":"postgis/#postgis","text":"PostGIS is a very popular open source extension for PostgreSQL that introduces support for storing GIS (Geographic Information Systems) objects in the database and be queried via SQL. Important This section assumes you are familiar with PostGIS and provides some basic information about how to create a new PostgreSQL cluster with a PostGIS database in Kubernetes via CloudNativePG. The CloudNativePG Community maintains container images that are built on top of the official PostGIS images hosted on DockerHub . For more information please visit: The postgis-containers project in GitHub The postgis-containers Container Registry in GitHub","title":"PostGIS"},{"location":"postgis/#basic-concepts-about-a-postgis-cluster","text":"Conceptually, a PostGIS-based PostgreSQL cluster (or simply a PostGIS cluster) is like any other PostgreSQL cluster. The only differences are: the presence in the system of PostGIS and related libraries the presence in the database(s) of the PostGIS extension Since CloudNativePG is based on Immutable Application Containers, the only way to provision PostGIS is to add it to the container image that you use for the operand. The \"Container Image Requirements\" section provides detailed instructions on how this is achieved. More simply, you can just use the PostGIS container images from the Community, as in the examples below. The second step is to install the extension in the PostgreSQL database. You can do this in two ways: install it in the application database, which is the main and supposedly only database you host in the cluster according to the microservice architecture, or install it in the template1 database so as to make it available for all the databases you end up creating in the cluster, in case you adopt the monolith architecture where the instance is shared by multiple databases Info For more information on the microservice vs monolith architecture in the database please refer to the \"How many databases should be hosted in a single PostgreSQL instance?\" FAQ or the \"Database import\" section .","title":"Basic concepts about a PostGIS cluster"},{"location":"postgis/#create-a-new-postgresql-cluster-with-postgis","text":"Let's suppose you want to create a new PostgreSQL 14 cluster with PostGIS 3.2. The first step is to ensure you use the right PostGIS container image for the operand, and properly set the .spec.imageName option in the Cluster resource. The postgis-example.yaml manifest below provides some guidance on how the creation of a PostGIS cluster can be done. Warning Please consider that, although convention over configuration applies in CloudNativePG, you should spend time configuring and tuning your system for production. Also the imageName in the example below deliberately points to the latest available image for PostgreSQL 14 - you should use a specific image name or, preferably, the SHA256 digest for true immutability. apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: postgis-example spec: instances: 3 imageName: ghcr.io/cloudnative-pg/postgis:14 bootstrap: initdb: postInitTemplateSQL: - CREATE EXTENSION postgis; - CREATE EXTENSION postgis_topology; - CREATE EXTENSION fuzzystrmatch; - CREATE EXTENSION postgis_tiger_geocoder; storage: size: 1Gi The example relies on the postInitTemplateSQL option which executes a list of queries against the template1 database, before the actual creation of the application database (called app ). This means that, once you have applied the manifest and the cluster is up, you will have the above extensions installed in both the template database and the application database, ready for use. Info Take some time and look at the available options in .spec.bootstrap.initdb from the API reference , such as postInitApplicationSQL . You can easily verify the available version of PostGIS that is in the container, by connecting to the app database (you might obtain different values from the ones in this document): $ kubectl exec -ti postgis-example-1 -- psql app Defaulted container \"postgres\" out of: postgres, bootstrap-controller (init) psql (17.5 (Debian 17.5-1.pgdg110+1)) Type \"help\" for help. app=# SELECT * FROM pg_available_extensions WHERE name ~ '^postgis' ORDER BY 1; name | default_version | installed_version | comment --------------------------+-----------------+-------------------+------------------------------------------------------------ postgis | 3.2.2 | 3.2.2 | PostGIS geometry and geography spatial types and functions postgis-3 | 3.2.2 | | PostGIS geometry and geography spatial types and functions postgis_raster | 3.2.2 | | PostGIS raster types and functions postgis_raster-3 | 3.2.2 | | PostGIS raster types and functions postgis_sfcgal | 3.2.2 | | PostGIS SFCGAL functions postgis_sfcgal-3 | 3.2.2 | | PostGIS SFCGAL functions postgis_tiger_geocoder | 3.2.2 | 3.2.2 | PostGIS tiger geocoder and reverse geocoder postgis_tiger_geocoder-3 | 3.2.2 | | PostGIS tiger geocoder and reverse geocoder postgis_topology | 3.2.2 | 3.2.2 | PostGIS topology spatial types and functions postgis_topology-3 | 3.2.2 | | PostGIS topology spatial types and functions (10 rows) The next step is to verify that the extensions listed in the postInitTemplateSQL section have been correctly installed in the app database. app=# \\dx List of installed extensions Name | Version | Schema | Description ------------------------+---------+------------+------------------------------------------------------------ fuzzystrmatch | 1.1 | public | determine similarities and distance between strings plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language postgis | 3.2.2 | public | PostGIS geometry and geography spatial types and functions postgis_tiger_geocoder | 3.2.2 | tiger | PostGIS tiger geocoder and reverse geocoder postgis_topology | 3.2.2 | topology | PostGIS topology spatial types and functions (5 rows) Finally: app=# SELECT postgis_full_version(); postgis_full_version ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- POSTGIS=\"3.2.2 628da50\" [EXTENSION] PGSQL=\"140\" GEOS=\"3.9.0-CAPI-1.16.2\" PROJ=\"7.2.1\" LIBXML=\"2.9.10\" LIBJSON=\"0.15\" LIBPROTOBUF=\"1.3.3\" WAGYU=\"0.5.0 (Internal)\" TOPOLOGY (1 row)","title":"Create a new PostgreSQL cluster with PostGIS"},{"location":"postgresql_conf/","text":"PostgreSQL Configuration Users that are familiar with PostgreSQL are aware of the existence of the following three files to configure an instance: postgresql.conf : main run-time configuration file of PostgreSQL pg_hba.conf : clients authentication file pg_ident.conf : map external users to internal users Due to the concepts of declarative configuration and immutability of the PostgreSQL containers, users are not allowed to directly touch those files. Configuration is possible through the postgresql section of the Cluster resource definition by defining custom postgresql.conf , pg_hba.conf , and pg_ident.conf settings via the parameters , the pg_hba , and the pg_ident keys. These settings are the same across all instances. Warning Please don't use the ALTER SYSTEM query to change the configuration of the PostgreSQL instances in an imperative way. Changing some of the options that are normally controlled by the operator might indeed lead to an unpredictable/unrecoverable state of the cluster. Moreover, ALTER SYSTEM changes are not replicated across the cluster. See \"Enabling ALTER SYSTEM\" below for details. A reference for custom settings usage is included in the samples, see cluster-example-custom.yaml . The postgresql section The PostgreSQL instance in the pod starts with a default postgresql.conf file, to which these settings are automatically added: listen_addresses = '*' include custom.conf The custom.conf file will contain the user-defined settings in the postgresql section, as in the following example: # ... postgresql: parameters: shared_buffers: \"1GB\" # ... PostgreSQL GUCs: Grand Unified Configuration Refer to the PostgreSQL documentation for more information on the available parameters , also known as GUC (Grand Unified Configuration). Please note that CloudNativePG accepts only strings for the PostgreSQL parameters. The content of custom.conf is automatically generated and maintained by the operator by applying the following sections in this order: Global default parameters Default parameters that depend on the PostgreSQL major version User-provided parameters Fixed parameters The global default parameters are: archive_timeout = '5min' dynamic_shared_memory_type = 'posix' full_page_writes = 'on' logging_collector = 'on' log_destination = 'csvlog' log_directory = '/controller/log' log_filename = 'postgres' log_rotation_age = '0' log_rotation_size = '0' log_truncate_on_rotation = 'false' max_parallel_workers = '32' max_replication_slots = '32' max_worker_processes = '32' shared_memory_type = 'mmap' shared_preload_libraries = '' ssl_max_protocol_version = 'TLSv1.3' ssl_min_protocol_version = 'TLSv1.3' wal_keep_size = '512MB' wal_level = 'logical' wal_log_hints = 'on' wal_sender_timeout = '5s' wal_receiver_timeout = '5s' Warning It is your duty to plan for WAL segments retention in your PostgreSQL cluster and properly configure either wal_keep_size or wal_keep_segments , depending on the server version, based on the expected and observed workloads. Alternatively, if the only streaming replication clients are the replica instances running in the High Availability cluster, you can take advantage of the replication slots feature, which adds support for replication slots at the cluster level. You can enable the feature with the replicationSlots.highAvailability option (for more information, please refer to the \"Replication\" section .) Without replication slots nor continuous backups in place, configuring wal_keep_size or wal_keep_segments is the only way to protect standbys from falling out of sync. If a standby did fall out of sync it would produce error messages like: \"could not receive data from WAL stream: ERROR: requested WAL segment ************************ has already been removed\" . This will require you to dedicate a part of your PGDATA , or the volume dedicated to storing WAL files, to keep older WAL segments for streaming replication purposes. The following parameters are fixed and exclusively controlled by the operator: archive_command = '/controller/manager wal-archive %p' hot_standby = 'true' listen_addresses = '*' port = '5432' restart_after_crash = 'false' ssl = 'on' ssl_ca_file = '/controller/certificates/client-ca.crt' ssl_cert_file = '/controller/certificates/server.crt' ssl_key_file = '/controller/certificates/server.key' unix_socket_directories = '/controller/run' Since the fixed parameters are added at the end, they can't be overridden by the user via the YAML configuration. Those parameters are required for correct WAL archiving and replication. Write-Ahead Log Level The wal_level parameter in PostgreSQL determines the amount of information written to the Write-Ahead Log (WAL). It accepts the following values: minimal : Writes only the information required for crash recovery. replica : Adds sufficient information to support WAL archiving and streaming replication, including the ability to run read-only queries on standby instances. logical : Includes all information from replica , plus additional information required for logical decoding and replication. By default, upstream PostgreSQL sets wal_level to replica . CloudNativePG, instead, sets wal_level to logical by default to enable logical replication out of the box. This makes it easier to support use cases such as migrations from external PostgreSQL servers. If your cluster does not require logical replication, it is recommended to set wal_level to replica to reduce WAL volume and overhead. Finally, CloudNativePG allows wal_level to be set to minimal only for single-instance clusters with WAL archiving disabled. Replication Settings The primary_conninfo , restore_command , and recovery_target_timeline parameters are managed automatically by the operator according to the state of the instance in the cluster. primary_conninfo = 'host=cluster-example-rw user=postgres dbname=postgres' recovery_target_timeline = 'latest' Log control settings The operator requires PostgreSQL to output its log in CSV format, and the instance manager automatically parses it and outputs it in JSON format. For this reason, all log settings in PostgreSQL are fixed and cannot be changed. For further information, please refer to the \"Logging\" section . Shared Preload Libraries The shared_preload_libraries option in PostgreSQL exists to specify one or more shared libraries to be pre-loaded at server start, in the form of a comma-separated list. Typically, it is used in PostgreSQL to load those extensions that need to be available to most database sessions in the whole system (e.g. pg_stat_statements ). In CloudNativePG the shared_preload_libraries option is empty by default. Although you can override the content of shared_preload_libraries , we recommend that only expert Postgres users take advantage of this option. Important In case a specified library is not found, the server fails to start, preventing CloudNativePG from any self-healing attempt and requiring manual intervention. Please make sure you always test both the extensions and the settings of shared_preload_libraries if you plan to directly manage its content. CloudNativePG is able to automatically manage the content of the shared_preload_libraries option for some of the most used PostgreSQL extensions (see the \"Managed extensions\" section below for details). Specifically, as soon as the operator notices that a configuration parameter requires one of the managed libraries, it will automatically add the needed library. The operator will also remove the library as soon as no actual parameter requires it. Important Please always keep in mind that removing libraries from shared_preload_libraries requires a restart of all instances in the cluster in order to be effective. You can provide additional shared_preload_libraries via .spec.postgresql.shared_preload_libraries as a list of strings: the operator will merge them with the ones that it automatically manages. Managed extensions As anticipated in the previous section, CloudNativePG automatically manages the content in shared_preload_libraries for some well-known and supported extensions. The current list includes: auto_explain pg_stat_statements pgaudit pg_failover_slots Some of these libraries also require additional objects in a database before using them, normally views and/or functions managed via the CREATE EXTENSION command to be run in a database (the DROP EXTENSION command typically removes those objects). For such libraries, CloudNativePG automatically handles the creation and removal of the extension in all databases that accept a connection in the cluster, identified by the following query: SELECT datname FROM pg_database WHERE datallowconn Note The above query also includes template databases like template1 . Enabling auto_explain The auto_explain extension provides a means for logging execution plans of slow statements automatically, without having to manually run EXPLAIN (helpful for tracking down un-optimized queries). You can enable auto_explain by adding to the configuration a parameter that starts with auto_explain. as in the following example excerpt (which automatically logs execution plans of queries that take longer than 10 seconds to complete): # ... postgresql: parameters: auto_explain.log_min_duration: \"10s\" # ... Note Enabling auto_explain can lead to performance issues. Please refer to the auto explain documentation Enabling pg_stat_statements The pg_stat_statements extension is one of the most important capabilities available in PostgreSQL for real-time monitoring of queries. You can enable pg_stat_statements by adding to the configuration a parameter that starts with pg_stat_statements. as in the following example excerpt: # ... postgresql: parameters: pg_stat_statements.max: \"10000\" pg_stat_statements.track: all # ... As explained previously, the operator will automatically add pg_stat_statements to shared_preload_libraries and run CREATE EXTENSION IF NOT EXISTS pg_stat_statements on each database, enabling you to run queries against the pg_stat_statements view. Enabling pgaudit The pgaudit extension provides detailed session and/or object audit logging via the standard PostgreSQL logging facility. CloudNativePG has transparent and native support for PGAudit on PostgreSQL clusters. For further information, please refer to the \"PGAudit\" logs section. You can enable pgaudit by adding to the configuration a parameter that starts with pgaudit. as in the following example excerpt: # postgresql: parameters: pgaudit.log: \"all, -misc\" pgaudit.log_catalog: \"off\" pgaudit.log_parameter: \"on\" pgaudit.log_relation: \"on\" # Enabling pg_failover_slots The pg_failover_slots extension by EDB ensures that logical replication slots can survive a failover scenario. Failovers are normally implemented using physical streaming replication, like in the case of CloudNativePG. You can enable pg_failover_slots by adding to the configuration a parameter that starts with pg_failover_slots. : as explained above, the operator will transparently manage the pg_failover_slots entry in the shared_preload_libraries option depending on this. Please refer to the pg_failover_slots documentation for details on this extension. Additionally, for each database that you intend to you use with pg_failover_slots you need to add an entry in the pg_hba section that enables each replica to connect to the primary. For example, suppose that you want to use the app database with pg_failover_slots , you need to add this entry in the pg_hba section: postgresql: pg_hba: - hostssl app streaming_replica all cert The pg_hba section pg_hba is a list of PostgreSQL Host Based Authentication rules used to create the pg_hba.conf used by the pods. Important See the PostgreSQL documentation for more information on pg_hba.conf . Since the first matching rule is used for authentication, the pg_hba.conf file generated by the operator can be seen as composed of four sections: Fixed rules User-defined rules Optional LDAP section Default rules Fixed rules: local all all peer hostssl postgres streaming_replica all cert hostssl replication streaming_replica all cert hostssl all cnpg_pooler_pgbouncer all cert Default rules: host all all all From PostgreSQL 14 the default value of the password_encryption database parameter is set to scram-sha-256 . Because of that, the default authentication method is scram-sha-256 from this PostgreSQL version. PostgreSQL 13 and older will use md5 as the default authentication method. The resulting pg_hba.conf will look like this: local all all peer hostssl postgres streaming_replica all cert hostssl replication streaming_replica all cert