AlloyDB Postgres source setup guide

Supported versions

To propagate data from your AlloyDB instance to ClickHouse Cloud using ClickPipes, your instance must be configured for logical replication. This is supported from AlloyDB Version 14.

Enable logical replication

To check if logical replication is enabled in your AlloyDB instance, run the following query against your primary instance:

SHOW  wal_level;

If the result is logical, logical replication is already enabled and you can skip to the next step. If the result is replica, you must set the alloydb.enable_pglogical and alloydb.logical_decoding flags to on in the primary instance.

As noted in the AlloyDB flags documentation, modifying the flags that enable logical replication requires a restart of the primary instance.

To enable these flags:

In the Google Cloud Console, navigate to the AlloyDB Clusters page. From the Actions menu for your primary instance, click Edit.
Scroll down to Advanced configuration options and expand the section. Under Flags, click Add a database flag.
- Add the allowdb.enable_pglogical flag and set its value to on
- Add the alloydb.logical_decoding flag and set its value to on
Click Update instance to save the configuration changes. It’s important to note that this action triggers a restart of the primary instance.
Once the status of the instance changes from Updating to Ready, run the following query against your primary instance to verify that logical replication is enabled:
SHOW wal_level;
The result should be logical.

Create a ClickPipes user and manage replication permissions

Connect to your AlloyDB instance as an admin user and execute the following commands:

Create a dedicated user for ClickPipes:

CREATE USER clickpipes_user PASSWORD 'some-password';

Grant schema-level, read-only access to the user you created in the previous step. The following example shows permissions for the public schema. Repeat these commands for each schema containing tables you want to replicate:

GRANT USAGE ON SCHEMA "public" TO clickpipes_user;
GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO clickpipes_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA "public" GRANT SELECT ON TABLES TO clickpipes_user;

Grant replication privileges to the user:

ALTER USER clickpipes_user WITH REPLICATION;

Create a publication with the tables you want to replicate. We strongly recommend only including the tables you need in the publication to avoid performance overhead.

Any table included in the publication must either have a primary key defined or have its replica identity configured to FULL. See the Postgres FAQs for guidance on scoping.

To create a publication for specific tables:

CREATE PUBLICATION clickpipes FOR TABLE table_to_replicate, table_to_replicate2;

To create a publication for all tables in a specific schema:

CREATE PUBLICATION clickpipes FOR TABLES IN SCHEMA "public";

The clickpipes publication defines the set of tables whose change events will be streamed to ClickPipes. We recommend against using FOR ALL TABLES unless you intend to replicate every table, as including unnecessary tables increases WAL traffic from Postgres to ClickPipes and reduces overall replication efficiency.

Configure network access

Next, choose how ClickPipes will connect to your AlloyDB instance.

Use Private Service Connect
Allow ClickPipes IPs
Use an SSH tunnel

For private connectivity from a ClickHouse Cloud service hosted on GCP, follow the GCP Private Service Connect guide to configure a reverse private endpoint for your AlloyDB instance.

In the Google Cloud Console, navigate to the AlloyDB Clusters page. Select your primary instance to open the Overview page.
Scroll down to Instances in your cluster and click Edit primary.
Check the Enable Public IP checkbox to allow connections to the instance over the public internet. Under Authorized external networks, enter the list of ClickPipes static IP addresses for the region your service is deployed in.

AlloyDB expects addresses to be specified in CIDR notation. You can adapt the provided list of ClickPipes static IP addresses to follow this notation by appending /32 to each address.

Under Network Security, select Require SSL Encryption (default) (if not already selected).
Click Update instance to save the network security configuration changes.

What’s next?

You can now create your ClickPipe and start ingesting data from your Postgres instance into ClickHouse Cloud. Make sure to note down the connection details you used while setting up your Postgres instance as you will need them during the ClickPipe creation process.

ClickPipes for Object Storage

ClickPipes for Postgres

ClickPipes for Kafka

ClickPipes for Kinesis

ClickPipes for GCP Pub/Sub

ClickPipes for MySQL

ClickPipes for MongoDB

ClickPipes for BigQuery

Programmatic access

Networking

Other

Supported versions

Enable logical replication

Create a ClickPipes user and manage replication permissions

Configure network access

What’s next?

​Supported versions

​Enable logical replication

​Create a ClickPipes user and manage replication permissions

​Configure network access

​What’s next?

Supported versions

Enable logical replication

Create a ClickPipes user and manage replication permissions

Configure network access

What’s next?