> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-vortex-format.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# BYOC Network Security

> Deploy ClickHouse on your own cloud infrastructure

export const Image = ({img, alt, size = "lg"}) => {
  const normalizedSize = ["sm", "md", "lg"].includes(size) ? size : "lg";
  return <div className={`ch-image-${normalizedSize}`}>
      <Frame>
        <img src={img} alt={alt} />
      </Frame>
    </div>;
};

<h2 id="connection-between-clickhouse-and-byoc">
  Connections between ClickHouse Control Plane and Your BYOC VPC
</h2>

The ClickHouse Cloud control plane maintains several types of connections to operate and support your BYOC deployment:

| Purpose                                      | Connection type                                                                 | Notes                                                                                                                                                                                                                                        |
| -------------------------------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Daily operations — Kubernetes API server** | Public with IP filtering (default), or private via Tailscale or AWS VPC Lattice | Management services talk to the Kubernetes (EKS/GKE/AKS) API server over the public network, restricted by IP allow lists. After initial deployment, you can optionally switch this to private access via Tailscale or, on AWS, VPC Lattice. |
| **Daily operations — cloud provider APIs**   | ClickHouse VPC → cloud provider                                                 | Management services call your cloud provider's APIs (e.g., EKS and EC2 on AWS, GKE on GCP, AKS on Azure) from ClickHouse Cloud’s own environment. This doesn't involve your VPC/VNet or Tailscale.                                           |
| **Troubleshooting — ClickHouse service**     | Tailscale                                                                       | ClickHouse engineers access the ClickHouse service (e.g., system tables) for diagnostics via Tailscale.                                                                                                                                      |
| **Troubleshooting — Kubernetes API server**  | Tailscale                                                                       | ClickHouse engineers access the Kubernetes API server for cluster diagnostics via Tailscale.                                                                                                                                                 |

<h2 id="cloud-api-vs-kubernetes-api">
  Cloud provider APIs vs the Kubernetes API
</h2>

Two distinct control paths are easy to conflate. They differ in where the traffic originates, how it authenticates, and whether Tailscale applies:

|                             | Cloud provider APIs                                                                                                                                                                                                                                                                  | Kubernetes API server                                                                                                                                                                                                                                                                                                 |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **What it manages**         | Cloud resources: the Kubernetes cluster itself, node groups, load balancers, storage buckets, DNS                                                                                                                                                                                    | Workloads inside the cluster: ClickHouse pods, operators, and their configuration                                                                                                                                                                                                                                     |
| **Traffic destination**     | The cloud provider's public API endpoints (for example `eks.amazonaws.com`, `ec2.amazonaws.com`) — this traffic never enters your VPC/VNet                                                                                                                                           | Your cluster's API endpoint in your account                                                                                                                                                                                                                                                                           |
| **Authentication**          | AWS: cross-account `sts:AssumeRole` into `ClickHouseManagementRole` and the per-infrastructure management role, protected by an external ID. GCP: service account impersonation (no keys). Azure: federated identity for the ClickHouse service principal (no credentials exchanged) | Short-lived Kubernetes credentials issued through the cloud provider's API (for example `eks:GetToken` using the assumed role)                                                                                                                                                                                        |
| **Tailscale applicability** | **None.** These calls go from ClickHouse Cloud's network directly to the cloud provider and cannot be routed through Tailscale or through your network                                                                                                                               | Public endpoint restricted to ClickHouse egress IPs by default; can be switched to private access via Tailscale or, on AWS, VPC Lattice                                                                                                                                                                               |
| **Your audit trail**        | AWS CloudTrail (or the GCP/Azure equivalents) in your account records every call with the assumed identity                                                                                                                                                                           | On AWS, EKS control-plane logs — including the audit log — are delivered to a CloudWatch log group in your account (see [billable AWS services](/products/bring-your-own-cloud/reference/billable-aws-services)); on GCP and Azure, contact support to confirm or enable control-plane audit logging for your cluster |

In short: only Kubernetes API and troubleshooting traffic can use Tailscale. Cloud provider API calls always originate from ClickHouse Cloud's network and terminate at the provider's endpoints — routing them through Tailscale isn't possible, because they never enter your network in the first place.

<h3 id="network-origin-permission-boundaries">
  Network-origin permission boundaries
</h3>

If your organization restricts IAM role assumption or cloud API calls by network origin (for example, AWS SCPs or role trust conditions using `aws:SourceIp` or `aws:SourceVpc`), those conditions will block ClickHouse's automation: the calls legitimately originate from ClickHouse Cloud's network, not yours. Exempt the ClickHouse-created roles from such conditions, or contact ClickHouse for the current egress IP ranges if you must allowlist by origin.

The following section describes how the **Tailscale** private network is used for troubleshooting and optional management access.

<h2 id="tailscale-private-network">
  Tailscale Private Network
</h2>

Tailscale provides a zero-trust, private network connection between ClickHouse Cloud's management services and your BYOC deployment. This secure channel enables ClickHouse engineers to perform troubleshooting and management operations without requiring inbound public network access or complex VPN configurations; the agents themselves make outbound-only connections and need outbound internet access to reach the Tailscale coordination service.

<h3 id="tailscale-overview">
  Overview
</h3>

Tailscale creates an encrypted, private network tunnel between the ClickHouse control plane (in ClickHouse's VPC) and your BYOC data plane (in your VPC). This connection is used exclusively for:

* **Management operations**: ClickHouse management services coordinating with your BYOC infrastructure
* **Troubleshooting access**: ClickHouse engineers accessing Kubernetes API servers and ClickHouse system tables for diagnostics
* **Metrics access**: ClickHouse’s centralized monitoring dashboards access metrics from the Prometheus stack deployed within your BYOC VPC, providing ClickHouse engineers observability into the environment.

<Warning>
  Tailscale is used **only for management and troubleshooting operations**. It is **never used for query traffic** or customer data access. All customer data remains within your VPC and is never transmitted through Tailscale connections.
</Warning>

<h3 id="how-tailscale-works">
  How Tailscale Works in BYOC
</h3>

<Image img="https://mintcdn.com/private-7c7dfe99-vortex-format/0Bwb4z8TuZiplPdq/images/cloud/reference/byoc-tailscale-1.webp?fit=max&auto=format&n=0Bwb4z8TuZiplPdq&q=85&s=b88172b5da3f4bcd745f9171b68c61d2" size="lg" alt="BYOC Tailscale" border width="3484" height="1792" data-path="images/cloud/reference/byoc-tailscale-1.webp" />

For each service or endpoint that needs to be accessed via Tailscale, ClickHouse BYOC deploys:

1. **Tailnet Address Registration**: Each endpoint registers a unique tailnet address (e.g., `k8s.xxxx.us-east-1.aws.byoc.clickhouse-prd.com` for the Kubernetes API server)

2. **Tailscale Agent Container**: A Tailscale agent container runs in your Kubernetes cluster, responsible for:
   * Connecting to the Tailscale coordination server
   * Registering services to make them discoverable
   * Coordinating network setup with Nginx pods

3. **Nginx Pod**: An Nginx pod that:
   * Terminates TLS traffic from Tailscale
   * Routes traffic to the appropriate IPs within your Kubernetes cluster

<h3 id="tailscale-connection-process">
  Network Connection Process
</h3>

The Tailscale connection establishment follows these steps:

1. **Initial Connection**:
   * Tailscale agents on both ends (ClickHouse engineer's environment and your BYOC Kubernetes cluster) connect to the Tailscale coordination server
   * The cluster agent registers the Kubernetes service to make it discoverable
   * ClickHouse engineers must escalate internally to gain visibility to the service

2. **Connection Mode**:
   * **Direct Mode**: Agents attempt to establish a direct connection via NAT traversal tunnel
   * **Relay Mode**: If direct mode fails, communication falls back to relay mode through a Tailscale DERP (Distributed Encrypted Relay Protocol) server

3. **Encryption**:
   * All communication is encrypted end-to-end
   * Each Tailscale agent generates its own public-private key pair (similar to PKI)
   * Traffic remains encrypted regardless of whether it uses direct or relay mode

<h3 id="tailscale-security">
  Security features
</h3>

**Outbound-only connections**:

* Tailscale agents in your Kubernetes cluster initiate outbound connections to the Tailscale coordination/relay servers
* **No inbound connections are required** — no security group rules need to allow inbound traffic to Tailscale agents
* This reduces the attack surface and simplifies network security configuration

**Access control**:

* Engineers must request access through an internal approval workflow before Tailscale can route them to a customer endpoint
* Access is time-bound and automatically expires
* All access is audited and logged

For the full data access policy — what engineers can see, certificate-based authentication, and customer-side auditing — see [ClickHouse data access](/products/bring-your-own-cloud/reference/clickhouse-data-access).

<h3 id="management-services-access">
  Management Services Access
</h3>

By default, ClickHouse management services access your BYOC Kubernetes cluster via the API server's public IP address, which is restricted to ClickHouse's NAT gateway IP addresses only.

**Optional Private Endpoint Configuration**:

* You can configure the Kubernetes API server to use only a private endpoint
* In this case, management services access the API server via Tailscale (similar to human troubleshooting access) or, on AWS, via VPC Lattice (see [Kubernetes API Private Connection](/products/bring-your-own-cloud/configuration/configurations#k8s-api-private-connection))
* By default, the public endpoint (restricted to ClickHouse's NAT IP addresses) is kept as a backup mechanism for emergency investigation and support needs; once private access is verified, it can be disabled entirely in coordination with ClickHouse

<h3 id="tailscale-traffic-flow">
  Network Traffic Flow
</h3>

**Tailscale Connection Flow**:

1. Tailscale agent in your Kubernetes cluster → Tailscale coordination server (outbound)
2. Tailscale agent on engineer's machine → Tailscale coordination server (outbound)
3. Direct or relayed connection established between agents
4. Encrypted traffic flows through the established tunnel
5. Nginx pod in EKS terminates TLS and routes to internal services

**No Customer Data Transmission**:

* Tailscale connections are used only for management and troubleshooting
* Query traffic and customer data never flow through Tailscale
* All customer data remains within your VPC

For more technical details about how Tailscale is implemented in BYOC, see the [Building ClickHouse BYOC on AWS blog post](https://clickhouse.com/blog/building-clickhouse-byoc-on-aws#tailscale-connection). For what ClickHouse engineers can read once connected and how ClickHouse audits that access, see [ClickHouse data access](/products/bring-your-own-cloud/reference/clickhouse-data-access).

<h2 id="network-boundaries">
  Network boundaries
</h2>

This section covers different network traffic to and from the customer BYOC VPC:

* **Inbound**: Traffic entering the customer BYOC VPC.
* **Outbound**: Traffic originating from the customer BYOC VPC and sent to an external destination.
* **Public**: A network endpoint accessible from the public internet.
* **Private**: A network endpoint accessible only through private connections, such as VPC peering, VPC Private Link, or Tailscale.

**Istio ingress is deployed behind an AWS NLB to accept ClickHouse client traffic.**

*Inbound, Public or Private*

The Istio ingress gateway terminates TLS. The certificate, provisioned by CertManager with Let's Encrypt, is stored as a secret within the EKS cluster. Traffic between Istio and ClickHouse is [encrypted by AWS](https://docs.aws.amazon.com/whitepapers/latest/logical-separation/encrypting-data-at-rest-and--in-transit.html#:~:text=All%20network%20traffic%20between%20AWS,supported%20Amazon%20EC2%20instance%20types) since they reside in the same VPC.

By default, ingress is publicly accessible with IP allow list filtering. Customers can configure VPC peering to make it private and disable public connections. We highly recommend setting up an [IP filter](/products/cloud/guides/security/connectivity/setting-ip-filters) to restrict access.

<h3 id="troubleshooting-access">
  Troubleshooting access
</h3>

*Inbound, Private*

ClickHouse Cloud engineers require troubleshooting access via Tailscale. They're provisioned with just-in-time certificate-based authentication for BYOC deployments. See [ClickHouse data access](/products/bring-your-own-cloud/reference/clickhouse-data-access) for the full access policy.

<h3 id="billing-scraper">
  Billing scraper
</h3>

*Outbound, Private*

The Billing scraper collects billing data from ClickHouse and sends it to an S3 bucket owned by ClickHouse Cloud.

It runs as a sidecar alongside the ClickHouse server container, periodically scraping CPU and memory metrics. Requests within the same region are routed through VPC gateway service endpoints.

<h3 id="alerts">
  Alerts
</h3>

*Outbound, Public*

AlertManager is configured to send alerts to ClickHouse Cloud when the customer's ClickHouse cluster is unhealthy.

Metrics and logs are stored within the customer's BYOC VPC. Logs are currently stored locally in EBS. In a future update, they will be stored in LogHouse, a ClickHouse service within the BYOC VPC. Metrics use a Prometheus and Thanos stack, stored locally in the BYOC VPC.

<h3 id="service-state">
  Service state
</h3>

*Outbound, Public*

State Exporter sends ClickHouse service and backup state information (operational status events, not backup contents) to an SQS queue owned by ClickHouse Cloud.
