Serhii Mazurok - founder of Segla; CKA/CKAD
2026-04-02 · 9 min read
A Kubernetes cluster spread across providers has no private network unless you build one. WireGuard is how I build it: a full mesh of kernel tunnels, one config per node, nothing else in the path.
To run Kubernetes across Hetzner, AWS and a rack of your own, give every node a WireGuard interface with a private address, peer each node with every other node, and bind kubelet and the pod network to that interface instead of the public one. The cluster then sees one flat 10.10.0.0/24 regardless of where the machines physically are, every byte between nodes is encrypted in the kernel, and no Kubernetes component listens on a public address. The rest of this post is the configuration, the integration points, and the one failure mode that will cost you an afternoon if nobody warns you.
The team this is written for has a cluster that grew across providers because that is where the cheap machines were: two Hetzner boxes, an AWS instance for the managed database's latency, a bare-metal server someone already paid for. No platform team; one person who knows the network, and they would like to stop being the only one.
Cloud providers give you a VPC inside their own walls. Across providers there is nothing:
What is needed is a private network that works on any Linux machine, is fast enough to be invisible, and can be generated by a program rather than configured by a person.
| Property | WireGuard | Where that comes from |
|---|---|---|
| Configuration | about a dozen lines per node - the example below is the whole thing | this post |
| Where it runs | in the Linux kernel, mainline since 5.6 | wireguard.com/install |
| Handshake | one round trip (Noise IK) | the WireGuard whitepaper |
| Code size | small enough to be reviewed by one person; the project contrasts itself with "behemoths like *Swan/IPsec or OpenVPN" | wireguard.com |
| Cryptography | Curve25519, ChaCha20, Poly1305, BLAKE2s - fixed, no negotiation | the whitepaper |
| Throughput | the project's own benchmarks put it ahead of IPsec and far ahead of OpenVPN, and call those benchmarks old | wireguard.com/performance |
The row that matters operationally is the first one. A WireGuard node is a private key, a public key and a list of peers. Every other row follows from the design being that small.
The mesh for a 3-node cluster:
┌──────────────┐ WireGuard tunnel ┌──────────────┐
│ Node 1 │◄────────────────────────►│ Node 2 │
│ 10.10.0.1 │ │ 10.10.0.2 │
│ Hetzner │ │ Hetzner │
└──────┬───────┘ └──────┬───────┘
│ │
│ WireGuard tunnel │
│ ┌──────────────┐ │
└───►│ Node 3 │◄────────────────────┘
│ 10.10.0.3 │
│ AWS EC2 │
└──────────────┘
Each node has a WireGuard interface, wg0, with a private address from 10.10.0.0/24. Every node peers with every other node - a full mesh. That is n(n-1)/2 tunnels: three for three nodes, 45 for ten, 1,225 for fifty. Up to a few dozen nodes it is the simplest thing that works; past that, look at a hub-and-spoke topology or a mesh manager.
The steps are the same whether a person or an agent does them:
[Interface] and one [Peer] block per other node10.10.0.x address before you start Kubernetes# Node 1
[Interface]
PrivateKey = <node1-private-key>
Address = 10.10.0.1/24
ListenPort = 51820
[Peer]
PublicKey = <node2-public-key>
Endpoint = 167.235.12.46:51820
AllowedIPs = 10.10.0.2/32
PersistentKeepalive = 25
[Peer]
PublicKey = <node3-public-key>
Endpoint = 3.121.45.67:51820
AllowedIPs = 10.10.0.3/32
PersistentKeepalive = 25
PersistentKeepalive = 25 is not decoration: without it a peer behind NAT goes silent once the mapping expires, and the tunnel looks up until the first packet that has to go through it.
Step 2 is the one that does not scale by hand. Ten nodes is ninety peer blocks to keep consistent; one rotated key is ten edits on ten machines. This is the part Segla's agent does - the mesh between your servers is built and kept for you, and a node that joins later gets its peers without anyone touching the others.
Once every node can reach every other node on 10.10.0.x, Kubernetes is told to live there:
--node-ip=10.10.0.X10.244.0.0/16 here) is routed over the overlay, so pod-to-pod traffic across nodes goes through the tunnelIf you also run the control plane on these nodes, etcd peers and the API server bind to the tunnel addresses too, and nothing about Kubernetes is reachable from the public internet. On Segla the control plane is hosted, so your servers are workers on the mesh and the API endpoint is the cluster's own <cluster>.kube-apiserver.segla.io.
Here is the afternoon. The mesh is up. wg show on every node lists every peer with a handshake seconds old. Nodes ping each other on 10.10.0.x. kubelet joins, nodes go Ready. You deploy something with two replicas and a Service in front of it, and requests to the Service work about half the time.
The half that works is the pods on the same node as the caller. The half that fails is anything that has to cross a node boundary: the packet goes into wg0, comes out on the other node, and has to be forwarded from the tunnel interface to the CNI bridge - and nothing comes back. CoreDNS times out for pods that are not on its node. There is no error anywhere, because a dropped packet is not an error.
The cause is one line of host firewall policy. On Ubuntu with UFW enabled, the default for routed traffic is deny:
$ sudo ufw status verbose
Status: active
Default: deny (incoming), allow (outgoing), deny (routed)
deny (routed) is the FORWARD chain. Traffic addressed to the node is allowed through the rules you wrote; traffic passing through the node - which is what every packet from a tunnel to a pod is - is dropped by default, silently, on every node that has the firewall on. WireGuard is fine. Kubernetes is fine. The host is doing exactly what it was told.
The fix is to allow forwarding between the interfaces that carry the cluster:
sudo ufw route allow in on wg0 out on cni0
sudo ufw route allow in on cni0 out on wg0
# or, if the host does nothing but run this node:
sudo ufw default allow routed
net.ipv4.ip_forward is 1 and the FORWARD policy is ACCEPT for those interfaces.What makes it a war story rather than a tip is how it presents: everything you would check says healthy, and the symptom - intermittent Service failures - points at the application. Segla's agent checks for it when a node joins, because this is the kind of thing that should fail loudly once, on the machine, and never again in production.
The handshake age is the best single signal of tunnel health: WireGuard rehandshakes roughly every two minutes while traffic flows, so a peer whose last handshake is three minutes old is a peer you cannot reach.
wg show through a Prometheus exporterlast_handshake_seconds, transfer_bytes_received, transfer_bytes_sent- alert: WireGuardPeerDown
expr: time() - wireguard_last_handshake_seconds > 180
for: 2m
labels:
severity: critical
annotations:
summary: "WireGuard peer {{ $labels.peer }} is unreachable"
--node-ip and the pod CIDR over the overlayPrivate networking between your servers is part of what Segla runs: the mesh is built when a node joins, checked before it carries traffic, and kept as nodes come and go. Add a server and it is already on the network.