Tailscale version 1.20.2
Tailscale proxy pod uses alpine as base running on different kubernetes clusters, nodes based on flatcar
Hi,
we using tailscale as proxy in a docker container (we used tailscale/docs/k8s at main · tailscale/tailscale · GitHub as template, and built our own helm chart from that, added some minor features as restricting proxy to dedicated ports - we plan to opensource that later)
We connect elastic agents via the tailscale network to elastic fleetserver (the fleetservers running in k8s with tailscale proxy pod next to it).
tailscale proxy config:
/ # iptables -L PREROUTING -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DNAT tcp -- 0.0.0.0/0 100.70.213.98 tcp dpt:8220 to:10.6.183.254:8220
The connection did not worked, i found in a tcpdump MTU problems.
10.6.69.211 is the pods ip address in the k8s network
10.6.183.254 is the k8s svc connected to the fleetservers.
/ # tcpdump -n -i eth0 host 10.6.183.254
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
09:14:06.300375 IP 10.6.69.211.58352 > 10.6.183.254.8220: Flags [S], seq 2041586597, win 64240, options [mss 1460,sackOK,TS val 454230264 ecr 0,nop,wscale 7], length 0
09:14:06.300642 IP 10.6.183.254.8220 > 10.6.69.211.58352: Flags [S.], seq 2097963821, ack 2041586598, win 64308, options [mss 1410,sackOK,TS val 2623417039 ecr 454230264,nop,wscale 7], length 0
09:14:06.309877 IP 10.6.69.211.58352 > 10.6.183.254.8220: Flags [.], ack 1, win 502, options [nop,nop,TS val 454230273 ecr 2623417039], length 0
09:14:06.310296 IP 10.6.69.211.58352 > 10.6.183.254.8220: Flags [P.], seq 1:284, ack 1, win 502, options [nop,nop,TS val 454230274 ecr 2623417039], length 283
09:14:06.310432 IP 10.6.183.254.8220 > 10.6.69.211.58352: Flags [.], ack 284, win 501, options [nop,nop,TS val 2623417049 ecr 454230274], length 0
09:14:06.313483 IP 10.6.183.254.8220 > 10.6.69.211.58352: Flags [P.], seq 1:1565, ack 284, win 501, options [nop,nop,TS val 2623417052 ecr 454230274], length 1564
09:14:06.313503 IP 10.6.69.211 > 10.6.183.254: ICMP 10.6.69.211 unreachable - need to frag (mtu 1280), length 556
I played around with MTU and MSS settings, but i do not want to change the MTU of the eth0 interface in general and found a (temporary) solution with:
iptables -t mangle -A FORWARD -i tailscale0 -o eth0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m tcpmss --mss 1241:1536 -j TCPMSS --set-mss 1240
I am still not sure about the best values for --mss, will figure them out and finetune them if neccessary later after some more research - i just wanted a first working solution.
I wonder that i could not find similar problems here in the forum and also on the tailscale websites and that leads me to the idea that we do some things fundamental wrong.
I know MTU problems from ancient vpn solutions, we run tailscale now for some time and had never this type of problem with it.
Are there any objections for that solution or should we consider to change other things?
Cheers,
Aleks