Same LAN, multiple devices. Only one can make direct connections via Tailscale

SUPPORT QUESTIONS
1

I’m trying to debug a situation in my home with Tailscale.

I have multiple nodes in the LAN that have the Tailscale daemon running, including (among others, less used):

  1. My laptop (macOS)
  2. A Linux server (Debian 10)

It seems that only one of the two at a given time can connect to other nodes in my Tailscale network without going through a relay–it seems only the first one to establish a connection wins. If I try to connect to external nodes from both my laptop AND the server, one of the two goes through a relay.

I’ve been trying to debug this and I can’t figure out what the issue is. It’s something wrong within my LAN, because the remote nodes have a public IP and all firewall ports open.

My setup in the LAN is a bit complicated, because I have double NAT: because of my ISP, I essentially have two routers. One of the two is a pfSense firewall, which I know is often cause of problems with Tailscale, but I have enabled NAT-PMP.

Any suggestion on how to fix this?

2

It is likely that all of the nodes attempting to use NAT-PMP to open port 41641 isn’t working. You can make the nodes on the tailnet each choose a random port by setting RandomizeClientPort in Tailscale, which might help.

{
  "RandomizeClientPort": true,

  "ACLs": [
3

@DGentry Thanks for the response.

Yesterday, just before your message, I got the assumption it had to do with a conflict on ports. So I actually changed the port tailscaled is listening on, hardcoding a new value, and then created a hard NAT rule in the inner firewall. (Sadly I have no control over the outer firewalls)

That doesn’t seem to solve the issue, however, as I’m still going through a relay.

I can’t wrap my head around this issue. From my laptop AND from my server in the LAN I can successfully make direct connections to the remote nodes on the port Tailscale is listening on:

$ nc -vu x.x.x.x 41641
Connection to x.x.x.x port 41641 [udp/*] succeeded!

If at least one of the two nodes has a port open with no firewall or NAT obstructing it, shouldn’t that make direct connections always possible?

4

After more fiddling with firewall ports, I was able to make it work. Still not exactly sure how, but setting static ports for tailscaled and ensuring they’re all open in all firewalls (at least those I can control) seems to have fixed it.

5

After rebooting the server, the problem came back.

Every other node in the network can make direct connections to the external server, which has a port open in the firewall. Except this specific server in the LAN that seem to always go through a relay. I’m not sure what else to do.

6

Here’s some commands I’ve tried running.

From the remote server (atlas), which is able to accept direct connections on port 31253. The port is open on the firewall.

user@atlas$ tailscale netcheck
Report:
	* UDP: true
	* IPv4: yes, X.X.X.X:32842
	* IPv6: no
	* MappingVariesByDestIP: false
	* HairPinning: false
	* PortMapping:
	* Nearest DERP: Frankfurt
	* DERP latency:
		- fra: 22.1ms  (Frankfurt)
		- lhr: 31.8ms  (London)
		- nyc: 107.9ms (New York City)
		- ord: 126.2ms (Chicago)
		- dfw: 137.8ms (Dallas)
		- sfo: 167.7ms (San Francisco)
		- sin: 169.2ms (Singapore)
		- sea: 172.1ms (Seattle)
		- tok: 225ms   (Tokyo)
		- sao: 233.5ms (São Paulo)
		- blr: 262.5ms (Bangalore)
		- syd: 283.7ms (Sydney)

user@atlas:~$ tailscale status
100.1.0.1 atlas                user@  linux   -
100.2.0.1  accesspi             user@  linux   active; direct <public_ip_X>:29679
100.2.0.2    macbook-pro user@  macOS   active; direct <public_ip_X>:42676, tx 1378736 rx 566980
100.2.0.3    truffle              user@  linux   active; relay "sin", tx 22612 rx 13564

In the LAN, I have 3 nodes currently connected to Tailscale, as you can see above:

  • “accesspi” is a Raspberry Pi, running Rasperry Pi OS “bullseye”.
  • “macbook-pro” is my laptop, running macOS
  • “truffle” is the server on the local LAN, running Debian 11

All the 3 nodes above have the same external public IP (<public_ip_X> is the same).

The remote server, atlas, can connect to all 3 of them, but for some reason connections to truffle seem to always go through a relay (usually “fra” or “sea”, this time “sin”!)

The LAN I’m in right now is a “difficult network”, with double NAT. The “inner” firewall is a pfSense box, and then my ISP is putting me behind another NAT (sadly, my only option here is T-Mobile Home Internet).

To make things easier, I configured truffle to use Tailscale on a fixed port, and then I opened that port in the pfSense firewall, creating a 1:1 NAT. I’m still behind one NAT, but at least it shouldn’t be double-NAT’d. Yet, I’m stuck with using a relay.

This is really odd and at this point I can’t explain it. If the remote node has no firewall at all, why am I not able to establish a direct connection to that from all nodes in my LAN?

7

i think i’m having the same issue. were you able to figure out a solution?

8

One cause of this behavior where only a single node can make direct connections is NAT-PMP/UPnP/etc, if they are fighting over port 41641. If that is the case, there are two ways to resolve it:

  • tailscaled takes a --port=N argument. Setting each node to use a port other than the default 41641 would likely allow both of them to get NAT-PMP entries simultaneously.
  • in https://login.tailscale.com/admin/acls one can add a setting which tells all devices on the tailnet to choose a random port and not use 41641:
{
"RandomizeClientPort": true,
"ACLs": [
9

Yes I was able to figure out a solution, although it is sometimes flaky (if the tailscaled daemons are restarted, sometimes they fall back to using a gateway).

I “solved” it by making sure that the remote node could accept direct connections from the Internet. It’s set to a port different from 41641 (changed by editing the tailscaled config file), and that port is open in every firewall so it can accept connections from the Internet.

On this side I still have double NAT.