I am currently using pfSense for a home network with 5 web servers for family and friends.
I have:
- 2 WANs, Fiber and Cable, and they are configured for fail-over
- DDNS to flip the DNS IPs when they fail-over or if the IP is refreshed
- HAProxy to handle the subdomains I configured for each web server
- a wildcard cert to handle the https within HAProxy and
- which redirects to the subdomained web servers using plain old http and
- handles auto updates of letsencrypt cert in pfSense, not at every web server
I also have my personal OpenVPN connection into pfSense so I can access and manage the servers from my laptop or my phone when I am travelling.
The above works great but now I am looking at moving the setup to a new location, a rural farm, which
has no wired services and only has 2 “reasonable” wireless services, Starlink and T-Mobile Home Internet, both of which use CGNAT. I am considering using these 2 services as my 2 WANs.
Is Tailscale a reasonable answer for this problem?
pfSense has installable packages for both tailscale and wireguard. How might I use one or both of these to get around the CGNAT problem of hosting my own servers from behind my firewall; behind CGNAT?
Failover?
Subdomain redirection?
Letsencrypt auto updates?
Or do I need to throw it away and start over? OR, give up? 
Thanks for any tips,
Chris.
1 Like
CGNAT is not really an issue with TS and a provider that also uses CGNAT. TS only routes the specific IPs that are part of your TS network via Tailscale, everything else goes to your default route/gateway. I use Starlink and use TS in various physically separated networks (AWS VPCs, my local network at home/office) with subnet routing, and it works fine. Just don’t add routes manually which route everything from the CGNAT subnet/CIDR block via Tailscale 
Thanks for responding.
I get it that outbound access should work OK. I figure that streaming netflix or browsing the web should be no problem with CGNAT and pass through the WAN connections on my pfSense device. Failover will probably work, also.
I’ve used ngrok but that creates a path through ngrok’s servers directly to a server under my control. Nice to show someone development progress on a web site. I used it to be able to access my raspi at the Maker Faire when I wanted to update at night when everything was locked up. It’s straight forward, works great, but I wouldn’t use that for each server in my “production” network.
Some questions, please:
- Does the “separate network” in TS parallel that which I have running through my pfSense Firewall?
- Does TS bypass the firewall?
- I still don’t see how TS can provide access for my friends (who will never, should never, know anything about TS) to the web servers behind my firewall
- where does letsencrypt get checked/terminate? In TS? In pfsense? In each individual server?
- Where is the location of the configuration for the subdomain redirection to my web servers? Is that in TS?
Examples:
- https://web.mydomain_org redirects to my nodejs/express web server on port 3000
- https://music.mydomain_org redirects to my sonic music server on port 4040
- https://drp.mydomain_.org redirects to my nextcloud server on port 80
- https://cams.mydomain_org redirects to my zomeminder security camera server on port 80
Just having trouble understanding the locality and the integration between pfsense, TS, and the CGNAT workaround.
Thanks again for any tips,
Chris.
PS: strange urls to trick the requirement of no more than 2 links; they aren’t real anyway.
Does the “separate network” in TS parallel that which I have running through my pfSense Firewall?
Yes, your tailnet runs in parallel, meaning that the rest of your network requests still go their normal route and only explicit connections for a tailnet node will be answered by your tailnet.
You can enable nodes as an exit node though, which makes it possible to reroute all traffic through it.
More details about that here:
https://tailscale.com/kb/1103/exit-nodes
Another nice feature is subnet routing which, when enabled on a tailnet node, allows anyone who connects to the node (via tailscale) to access the network behind that node:
https://tailscale.com/kb/1019/subnets
You can use this to e.g. connect two separate sites’ networks together. (I believe there’s actually a separate article rearding site-to-site as well).
Does TS bypass the firewall?
Yes, it usually does by using various techniques.
In some situations you might need to open specific ports.
This is a helpful article explaining everything:
https://tailscale.com/kb/1181/firewalls
I still don’t see how TS can provide access for my friends (who will never, should never, know anything about TS) to the web servers behind my firewall
There’s currently a beta feature called “funnel” which allows you to expose services to the internet via HTTPS, TCP or secure TCP:
https://tailscale.com/kb/1223/tailscale-funnel
where does letsencrypt get checked/terminate? In TS? In pfsense? In each individual server?
According to the previously mentioned article about funnel (In the “How it works” section), your Tailscale node terminates the TLS certificate.
Where is the location of the configuration for the subdomain redirection to my web servers?
If I understand your question correctly, then that is also answered in the funnel article (i.e. public DNS records in their ingress servers, that then pass the raw connections along to your tailnet).
@bluefish
That’s a very thorough response. I’ll look up the references over the weekend.
Generally, it leads me to believe that I will be tossing all of the letsencrypt, haproxy, subdomain redirects, and private access openvpn configurations in pfSense and starting over with full dependency on tailscale to afford access to my self hosted servers.
Thank you,
Chris.
1 Like
@blufish
Looking at the tailscale exit nodes, subnets, firewalls, and the new funnel feature now leaves me feeling like tailscale might not help.
I read the notes on exit nodes and it seems summarized by, “like a consumer VPN” - not what I’m looking for right now.
On the subnet reference, I see this, “the client app is installed directly on every client, server, and VM in your organization”. Whoa? What the heck is this for? I only have 5 servers I’d like to expose publicly from behind my pfsense firwall/router which has a CGNAT WAN service. I also have no need of site to site connectivity, for now.
Regarding firewall bypass, I see the notes about how to use TS with pfsense and enabling NAT-PMP and UPnP. This seems reasonable, I suppose.
There is a package now available in pfsense for TS which leads to a possible solution. The hope is that with TS installed on pfsense, it will itself be the termination point for all of TS connectivity into my home network and I will not have to install the TS client on every server. More research required here.
In the TS package I can choose to Advertise Exit Node, Accept Subnet Routes, and define multiple Advertised Routes.
I also looked at funnel it might be part of the solution but I will need to research more deeply.
LetEncrypt - Enabling HTTPS · Tailscale
Looks like I will have to set let’s encrypt web install and updates on each web server and do all the tedious management of updating every 3 months. On my current setup it uses acme and haproxy in pfSense to manage all that in one place.
Damned you, CGNAT!
Thank you,
Chris.
You don’t need to install TS on every device in your LAN if you use a subnet router. Tailscale just encourages that. But you really just need to put TS on one machine in your LAN and enable it as a subnet router.
If you’re then connecting to your tailnet from outside your LAN, you’ll have access not only to the machine inside your network that has TS installed, but also to the rest of your LAN that the internal machine is a subnet router for.
So it’s not only site-to-site, but you can also do device-to-site.
Integrating tailscale into your firewall or router could work as well I guess, though I haven’t tried that myself yet, so I can’t give you any more specific pointers on that.
@bluefish @guenther.wieser …
I now have 2 WAN services, Starlink and T-Mobile Home Internet (both CGNAT services), configured in my remote Netgate 2100 pfsense firewall with failover. I also have tailscale installed as a package in pfsense and I can now access and manage the firewall remotely. With subnet routing I can get to all accessible servers behind the firewall.
Meanwhile the occupants of two homes at the remote site are able to stream movies with no detectable interruptions. From the Netgate 2100 LAN connections are 2 tp-link cpe510 APs connected to respective clients each with an Archer A8 wifi router. The performance from the WAN services are in the range of 140/30 Mbps but is only about 30-40Mbps in and 10Mbps out. I’ll be looking at tuning the performance on the wifi bridges.
But, I have happy TV watchers and I can access and manage the firewall remotely. So it is good all around.
Thanks for the encouragement.
1 Like