[solved] RHEL8 as subnet router not working with firewall enabled

I’ve been trying to test out the subnet router feature, since this will be critical for my org using TS to replace OpenVPN-AS. From the TS documentation, it says something along the lines of “no firewall configuration is needed, it’ll just work”. But I am only seeing subnet routing work when I disable the firewall. This is a default, minimal, fresh RHEL8 server. Followed the TS docs to the letter. But if I have the firewall active, any attempt to SSH to a machine on the subnet is immediately denied. If I disable firewall and restart tailscale (on the subnet router) I can SSH (or RDP, http, etc) without a problem to machines on that subnet.

Since I have reread the docs many times, I’ve come here (and googled widely) looking for info. I’ve seen a bunch of posts that incidentally mention that they of course disable firewall on their subnet router…like that’s just a given. What am I missing? I can’t keep the firewall disabled on the server in question, as a matter of policy, so I’d love some help on this.

Thanks!

Does RHEL 8 use firewalld? I remember that Centos 8 machines had similar difficulties because of firewalld blowing away Tailscale’s iptables config. Another workaround is to enable userspace networking mode, which you can do by editing /etc/default/tailscaled's FLAGS variable to contain --tun=userspace-networking. This will make it slightly less seamless (you need to use a SOCKS proxy instead of direct network sockets for connecting to things on Tailscale on that machine in particular), but it should work fine with firewalld.

It’s worth noting that disabling firewalld doesn’t actually disable the iptables firewall in RHEL 8, it only disables firewalld’s easy configuration interface on top of iptables. But yeah, disabling firewalld is kind of a measure of last resort.

Yeah, the hope is to also still be able to have OSSEC auto block baddies, which I assume uses firewalld (not direct iptable manipulation) on RHEL8…but I should dig into that too. Of course, OSSEC isn’t even in play yet, until I get the TS part sorted out.

Thanks for mentioning the userspace option. I’ll have to think through that, but it seems less than ideal.

In digging deeper into this, what you said isn’t quite right, and this might help?

In Centos/RHEL 8, firewalld is the frontend, but the default backend is nftables, not iptables. (iptables was the backend in 7, and the iptables command is still there for compatibility, but it’s actually going to nft.)

Also, it seems like the default is for firewalld to “cleanup” when it’s stopped. It certainly looks like that means the nft ruleset is wiped, and set back to a default accept across the board. This makes sense from a user perspective too, where if you disable the firewall, you expect all traffic to get through.

Next, this setting in firewalld.conf…

# FlushAllOnReload
# Flush all runtime rules on a reload. In previous releases some runtime
# configuration was retained during a reload, namely; interface to zone
# assignment, and direct rules. This was confusing to users. To get the old
# behavior set this to "no".
# Default: yes
FlushAllOnReload=yes

didn’t exist in CentOS/RHEL7.

I thought maybe that change in behavior might account for the trouble with subnet routing in 8. Like if TS is using direct rules that would’ve survived a reload on CentOS7, but now get wiped during reload on CentOS8. But changing this to “no” and restarting everything didn’t have any apparent impact on TS, or on the nft ruleset.

I feel like there’s gotta be some solution here, but I’m coming at it with no insight into the inner workings of TS, and have never had to dig this deep into firewalld/iptables/nftables before.

Thanks again for any help. And Happy Thanksgiving to anyone who celebrates it.

Just to close the loop, I ended up finding a solution. I noticed from firewall-cmd --list-all that forward and masquerade were both reporting “no”. I read the docs to confirm that masquerade implies forward, and confirmed that just doing…

firewall-cmd --permanent --add-masquerade

…fixes the problem. No need to separately --add-forward.

Happy Thanksgiving!