RDS Guide - not connecting

1

I’m following the RDS access instructions (here). I confirm my EC2 node is showing up in Tailscale. I confirm that, even after removing 22 inbound, I’m able to connect to my EC2 when logged in to Tailscale via its private IP. :confetti_ball:

However, I’m unable to connect to my single RDS instance. After configuring split DNS (172.3.0.2 us-west-2.compute.internal) I attempt mysqlsh --uri=admin@database-1.XYZ.us-west-2.rds.compute.internal:3306. The client prompts for a password, but then it immediately returns Unknown MySQL server host.

I’ve confirmed that my RDS security group allows inbound from my EC2. They’re in the same VPC. They are in different AZs (RDS is in us-west-2d and EC2 is in us-west-2c) and I’ve even successfully connected to RDS directly from my EC2. I know they can talk to one another.

My hunch is it’s related to split DNS. I believe us-west-2c|d would use the same DNS server, but the fact that it can’t seem to find the host tells me it’s DNS related. I’d expect if it were network related once I’m within AWS, it would at least hang and time out.

Help! Thank you

1 Like
2

Having similar issues. I can SSH to the EC2 (AWS Linux, with Tailscale installed) subnet router but can’t connect to RDS via tailnet.

The EC2 box can connect to the RDS directly. so the AWS networking side appears to be good.

What’s next?

3

Tailscale let me know it’s due to DNS changes in AWS and the directions may no longer work. There’s no solution at this time. Scary to think a database connection could suddenly be dropped indefinitely.

1 Like
4

Thanks for that. Did they give any indication of a timeframe for some form of resolution one way or another?

5

They did not unfortunately.

6

Think I figured out a possible solution.

What worked for me was to use split DNS to override the public search domain for my database (i.e. us-west-1.rds.amazonaws.com) and set it to the private subnet IP as described in the tutorial (i.e. 172.31.0.2)

My ingress node in AWS advertises 172.31.0.0/16 as a route

Now my other devices on the Tailnet (provided that they’re accepting routes), properly resolve the private IP of the database.

7

I was excited to try this but it isn’t working for me. :thinking:

8

Looks like I have the same issue. I followed all steps, and my RDS private IP address is event being resolved correctly, but I still can’t connect to it from my laptop, only from the EC2 instance. :frowning:
Are there any updates on this issue?

9

Hello,

It’d be helpful if you could post all the steps and commands you run.

When you say: “I can’t connect from my laptop”, does that mean you cannot connect via IP level (no connectivity) or at dns level. If you can connect via IP but not hostname then it is a DNS issue. It is a matter of finding the correct split DNS to revolve the AWS queries. You have to also make sure you are allowing traffic (check your ACL setup). If you don’t use the ACL then it should be alright as everything is connected.

I setup what you are trying to do a few weeks back and it works well for me. I have to point out that my RDS is public, meaning it exposes a public endpoint. The kb document talks about a private endpoint which I believe what you have setup.

It is important to keep in mind the high level steps to set this up. If I look at my notes the steps are:

  1. Spin an EC2 instance and add it to your tailnet.
  2. Add a security group to make sure we have access to the RDS subnet from our node.
  3. Enable split DNS so we resolve AWS queries via an AWS dns server.
  4. Remove public ssh access from the new node
  5. Profit

Things that you can check:

  1. Confirm you are running a private RDS (and not public). If it is public, you should not be able to get a private IP for the service.
  2. Can you psql to the IP address of the RDS service from the EC2 box?
  3. Can you do the same using the hostname?
  4. Did you create the security group?
  5. Are the ACL entries correct so you allow traffic between the nodes that want to use the RDS?

I hope that helps,
-drd