Pfsense DNS Resolution - VM VLAN

[vinny147]

Hi -

TLDR: I cannot reach my VMs via hostname when I SSH but I can via IP. How do I fix this?

Network Setup:

• Pfsense Firewall (used for DHCP as well)
• Ubiquiti Managed Switch

My Proxmox Host is on dedicated hardware and the management interface is on its own VLAN (60) with VMs assigned on a different VLAN (80). When I try to ssh into a VM on VLAN 80 from my personal computer (VLAN 50) I can only do so via IP address, not hostname. If I can try to SSH into physical hardware on other VLANs from my personal computer it works fine, resolving the hostname only has issues if it's a VM on VLAN 80.

Things I've troubleshooted:
1. DHCP is enabled for VLAN 80 on the firewall.
2. From a VM on VLAN 80, I can resolve websites such as google.com.
3. From a VM on VLAN 80, I cannot resolve hostnames on my LAN.
4. From my personal computer on another VLAN, I cannot resolve hostnames on VLAN 80 but I can for all other VLANS on my network.
5. For VLAN 80 I put an any/any rule in place to make sure ACLs aren't blocking me.

Here's how I have my networking setup in Promox.



I'm not sure where to troubleshoot from here.....

Thanks in advance for any advice!

 

[tempacc375924]

Hi -

TLDR: I cannot reach my VMs via hostname when I SSH but I can via IP. How do I fix this?

Network Setup:

• Pfsense Firewall (used for DHCP as well)
• Ubiquiti Managed Switch

My Proxmox Host is on dedicated hardware and the management interface is on its own VLAN (60) with VMs assigned on a different VLAN (80). When I try to ssh into a VM on VLAN 80 from my personal computer (VLAN 50) I can only do so via IP address, not hostname. If I can try to SSH into physical hardware on other VLANs from my personal computer it works fine, resolving the hostname only has issues if it's a VM on VLAN 80.

Things I've troubleshooted:
1. DHCP is enabled for VLAN 80 on the firewall.
2. From a VM on VLAN 80, I can resolve websites such as google.com.
3. From a VM on VLAN 80, I cannot resolve hostnames on my LAN.
4. From my personal computer on another VLAN, I cannot resolve hostnames on VLAN 80 but I can for all other VLANS on my network.
5. For VLAN 80 I put an any/any rule in place to make sure ACLs aren't blocking me.

Here's how I have my networking setup in Promox.

View attachment 58387

I'm not sure where to troubleshoot from here.....

Thanks in advance for any advice!

This is more of a networking question, but which DNS server exactly would you expect to resolve the names for you? It's not done automatically, PVE is all IP based (the node themselves, let alone guests), the names are not FQDNs and the hostnames (within the VM) resolve basically locally (depending on what they were made from) at best.

You need a DNS server for the names to resolve that the VLAN 50 (and 80 if you want VMs to be able to resolve each other). You can run one any place of your choice, even in another VM or the pfSense (as you already use that for DHCP). The managed switch from UI is Unifi or Edge series?

 

[tempacc375924]

View attachment 58387

You may want to have the VMs get their own IPs via DHCP from the pfSense, they probably already do. Not sure how pfSense does it internally, but if you were to use dnsmasq for DHCP+DNS, you could literally have the names be resolved sort of automatically for all the given out IPs (and you can statically reserve them, but don't have to). But then of course it's all dependent on your reservations you make on the box, not with names you give your guests in PVE.

 

[vinny147]

This is more of a networking question, but which DNS server exactly would you expect to resolve the names for you? It's not done automatically, PVE is all IP based (the node themselves, let alone guests), the names are not FQDNs and the hostnames (within the VM) resolve basically locally (depending on what they were made from) at best.

You need a DNS server for the names to resolve that the VLAN 50 (and 80 if you want VMs to be able to resolve each other). You can run one any place of your choice, even in another VM or the pfSense (as you already use that for DHCP). The managed switch from UI is Unifi or Edge series?

So I'm actually using Pfsense for DHCP and as a DNS Resolver, each VLAN is enabled for use with pfsense DNS, and that's what I'd like to keep. Generally I'd expect each vlan to use it's respective gateway (192.168.x.1) to get to DNS. As of right now I'm just spinning up a VM (ubuntu in this case) and expecting that the VM is inheriting the correct DNS configuration, is this a bad assuming?

Could it be my DNS settings in Proxmox?


The managed switch is UI (Lite 8 PoE). The used port is set to default mode to support multiple vlans.

 

[tempacc375924]

The first time I see someone using correctly home.arpa!

So I'm actually using Pfsense for DHCP and as a DNS Resolver, each VLAN is enabled for use with pfsense DNS, and that's what I'd like to keep. Generally I'd expect each vlan to use it's respective gateway (192.168.x.1) to get to DNS.

All makes sense. I would edit title of your thread to include pfSense so that someone expert in that might chip in.

As of right now I'm just spinning up a VM (ubuntu in this case) and expecting that the VM is inheriting the correct DNS configuration, is this a bad assuming?

I think we need to clarify (maybe I am just slow tonight) that the expected is (correct me where wrong):

1) VMs should start up, bridged onto the VLAN 80
2) VMs should go ask out for IP lease from DHCP server in that bridged VLAN 80
3) VMs should get their DNS resolver from the DHCP response
4) DHCP server on pfSense will be giving out addresses randomly (or static leases?)
5) DNS server on pfSense will have the names-IPs filled in by you manually (or do you expect to update itself from a DHCP Option that the client sends back to the server)?

Could it be my DNS settings in Proxmox?
View attachment 58391

As long as your VMs are on the bridged VLAN 80, I would not worry about this. I would go get into any VM's console and simply have a look what it thinks itself in terms of /etc/resolv.conf and ip a (or netplan, nmcli or whatever applicable depending on what that VM is.

The managed switch is UI (Lite 8 PoE). The used port is set to default mode to support multiple vlans.

Just curious, as long as your VLANs are working fine, the switch will not be of our interest in this post further.

 

[vinny147]

I think we need to clarify (maybe I am just slow tonight) that the expected is (correct me where wrong):

1) VMs should start up, bridged onto the VLAN 80
2) VMs should go ask out for IP lease from DHCP server in that bridged VLAN 80
3) VMs should get their DNS resolver from the DHCP response
4) DHCP server on pfSense will be giving out addresses randomly (or static leases?)
5) DNS server on pfSense will have the names-IPs filled in by you manually (or do you expect to update itself from a DHCP Option that the client sends back to the server)?

This is correct.
4) For the purpose of troubleshooting I've tried VMs with dynamic IP allocation & static, same result.
5) It picks up the hostname automatically.

I've tried performing a DNS lookup with the hostname & IP from Pfsense and the VMs on VLAN 80 do not resolve.

 

[tempacc375924]

This is correct.
4) For the purpose of troubleshooting I've tried VMs with dynamic IP allocation & static, same result.
5) It picks up the hostname automatically.

I've tried performing a DNS lookup with the hostname & IP from Pfsense and the VMs on VLAN 80 do not resolve.

Ok I just want to dissect this ... your original description of the issue was:

When I try to ssh into a VM on VLAN 80 from my personal computer (VLAN 50) I can only do so via IP address, not hostname. If I can try to SSH into physical hardware on other VLANs from my personal computer it works fine, resolving the hostname only has issues if it's a VM on VLAN 80.

Adding it all up. You have an issue to do a DNS lookup of the VM's hostname even though you believe the pfSense "picks them automatically" upon DHCP assignment. And I assume the DNS lookups that are failing you are made from the other VLAN of your computer.

1) Did you check if the DNS lookup works from a VM itself?
# dig @192.168.8.1 hostname

2) Did you check the same from your station/computer?

3) What did you mean by "lookup with the IP from pfSense"? Can you ping the machines? Do you see the DHCP assign addresses? Can you console check within the VM if they get connectivity out as suggested above?

 

[tempacc375924]

(Oh yeah, I think I cannot see your post in the public list, it would have likely been flagged as potential spam and won't get public till approved manually later on.)

 

[vinny147]

I have good news, I managed to fix it.

Good ol fashion flush DNS and magic happened. I appreciate you thinking through this with me!!

 

[tempacc375924]

I have good news, I managed to fix it.

Good ol fashion flush DNS and magic happened. I appreciate you thinking through this with me!!

Good to hear. Not sure about your default settings, but lots of people do not realise it is perfectly reasonable to have e.g. 5 minutes set TTL on the DNS for e.g. VMs. Same for DHCP lease time, especially when setting up something from scratch, it helps with being able to assume within 5 minutes everything is up to date and propagated throough. It will not disrupt anything.

 

[vinny147]

Good to hear. Not sure about your default settings, but lots of people do not realise it is perfectly reasonable to have e.g. 5 minutes set TTL on the DNS for e.g. VMs. Same for DHCP lease time, especially when setting up something from scratch, it helps with being able to assume within 5 minutes everything is up to date and propagated throough. It will not disrupt anything.

Not a bad suggestion! I'll get this setup.