After Upgrading from v8 to 9.2, one node cannot resolve DNS

[DBailey635]

Hi,

I've just updated five Dell hardware- and config-identical Proxmox VE nodes from v8 to 9.2, and while four of them work fine, the fifth one cannot resolve DNS. I cannot therefore apply further updates.

The output of cat /etc/resolv.conf on the broken node is:

# This is /run/systemd/resolve/resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

# No DNS servers known.
search .

The other identical nodes correctly upgraded and the output of cat /etc/resolv.conf file was:

search local
nameserver 192.168.5.1

I have already removed systemd-resolved, to no avail.

Any ideas on how to fix the broken node?

 

[bbgeek17]

Your /etc/resolv.conf is a symlink to systemd-resolved's generated file (as evident by the text at the start), and resolved has no DNS servers, hence "No DNS servers known." Removing the package leaves the symlink behind, pointing at a stale file.


Replace the symlink with a real file:

ls -l /etc/resolv.conf
rm /etc/resolv.conf
printf 'search local\nnameserver 192.168.5.1\n' > /etc/resolv.conf

Then test with getent hosts deb.debian.org and apt update. Also make sure the DNS server is set under Node > System > DNS in the GUI so it persists.

To see why this node differs, compare ls -l /etc/resolv.conf against a working one.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[DBailey635]

Thanks. That worked.

Any idea why systemd-resolved got installed during the upgrade on just this one machine? I'm pretty sure it wasn't there before (although these machines are available to our MSc students, and they do funny things when we aren't watching them like a hawk).

 

[bbgeek17]

Any idea why systemd-resolved got installed during the upgrade on just this one machine? I'm pretty sure it wasn't there before (although these machines are available to our MSc students, and they do funny things when we aren't watching them like a hawk).

Its not part of our installation here that went through 7>8>9 upgrades. I don't know off hand whether it is part of any standard PVE install, I suspect not.
Somehow you, or someone on your team brought it in. I'd examine the journal and dpkg logs to see when it happened. And put better restrictions in place.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[fabian]

/var/log/apt/history.log (or one the rotated copies) will tell you when/how it got installed