CT network manager not passing all info in to CentOS8 guest /etc/resolv.conf

Z

zimny

Member
Jun 17, 2019
25
0
21
51
Hi there,

After recent update pve-manager/6.4-8/185e14db I'm having problem with my all CentOS8 CT's.

Basically like in topic. All CT's pve config files have searchdomain and nameserver details in it but guest's CentOS8 /etc/resolve.conf contain only searchdomain.
This give an error6 with curl inside CentOS8 CT's even the basic networking is working and can be checked by ping.

After manually add in CentOS8 CT's gest /etc/resolv.conf nameserver and my LAN DNS IP field, the curl command is working again but this change is not persistent in CT's CentOS8 guests after reboot them.
I have also CentOS7 VM's and they are fine. Also have Debian CT and they are fine too.

All of them are up to date from latest official repos.
All CT's installed from PVE templates btw and I didn't tried any other OS from them.
On all my Linux machines I'm tuning a bit in sysctl.conf and disabling IP6 by net.ipv6.conf.all.disable_ipv6 but don't think this can be relative, I have the same tuning in Debian CT's and all is fine, and should be because I don't have IP6 networks. Also all of them are internals so in PVE CT's firewall option all are disabled -> Firewall, DHCP, NDP, Router Advertisement.

Hope you can update that soon.

Thanks in advance
 
Last edited:
can you post your container config (pct config ID) as well as your versions (pveversion -v) ?
 
Hi Dominik,

root@pr3:~# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.4.119-1-pve)
pve-manager: 6.4-8 (running version: 6.4-8/185e14db)
pve-kernel-5.4: 6.4-3
pve-kernel-helper: 6.4-3
pve-kernel-5.4.119-1-pve: 5.4.119-1
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.4-1
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.9-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.5-6
pve-cluster: 6.4-1
pve-container: 3.3-5
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.2-4
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.4-pve1
======
arch: amd64
cores: 4
features: mount=cifs
hostname: emby
memory: 1792
nameserver: 192.168.0.1
net0: name=eth0,bridge=vmbr1,gw=192.168.0.1,hwaddr=6E:BB:99:C0:CC:0F,ip=192.168.0.25/24,type=veth
ostype: centos
parent: Fresh
rootfs: local-zfs:subvol-325-disk-0,size=15G
searchdomain: local.lan
swap: 256
timezone: host
 
ok thats weird, we do not handle centos differently for the /etc/resolv.conf than other containers
also, the setion written by us should begin with
# --- BEGIN PVE ---
and end with
# --- END PVE ---

can you post the file maybe?

also check if something in your container overwrites the resolv.conf (systemd-resolved? some other dns utility?)
sadly resolv.conf is one file where many tools/services try to insert/write to
 
dcsapak said:
ok thats weird, we do not handle centos differently for the /etc/resolv.conf than other containers
also, the setion written by us should begin with
# --- BEGIN PVE ---
and end with
# --- END PVE ---

can you post the file maybe?

also check if something in your container overwrites the resolv.conf (systemd-resolved? some other dns utility?)
sadly resolv.conf is one file where many tools/services try to insert/write to
Click to expand...
Hi dcsapack,
Under CentOS8 CT in /etc/resolv.conf we have:
# Generated by NetworkManager
search local.lan

I need manually add nameserver field and need to do it all the time when CT reboots

Like I included in my previous posts. Is it any good if someone will try to replicate this?
I have small 5 nodes cluster and on all pve hosts CT's behave with the same error.
 
Could you paste the output of journalctl -xu NetworkManager?
CentOS7 does not depends on NetworkManager, it already worked without. CentOS8 seems to depend on NetworkManager.
NetworkManager needs a full config in /etc/sysconfig/network-scripts, so Resolver-entries must be listed there. It rewrites /etc/resolv.conf!
In my setup, we use IPv6 with RouterAdvertisement with Resolver-entries, so NetworkManager gets the config via RADV. But you do not use it.

So to support resolver in CentOS8, Proxmox must write the config for NetworkManager instead of put it in /etc/resolv.conf:

Bash: 
DNS1=8.8.8.8
DNS2=8.8.4.4
DOMAIN=local.lan

You could try this patch (it includes an other patch, too):

Diff: 
--- /usr/share/perl5/PVE/LXC/Setup/CentOS.pm        2021-06-15 16:18:28.954612241 +0200
+++ /usr/share/perl5/PVE/LXC/Setup/CentOS.pm    2021-06-15 16:31:23.890696662 +0200
@@ -10,6 +10,7 @@
 use PVE::LXC::Setup::Base;
 
 use base qw(PVE::LXC::Setup::Base);
+use UUID;
 
 sub new {
     my ($class, $conf, $rootdir) = @_;
@@ -185,7 +186,7 @@
        my $routes = '';
        my $routes6 = '';
 
-       my $header = "DEVICE=$d->{name}\nONBOOT=yes\n";
+       my $header = "DEVICE=$d->{name}\nONBOOT=yes\nUUID=".UUID::uuid()."\n";
        my $data = '';
        my $bootproto = '';
 
@@ -213,7 +214,7 @@
            if ($d->{ip6} eq 'auto') {
                $data .= "IPV6_AUTOCONF=yes\n";
            }
-           if ($d->{ip6} eq 'dhcp') {
+           elsif ($d->{ip6} eq 'dhcp') {
                $data .= "DHCPV6C=yes\n";
            } else {
                $data .= "IPV6ADDR=$d->{ip6}\n";
@@ -229,6 +230,15 @@
            }
        }
 
+       my ($searchdomains, $nameserver) = $self->lookup_dns_conf($conf);
+       my @nameservers = PVE::Tools::split_list($nameserver);
+
+       for my $i (0 .. $#nameservers) {
+           $data .= "DNS".($i+1)."=$nameservers[$i]\n";
+       }
+       $data .= "DOMAIN=".join(' ', PVE::Tools::split_list($searchdomains))."\n"
+           if $searchdomains;
+
        next unless $data || $bootproto;
        $header .= "BOOTPROTO=$bootproto\n";
        $self->ct_file_set_contents($filename, $header . $data);
 
  • Like
Reactions: mhaluska
Hi all,

Thank you for the explanation from deac. The output of journalctl -xu NetworkManager:
==
-- Logs begin at Thu 2021-06-10 21:55:41 BST, end at Wed 2021-06-16 11:20:21 BST. --
Jun 10 21:55:41 emby systemd[1]: Starting Network Manager...
-- Subject: Unit NetworkManager.service has begun start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit NetworkManager.service has begun starting up.
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.4858] NetworkManager (version 1.30.0-7.el8) is starting... (for the first time)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.4860] Read config: /etc/NetworkManager/NetworkManager.conf
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.4884] bus-manager: acquired D-Bus service "org.freedesktop.NetworkManager"
Jun 10 21:55:41 emby systemd[1]: Started Network Manager.
-- Subject: Unit NetworkManager.service has finished start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit NetworkManager.service has finished starting up.
--
-- The start-up result is done.
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.4902] manager[0x558f72cba080]: monitoring kernel firmware directory '/lib/firmware'.
Jun 10 21:55:41 emby systemd[1]: NetworkManager.service: Failed to reset devices.list: Operation not permitted
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5330] hostname: hostname: using hostnamed
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5334] dns-mgr[0x558f72cb3130]: init: dns=default,systemd-resolved rc-manager=symlink
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5357] manager: rfkill: Wi-Fi enabled by radio killswitch; enabled by state file
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5358] manager: rfkill: WWAN enabled by radio killswitch; enabled by state file
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5359] manager: Networking is enabled by state file
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5360] dhcp-init: Using DHCP client 'internal'
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5366] settings: Loaded settings plugin: ifcfg-rh ("/usr/lib64/NetworkManager/1.30.0-7.el8/libnm-settings-plugin-ifcfg-rh.so")
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5366] settings: Loaded settings plugin: keyfile (internal)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5389] device (lo): carrier: link connected
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5392] manager: (lo): new Generic device (/org/freedesktop/NetworkManager/Devices/1)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5403] manager: (eth0): new Veth device (/org/freedesktop/NetworkManager/Devices/2)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5417] device (eth0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5473] device (eth0): carrier: link connected
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5520] device (eth0): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5531] policy: auto-activating connection 'System eth0' (5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5548] device (eth0): Activation: starting connection 'System eth0' (5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5550] device (eth0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5560] manager: NetworkManager state is now CONNECTING
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5564] device (eth0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5570] device (eth0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5584] device (eth0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5604] device (eth0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5609] device (eth0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5614] manager: NetworkManager state is now CONNECTED_LOCAL
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5626] manager: NetworkManager state is now CONNECTED_SITE
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5627] policy: set 'System eth0' (eth0) as default for IPv4 routing and DNS
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5631] device (eth0): Activation: successful, device activated.
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5637] manager: NetworkManager state is now CONNECTED_GLOBAL
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5668] policy: set-hostname: set hostname to 'emby.zimni.local.lan' (from address lookup)
Jun 10 21:55:41 emby.zimni.local.lan NetworkManager[61]: <info> [1623358541.5691] manager: startup complete
==

I believe the fix should be included in pve updates so fix will be permanent and for all users.
 
Last edited:
zimny said:
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5403] manager: (eth0): new Veth device (/org/freedesktop/NetworkManager/Devices/2)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5417] device (eth0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5473] device (eth0): carrier: link connected
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5520] device (eth0): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5531] policy: auto-activating connection 'System eth0' (5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5548] device (eth0): Activation: starting connection 'System eth0' (5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03)
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5550] device (eth0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5560] manager: NetworkManager state is now CONNECTING
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5564] device (eth0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5570] device (eth0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5584] device (eth0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5604] device (eth0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5609] device (eth0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5614] manager: NetworkManager state is now CONNECTED_LOCAL
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5626] manager: NetworkManager state is now CONNECTED_SITE
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5627] policy: set 'System eth0' (eth0) as default for IPv4 routing and DNS
Jun 10 21:55:41 emby NetworkManager[61]: <info> [1623358541.5631] device (eth0): Activation: successful, device activated.
Click to expand...
System eth0 is the name of the interface, if there is no/not found a config.
So the ifcfg-eth0 is ignored.
I believe the fix should be included in pve updates so fix will be permanent and for all users.
Click to expand...
I've already filed a bug: https://bugzilla.proxmox.com/show_bug.cgi?id=3471
 
  • Like
Reactions: mhaluska and zimny
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom