Random network pb on VM (samba related ?)

reabran

New Member
Sep 15, 2021
2
0
1
45
Hello

I've been trying to solve a problem (bug ?) with Proxmox VMs for a few hours now (ok maybe somme days ?), it's driving me crazy. I'll try to be clear and as precise as possible to explain.
This bug is actually quite simple to reproduce once identified (in my context), what at first seems to be a random network problem. Symptom: one time the VM is on the network (ping ok from a client computer), one time no (no ping), one time yes, etc.
Here are my configurations and the steps to reproduce the bug.

Physical server, Proxmox installed from ISO in 6 originally and upgraded to 7 with dist-upgrade method. The machine is clean, there are only 2 test VMs on it (to try to solve this bug). While one VM is used to reproduce the network pb, the other is just here to check if my network si going well (and it is).

Bash:
root@hostprxmx:~# pveversion -v
proxmox-ve: 7.0-2 (running kernel: 5.11.22-4-pve)
pve-manager: 7.0-11 (running version: 7.0-11/63d82f4e)
pve-kernel-5.11: 7.0-7
pve-kernel-helper: 7.0-7
pve-kernel-5.4: 6.4-5
pve-kernel-5.3: 6.1-6
pve-kernel-5.11.22-4-pve: 5.11.22-8
pve-kernel-5.11.22-3-pve: 5.11.22-7
pve-kernel-5.4.128-1-pve: 5.4.128-2
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.10-1-pve: 5.3.10-1
ceph-fuse: 14.2.21-1
corosync: 3.1.2-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.21-pve1
libproxmox-acme-perl: 1.3.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-6
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-11
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.9-2
proxmox-backup-file-restore: 2.0.9-2
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-9
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-2
pve-firmware: 3.3-1
pve-ha-manager: 3.3-1
pve-i18n: 2.5-1
pve-qemu-kvm: 6.0.0-4
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-13
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1

Bash:
root@hostprxmx:~# more /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual
#BCM57800 10Gb DAC

iface eno2 inet manual
#BCM57800 10Gb DAC

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.2/24
        gateway 192.168.1.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

Bash:
root@hostprxmx:~# ip -c link

6: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 4c:d9:8f:a3:1f:30 brd ff:ff:ff:ff:ff:ff
    altname enp25s0f0
7: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 4c:d9:8f:a3:1f:32 brd ff:ff:ff:ff:ff:ff
    altname enp25s0f1
10: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 4c:d9:8f:a3:1f:30 brd ff:ff:ff:ff:ff:ff
17: tap122i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr122i0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether ea:4e:93:cf:92:c4 brd ff:ff:ff:ff:ff:ff
18: fwbr122i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether ee:42:1a:07:d5:48 brd ff:ff:ff:ff:ff:ff
19: fwpr122p0@fwln122i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 7a:aa:f0:fd:a3:95 brd ff:ff:ff:ff:ff:ff
20: fwln122i0@fwpr122p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr122i0 state UP mode DEFAULT group default qlen 1000
    link/ether 02:ca:47:e8:cb:52 brd ff:ff:ff:ff:ff:ff
21: tap119i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr119i0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 82:57:7b:a6:12:d1 brd ff:ff:ff:ff:ff:ff
22: fwbr119i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 86:93:ef:74:b6:20 brd ff:ff:ff:ff:ff:ff
23: fwpr119p0@fwln119i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 0a:08:0f:c2:af:f3 brd ff:ff:ff:ff:ff:ff
24: fwln119i0@fwpr119p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr119i0 state UP mode DEFAULT group default qlen 1000
    link/ether 3e:fc:fb:e6:1b:c4 brd ff:ff:ff:ff:ff:ff

Bash:
root@hostprxmx:~# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
Bash:
root@hostprxmx:~# ebtables -L
Bridge table: filter

Bridge chain: INPUT, entries: 0, policy: ACCEPT

Bridge chain: FORWARD, entries: 0, policy: ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

I installed a VM guest, with debian 11, minimal version, no packages, just ssh. The guest work like a charm, network is ok. Host / Guest / client are on different vlan by the way but no pb here as you can see.

Bash:
root@guest-vm:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 4a:c3:36:dc:5c:34 brd ff:ff:ff:ff:ff:ff
    altname enp0s18
    inet 192.168.1.3/24 brd 192.168.1.255 scope global ens18
       valid_lft forever preferred_lft forever

from another computer in same vlan
Bash:
root@192.168.1.4:~# mtr guest-vm
                                                                                    Packets               Pings
 Host                                                                              Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.1.1                                                                     0.0% 80123    0.6   0.3   0.3 623.0   9.2
 2. 192.168.1.1                                                                     0.0% 80122    0.6   0.3   0.3 651.2   9.0
 3. guest-vm                                                                        0.0% 80122    0.4   0.2   0.2 1002.   5.5

Now I want to make a samba files server on VM so I install some packages,
Bash:
root@guest-vm:~# apt install krb5-user samba winbind libnss-winbind

These few packages installed all these dependencies too :
Bash:
attr bind9-host bind9-libs ibverbs-providers krb5-config libavahi-client3 libavahi-common-data libavahi-common3 libboost-iostreams1.74.0 libboost-thread1.74.0 libcephfs2 libcups2 libfstrm0 libgfapi0 libgfrpc0 libgfxdr0 libglusterfs0 libgpgme11 libgssrpc4 libibverbs1 libicu67
  libkadm5clnt-mit12 libkadm5srv-mit12 libkdb5-10 libldb2 liblmdb0 libmaxminddb0 libnl-3-200 libnl-route-3-200 libnspr4 libnss3 libprotobuf-c1 libpython3.9 librados2 librdmacm1 libtalloc2 libtdb1 libtevent0 liburing1 libuv1 libwbclient0 libxml2 libyaml-0-2 python3-dnspython python3-gpg
  python3-ldb python3-markdown python3-pygments python3-requests-toolbelt python3-samba python3-talloc python3-tdb python3-yaml samba-common samba-common-bin samba-dsdb-modules samba-libs samba-vfs-modules tdb-tools

I do not modify any configuration files, nothing, but just after installation my VM just immediatly going crazy, as you can see with this mtr command. I have many lost of packets from another client (but same thing happen with mtr launched from hostprxmx).

Bash:
root@192.168.1.4:~# mtr guest-vm                                                                  Packets               Pings
 Host                                                                              Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.1.1                                                                     0.0% 11696    0.3   0.3   0.3 504.8   7.3
 2. 192.168.1.1                                                                     0.0% 11696    0.3   0.3   0.3 443.8   4.5
 3. guest-vm                                                                       30.1% 11695    0.3   0.3   0.2 739.3   8.2

If I remove the samba package (but not all dependencies) VM work again perfectly...
Bash:
root@guest-vm:~# apt remove krb5-user samba winbind libnss-winbind

dmesg revealed nothing when the pb occured.

I do not understand what's going on, I tried to find similar pb in the forum and tried many stuff but nothing was conclusive. Now, I do not even know where to investiguate more...I'm very confused about this pb and any clue, lead or light is welcome.
Thanks a lot !
 
Thx for taking time to answer. Good eyes and deduction, I copied and pasted a mtr from a different subnet but it's not the pb here.
As you can see from this 2 new mtr below, this time one from same subnet and one another subnet, launched at the same time.
I wasn't very carreful about that because we are using Cisco ACI where subnet and vlan are not used as usual (In fact I should have talked about EPG but I didn't want to complicate it cause I don't think this is useful for resolution, maybe I'm wrong that said, but anyway...)
From Cisco it appears this double jump is "normal" but don't know exactly why ^^
So....to be more precise here are 2 mtr from same EPG but 2 different hosts from different subnet IP.

Bash:
192.168.1.2 -> 192.168.1.3                               2021-09-17T12:03:41+0200
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                                       Packets               Pings
 Host                                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.1.3                                               10.8%  2768    0.2   0.2   0.1   4.0   0.1

Bash:
192.168.3.2                                                     Fri Sep 17 12:11:09 2021
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                                             Packets               Pings
 Host                                                            Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.3.1                                                   0.0%  3198    0.4   0.4   0.3 290.9   8.3
 2. 192.168.3.1                                                   0.0%  3198    0.6   0.3   0.3 339.1   7.2
 3. 192.168.1.3                                                  10.0%  3198    0.3   0.3   0.2   2.3   0.0

Thx again !
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!