First time poster, frequent lurker
I typically use a CIFS/SMB share mounted to my cluster nodes as the target storage for my VMs. When the upgrade the other week brought the newer 5.15 kernels I started having issues with the VMs throwing IO errors and locking up. At the time I pinned the kernel on the cluster nodes back to 5.13.19-6-pve and that allowed everything to work as it always has.
But as it has been several weeks and several kernel updates and the issue still persists I decided to see what I could do to better mitigate the issue so I could get onto the newer kernels.
This let me to unpin the older kernel and go through the journal on one of the nodes while I powered on a VM to watch for errors. The below is what I found:
As it turns out I think I was able to get around the issue by mounting the same storage location as an NFS 4.1 share and so far it seems to have resolved the issue on one of the nodes. I am fine moving it all to NFS going forward I just thought I would post what I found in hopes of gaining some understanding of what might be going on with CIFS/SMB or in case it helps someone else. Here is the version output from the node I mounted the NFS share on if it is useful as well.
I typically use a CIFS/SMB share mounted to my cluster nodes as the target storage for my VMs. When the upgrade the other week brought the newer 5.15 kernels I started having issues with the VMs throwing IO errors and locking up. At the time I pinned the kernel on the cluster nodes back to 5.13.19-6-pve and that allowed everything to work as it always has.
But as it has been several weeks and several kernel updates and the issue still persists I decided to see what I could do to better mitigate the issue so I could get onto the newer kernels.
This let me to unpin the older kernel and go through the journal on one of the nodes while I powered on a VM to watch for errors. The below is what I found:
Code:
kernel: CIFS: VFS: \\cifs/smb_IP Error -512 sending data on socket to server
kernel: CIFS: VFS: \\cifs/smb_IP Error -512 sending data on socket to server
kernel: CIFS: VFS: \\cifs/smb_IP Error -512 sending data on socket to serverAs it turns out I think I was able to get around the issue by mounting the same storage location as an NFS 4.1 share and so far it seems to have resolved the issue on one of the nodes. I am fine moving it all to NFS going forward I just thought I would post what I found in hopes of gaining some understanding of what might be going on with CIFS/SMB or in case it helps someone else. Here is the version output from the node I mounted the NFS share on if it is useful as well.
Code:
proxmox-ve: 7.2-1 (running kernel: 5.15.35-3-pve)
pve-manager: 7.2-5 (running version: 7.2-5/12f1e639)
pve-kernel-5.15: 7.2-5
pve-kernel-helper: 7.2-5
pve-kernel-5.13: 7.1-9
pve-kernel-5.11: 7.0-10
pve-kernel-5.4: 6.4-5
pve-kernel-5.15.35-3-pve: 5.15.35-6
pve-kernel-5.15.35-2-pve: 5.15.35-5
pve-kernel-5.15.35-1-pve: 5.15.35-3
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-4-pve: 5.13.19-9
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.4.128-1-pve: 5.4.128-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-2
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-2
libpve-storage-perl: 7.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.12-1
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.3-1
proxmox-backup-file-restore: 2.2.3-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-1
pve-container: 4.2-1
pve-docs: 7.2-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.4-2
pve-ha-manager: 3.3-4
pve-i18n: 2.7-2
pve-qemu-kvm: 6.2.0-10
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1