zvol blocked

[andy77]

Hi @ all,

I expirienced a total freez of an machine a few days ago. Only a reboot helped and then the machine was running again. I do have arrount 10 of the same machines which are running since an year without any problem, and now wondering what has caused the total freeze of this one machine.

Unfortunately there are no logs in systemlog etc. at all about the freez.

On the screen I saw the following, which lets me assume that zfs froze totaly, and this is the reason why no logs where written at all.

http://prntscr.com/n5qafy

Here the running version:

proxmox-ve: 5.3-1 (running kernel: 4.15.18-11-pve)
pve-manager: 5.3-11 (running version: 5.3-11/d4907f84)
pve-kernel-4.15: 5.3-2
pve-kernel-4.15.18-11-pve: 4.15.18-34
pve-kernel-4.15.18-9-pve: 4.15.18-30
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-3
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-47
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-11
libpve-storage-perl: 5.0-38
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-2
proxmox-widget-toolkit: 1.0-22
pve-cluster: 5.0-33
pve-container: 2.0-34
pve-docs: 5.3-3
pve-edk2-firmware: 1.20181023-1
pve-firewall: 3.0-17
pve-firmware: 2.0-6
pve-ha-manager: 2.0-6
pve-i18n: 1.0-9
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 2.12.1-2
pve-xtermjs: 3.10.1-1
pve-zsync: 1.7-2
qemu-server: 5.0-46
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.12-pve1~bpo1

Any idea what could cause such freeze?

 

[andy77]

Today I had the second server (this time it was our proxmox node which is also our iscsi target) with the same error:
https://prnt.sc/n5sjcb

The only thing I did the past few days was to add a iscsi device to a VM on the "first frozen system" which is pointing to the storage on the "second frozen system".

 

[wolfgang]

Hi,

are the system under high load?

 

[andy77]

Hi, as you can see in the graphs both servers are not really unter high load:

First server
http://prntscr.com/n6utjx

Second server
http://prntscr.com/n6uuxt