ZFS Trouble, zed utilize 100% cpu

syn

Renowned Member
May 28, 2013
12
0
66
Hi! May be somebody know how fix this, after upgrade zed process eat 100% from one cpu core.

Now i on Linux srv1 4.4.83-1-pve #1 SMP PVE 4.4.83-96 (Tue, 19 Sep 2017 10:30:12 +0200) x86_64 GNU/Linux

I think this is because versions of zfs modules and zfs utils mismatch:

root@srv1:/etc/apt# dpkg -l |grep zfs
ii libzfs2linux 0.7.3-pve1~bpo9
ii zfs-initramfs 0.7.3-pve1~bpo9
ii zfs-zed 0.7.3-pve1~bpo9
ii zfsutils-linux 0.7.3-pve1~bpo9

root@srv1:/etc/apt# dmesg|grep ZFS
[ 25.476228] ZFS: Loaded module v0.6.5.9-1, ZFS pool version 5000, ZFS filesystem version 5

How can i upgrade modules version ? or downgrade utils ? For this upgrade i use proxmox no-subscription repo only.
And sure i rebooted my system after upgrade.
 
you are running a kernel from PVE 4 and ZFS user space from PVE 5. reboot into a PVE 5 kernel..
 
All,
I do have a similar problem:

ii libzfs2 0.6.5.9-pve15~bpo80 all transitional package
ii libzfs2linux 0.7.3-pve1~bpo9 amd64 OpenZFS filesystem library for Linux
ii zfs-initramfs 0.7.3-pve1~bpo9 all OpenZFS root filesystem capabilities for Linux - initramfs
ii zfs-zed 0.7.3-pve1~bpo9 amd64 OpenZFS Event Daemon
ii zfsutils 0.6.5.9-pve15~bpo80 all transitional package
ii zfsutils-linux 0.7.3-pve1~bpo9 amd64 command-line tools to manage OpenZFS filesystems

proxmox-ve: 5.1-32 (running kernel: 4.13.13-1-pve)
pve-manager: 5.1-41 (running version: 5.1-41/0b958203)
pve-kernel-4.4.40-1-pve: 4.4.40-82
pve-kernel-4.4.35-2-pve: 4.4.35-79
pve-kernel-4.4.13-2-pve: 4.4.13-58
pve-kernel-4.4.24-1-pve: 4.4.24-72
pve-kernel-4.13.4-1-pve: 4.13.4-26
pve-kernel-4.4.62-1-pve: 4.4.62-88
pve-kernel-4.13.8-2-pve: 4.13.8-28
pve-kernel-4.13.13-2-pve: 4.13.13-32
pve-kernel-4.10.17-4-pve: 4.10.17-24
pve-kernel-4.4.19-1-pve: 4.4.19-66
pve-kernel-4.10.17-2-pve: 4.10.17-20
pve-kernel-4.4.6-1-pve: 4.4.6-48
pve-kernel-4.4.49-1-pve: 4.4.49-86
pve-kernel-4.10.15-1-pve: 4.10.15-15
pve-kernel-4.4.35-1-pve: 4.4.35-77
pve-kernel-4.4.21-1-pve: 4.4.21-71
pve-kernel-4.4.44-1-pve: 4.4.44-84
pve-kernel-4.4.16-1-pve: 4.4.16-64
pve-kernel-4.4.67-1-pve: 4.4.67-90
pve-kernel-4.13.13-1-pve: 4.13.13-31
pve-kernel-4.10.17-3-pve: 4.10.17-23
pve-kernel-4.4.59-1-pve: 4.4.59-87
libpve-http-server-perl: 2.0-8
lvm2: 2.02.168-pve6
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-19
qemu-server: 5.0-18
pve-firmware: 2.0-3
libpve-common-perl: 5.0-25
libpve-guest-common-perl: 2.0-14
libpve-access-control: 5.0-7
libpve-storage-perl: 5.0-17
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-3
pve-docs: 5.1-12
pve-qemu-kvm: 2.9.1-5
pve-container: 2.0-18
pve-firewall: 3.0-5
pve-ha-manager: 2.0-4
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.1-2
lxcfs: 2.0.8-1
criu: 2.11.1-1~bpo90
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.7.3-pve1~bpo9

I do have six nodes here, only one is acting up...

Any ideas?
Thanks
Tobias
 
check the running kernel on the affected node, and reboot it into the correct one.
 
  • Like
Reactions: iprigger
check the running kernel on the affected node, and reboot it into the correct one.

Yes, that was the last facepalm in 2017 for me.... it's been the wrong kernel and I checked and didn't see it. In addition, the update to proxmox 5.2 wasn't somehow complete. after an apt-get install proxmox-ve again it works now flawless.

tobias
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!