zfs rpool performance issue (deadman)

jhyland

New Member
Oct 1, 2024
1
0
1
I'm running proxmox on bare metal. I currently have one lxc container (casaos) which runs about 15 docker containers. The rootfs for proxmox is on 2 2tb SSDs in a ZFS mirror. Its name is rpool. I have another ZFS pool named tank that is about 15tb of 6 disks. Everything is working functionally. But the performance is very poor. I think I've narrowed it down to the rpool. I see a lot of deadman events when the system becomes less responsive. This is my first lab build, so I've tried some things such as dedup that I wish I hadn't. I'm inclined to reinstall proxmox on a single 2T SSD using ext4. I'm hoping this will resolve the problem. I don't want to lose my my proxmox config or my one lxc. I have a vzdump of that on an external HDD. I also don't want to lose my larger tank pool. Do you recommend to continue to try to troubleshoot this or should I resinstall and what would the steps for that be? Thanks!

Code:
Oct 03 17:46:40 pve zed[3365279]: eid=37239 class=deadman pool='rpool' vdev=ata-JAJS600M2TB_AA202200000031005220-part3 size=20480 offset=156177002496 priority=3 err=0 flags=0x180080 bookmark=0:20681:0:6193
Oct 03 17:46:40 pve pve-ha-lrm[3010]: loop take too long (33 seconds)
Oct 03 17:46:40 pve pve-firewall[2964]: firewall update time (29.296 seconds)
Oct 03 17:46:40 pve pvestatd[2980]: status update time (29.479 seconds)
Oct 03 17:46:40 pve pmxcfs[2819]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-node/pve: -1
Oct 03 17:46:40 pve pmxcfs[2819]: [status] notice: RRD update error /var/lib/rrdcached/db/pve2-node/pve: /var/lib/rrdcached/db/pve2-node/pve: illegal attempt to update using time 1727991991 when last update time is 1727991991 (minimum one second step)
Oct 03 17:46:40 pve pmxcfs[2819]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-vm/102: -1
Oct 03 17:46:40 pve pmxcfs[2819]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/pve/nas-zfs: -1
Oct 03 17:46:40 pve pmxcfs[2819]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/pve/local-zfs: -1
Oct 03 17:46:40 pve pmxcfs[2819]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/pve/local: -1
Oct 03 17:46:40 pve pmxcfs[2819]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/pve/USB16tb: -1
Oct 03 17:47:25 pve zed[2544]: Missed 95 events
Oct 03 17:47:25 pve zed[3365791]: eid=37241 class=deadman pool='rpool' vdev=ata-JAJS600M2TB_AA202200000031005220-part3 size=16384 offset=1065359745024 priority=3 err=0 flags=0x180080 bookmark=0:20681:1:24
Oct 03 17:47:25 pve zed[3365789]: eid=37240 class=deadman pool='rpool' vdev=ata-JAJS600M2TB_AA202200000031005220-part3 size=16384 offset=155069849600 priority=3 err=0 flags=0x180080 bookmark=0:20681:1:24
Oct 03 17:47:25 pve zed[3365793]: eid=37242 class=deadman pool='rpool' vdev=ata-JAJS600M2TB_AA202200000031005220-part3 size=16384 offset=639922794496 priority=3 err=0 flags=0x180080 bookmark=0:20681:1:24
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!