Hi,
as already postet in Fresh 8.0.3 install renames disk names when ZFS is used and crashes on reboot - weird I did a fresh install with the 7.4-1 iso and all works as expected.
The 7.4-1 installation did also rename all the disk added to the zfs pool and hid the SAS-Shelf...
Before creation of ZFS-Pool an reboot the Shelf is visible:
After reboot the disk /dev/sdb - /dev/sdz will become to /dev/sdab - /dev/sday
and the shelf is "gone":
This behavior is therefore identical to that of version 8.0, but here the ZFS pool is not correctly mounted at boot time. I suspect that the mount happens during the time when the disks are renamed / reimported.
v8 Status after first reboot:
Any ideas?
See also https://forum.proxmox.com/threads/f...s-is-used-and-crashes-on-reboot-weird.130951/
Thanks for your help!
ramon
as already postet in Fresh 8.0.3 install renames disk names when ZFS is used and crashes on reboot - weird I did a fresh install with the 7.4-1 iso and all works as expected.
The 7.4-1 installation did also rename all the disk added to the zfs pool and hid the SAS-Shelf...
Before creation of ZFS-Pool an reboot the Shelf is visible:
Code:
# ls /sys/class/enclosure/1\:0\:24\:0/
0 1 10 11 12 13 14 15 16 17 18 19 2 20 21 22 23 3 4 5 6 7 8 9 components device id power subsystem uevent
Code:
# lsblk -d -o VENDOR,MODEL,SERIAL,HCTL,SIZE,PHY-SEC,LOG-SEC,NAME | sed -e "`ls -1d /sys/class/enclosure/*/*/device/block/*|sed "s+.*enclosure/\(.*\)/device/block/\(.*\)+s-\2\\$-\2 \1-+"`"
VENDOR MODEL SERIAL HCTL SIZE PHY-SEC LOG-SEC NAME
LSI RAID 5/6 SAS 6G 00b98e2f23aff7c827a0ba0001570003 0:2:0:0 1.4T 4096 512 sda
LSI RAID 5/6 SAS 6G 0035534f2805f8c827a0ba0001570003 0:2:1:0 278.9G 4096 512 sdb
NETAPP X477_SMEGX04TA07 S1Z2ZM13 1:0:0:0 3.6T 512 512 sdc 1:0:24:0/0
NETAPP X477_SMEGX04TA07 S1Z2ZLQ0 1:0:1:0 3.6T 512 512 sdd 1:0:24:0/1
NETAPP X477_SMEGX04TA07 S1Z2ZFGP 1:0:2:0 3.6T 512 512 sde 1:0:24:0/2
NETAPP X477_SMEGX04TA07 S1Z2ZGJ4 1:0:3:0 3.6T 512 512 sdf 1:0:24:0/3
NETAPP X477_SMEGX04TA07 S1Z2ZFGY 1:0:4:0 3.6T 512 512 sdg 1:0:24:0/4
NETAPP X477_SMEGX04TA07 S1Z2ZLTS 1:0:5:0 3.6T 512 512 sdh 1:0:24:0/5
NETAPP X477_SMEGX04TA07 S1Z2ZFML 1:0:6:0 3.6T 512 512 sdi 1:0:24:0/6
NETAPP X477_SMEGX04TA07 S1Z2ZLYY 1:0:7:0 3.6T 512 512 sdj 1:0:24:0/7
NETAPP X477_SMEGX04TA07 S1Z2ZNHN 1:0:8:0 3.6T 512 512 sdk 1:0:24:0/8
NETAPP X477_SMEGX04TA07 S1Z2ZFME 1:0:9:0 3.6T 512 512 sdl 1:0:24:0/9
NETAPP X477_SMEGX04TA07 S1Z2ZF7V 1:0:10:0 3.6T 512 512 sdm 1:0:24:0/10
NETAPP X477_SMEGX04TA07 S1Z2ZLG2 1:0:11:0 3.6T 512 512 sdn 1:0:24:0/11
NETAPP X477_SMEGX04TA07 S1Z2YGVY 1:0:12:0 3.6T 512 512 sdo 1:0:24:0/12
NETAPP X477_SMEGX04TA07 S1Z2ZFRJ 1:0:13:0 3.6T 512 512 sdp 1:0:24:0/13
HL-DT-ST HL-DT-ST DVDRAM GT80N B3GRNC1041458 3:0:0:0 1024M 512 512 sr0
KVM vmDisk-CD 212052060601 10:0:0:0 1024M 512 512 sr1
NETAPP X477_SMEGX04TA07 S1Z2ZM0Z 1:0:14:0 3.6T 512 512 sdq 1:0:24:0/14
NETAPP X477_SMEGX04TA07 S1Z2ZFDH 1:0:15:0 3.6T 512 512 sdr 1:0:24:0/15
NETAPP X477_SMEGX04TA07 S1Z2YHCZ 1:0:16:0 3.6T 512 512 sds 1:0:24:0/16
NETAPP X477_SMEGX04TA07 S1Z2YH4S 1:0:17:0 3.6T 512 512 sdt 1:0:24:0/17
NETAPP X477_SMEGX04TA07 S1Z2ZF6B 1:0:18:0 3.6T 512 512 sdu 1:0:24:0/18
NETAPP X477_SMEGX04TA07 S1Z2ZH23 1:0:19:0 3.6T 512 512 sdv 1:0:24:0/19
NETAPP X477_SMEGX04TA07 S1Z2YFVA 1:0:20:0 3.6T 512 512 sdw 1:0:24:0/20
NETAPP X477_SMEGX04TA07 S1Z2YGVX 1:0:21:0 3.6T 512 512 sdx 1:0:24:0/21
NETAPP X477_SMEGX04TA07 S1Z2YGZ5 1:0:22:0 3.6T 512 512 sdy 1:0:24:0/22
NETAPP X477_SMEGX04TA07 S1Z2YHB5 1:0:23:0 3.6T 512 512 sdz 1:0:24:0/23
KVM vmDisk 212052060601 11:0:0:0 0B 512 512 sdaaAfter reboot the disk /dev/sdb - /dev/sdz will become to /dev/sdab - /dev/sday
Code:
lsblk -d -o VENDOR,MODEL,SERIAL,HCTL,SIZE,PHY-SEC,LOG-SEC,NAME | sed -e "`ls -1d /sys/class/enclosure/*/*/device/block/*|sed "s+.*enclosure/\(.*\)/device/block/\(.*\)+s-\2\\$-\2 \1-+"`" | grep -i netapp
ls: cannot access '/sys/class/enclosure/*/*/device/block/*': No such file or directory
NETAPP X477_SMEGX04TA07 S1Z2ZM13 1:0:26:0 3.6T 512 512 sdab
NETAPP X477_SMEGX04TA07 S1Z2ZLQ0 1:0:27:0 3.6T 512 512 sdac
NETAPP X477_SMEGX04TA07 S1Z2ZFGP 1:0:28:0 3.6T 512 512 sdad
NETAPP X477_SMEGX04TA07 S1Z2ZGJ4 1:0:29:0 3.6T 512 512 sdae
NETAPP X477_SMEGX04TA07 S1Z2ZFGY 1:0:30:0 3.6T 512 512 sdaf
NETAPP X477_SMEGX04TA07 S1Z2ZLTS 1:0:31:0 3.6T 512 512 sdag
NETAPP X477_SMEGX04TA07 S1Z2ZFML 1:0:32:0 3.6T 512 512 sdah
NETAPP X477_SMEGX04TA07 S1Z2ZLYY 1:0:33:0 3.6T 512 512 sdai
NETAPP X477_SMEGX04TA07 S1Z2ZNHN 1:0:34:0 3.6T 512 512 sdaj
NETAPP X477_SMEGX04TA07 S1Z2ZFME 1:0:35:0 3.6T 512 512 sdak
NETAPP X477_SMEGX04TA07 S1Z2ZF7V 1:0:36:0 3.6T 512 512 sdal
NETAPP X477_SMEGX04TA07 S1Z2ZLG2 1:0:37:0 3.6T 512 512 sdam
NETAPP X477_SMEGX04TA07 S1Z2YGVY 1:0:38:0 3.6T 512 512 sdan
NETAPP X477_SMEGX04TA07 S1Z2ZFRJ 1:0:39:0 3.6T 512 512 sdao
NETAPP X477_SMEGX04TA07 S1Z2ZM0Z 1:0:40:0 3.6T 512 512 sdap
NETAPP X477_SMEGX04TA07 S1Z2ZFDH 1:0:41:0 3.6T 512 512 sdaq
NETAPP X477_SMEGX04TA07 S1Z2YHCZ 1:0:42:0 3.6T 512 512 sdar
NETAPP X477_SMEGX04TA07 S1Z2YH4S 1:0:43:0 3.6T 512 512 sdas
NETAPP X477_SMEGX04TA07 S1Z2ZF6B 1:0:44:0 3.6T 512 512 sdat
NETAPP X477_SMEGX04TA07 S1Z2ZH23 1:0:45:0 3.6T 512 512 sdau
NETAPP X477_SMEGX04TA07 S1Z2YFVA 1:0:46:0 3.6T 512 512 sdav
NETAPP X477_SMEGX04TA07 S1Z2YGVX 1:0:47:0 3.6T 512 512 sdaw
NETAPP X477_SMEGX04TA07 S1Z2YGZ5 1:0:48:0 3.6T 512 512 sdax
NETAPP X477_SMEGX04TA07 S1Z2YHB5 1:0:49:0 3.6T 512 512 sdayand the shelf is "gone":
Code:
# ls /sys/class/enclosure/This behavior is therefore identical to that of version 8.0, but here the ZFS pool is not correctly mounted at boot time. I suspect that the mount happens during the time when the disks are renamed / reimported.
v8 Status after first reboot:
Code:
# zpool status
pool: pve-pool-01
state: SUSPENDED
status: One or more devices are faulted in response to IO failures.
action: Make sure the affected devices are connected, then run 'zpool clear'.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-HC
config:
NAME STATE READ WRITE CKSUM
pve-pool-01 ONLINE 0 0 0
raidz2-0 ONLINE 0 24 0
scsi-35000c50099acb2b3 ONLINE 3 8 0
scsi-35000c50099acd897 ONLINE 3 8 0
scsi-35000c50099b03c9f ONLINE 3 6 0
scsi-35000c50099afd167 ONLINE 3 6 0
scsi-35000c50099b06867 ONLINE 3 4 0
scsi-35000c50099acc6db ONLINE 3 4 0
scsi-35000c50099b03067 ONLINE 3 2 0
scsi-35000c50099acb9fb ONLINE 0 0 0
scsi-35000c50099ac1a63 ONLINE 0 0 0
scsi-35000c50099b030cb ONLINE 3 2 0
scsi-35000c50099b05013 ONLINE 3 2 0
scsi-35000c50099acf193 ONLINE 3 4 0
scsi-35000c50099ad1ed7 ONLINE 3 6 0
scsi-35000c50099b0060b ONLINE 3 6 0
scsi-35000c50099acb317 ONLINE 3 10 0
scsi-35000c50099b03fe3 ONLINE 3 10 0
scsi-35000c50099acf2bf ONLINE 3 12 0
scsi-35000c50099ad183f ONLINE 3 8 0
scsi-35000c50099b054ef ONLINE 3 6 0
scsi-35000c50099afa22b ONLINE 3 4 0
scsi-35000c50099ae007b ONLINE 3 2 0
scsi-35000c50099ad1ea7 ONLINE 3 4 0
scsi-35000c50099ad15cb ONLINE 3 6 0
scsi-35000c50099acf82f ONLINE 3 6 0
errors: List of errors unavailable: pool I/O is currently suspendedAny ideas?
See also https://forum.proxmox.com/threads/f...s-is-used-and-crashes-on-reboot-weird.130951/
Thanks for your help!
ramon