zfs disk entry below /dev/zvol/rpool/data was not

udo

Distinguished Member
Apr 22, 2009
5,975
196
163
Ahrensburg; Germany
Hi,
I have an pve 5.1 system, wich are updated last week, where I create an new VM, but the VM don't start, because the links below /dev/zvol/rpool/data/ wasn't created.
Code:
qm start 206
kvm: -drive file=/dev/zvol/rpool/data/vm-206-disk-1,if=none,id=drive-scsi0,discard=on,format=raw,cache=none,aio=native,detect-zeroes=unmap: Could not open '/dev/zvol/rpool/data/vm-206-disk-1': No such file or directory
I've tried this with different pools, but the same happens on the other zfs-pools.
The disk was created:
Code:
# syslog
Mar 21 14:43:57 pve01 pvedaemon[21527]: <udo@pve> update VM 206: -scsi0 local-zfs:50,discard=on
zfs list | grep 206
rpool/data/vm-206-disk-1      56K   394G    56K  -
Code:
pveversion -v
proxmox-ve: 5.1-41 (running kernel: 4.13.13-6-pve)
pve-manager: 5.1-46 (running version: 5.1-46/ae8241d4)
pve-kernel-4.13.13-6-pve: 4.13.13-41
pve-kernel-4.13.13-5-pve: 4.13.13-38
corosync: 2.4.2-pve3
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-common-perl: 5.0-28
libpve-guest-common-perl: 2.0-14
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-17
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 2.1.1-3
lxcfs: 2.0.8-2
novnc-pve: 0.6-4
openvswitch-switch: 2.7.0-2
proxmox-widget-toolkit: 1.0-11
pve-cluster: 5.0-20
pve-container: 2.0-19
pve-docs: 5.1-16
pve-firewall: 3.0-5
pve-firmware: 2.0-3
pve-ha-manager: 2.0-5
pve-i18n: 1.0-4
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.9.1-9
pve-xtermjs: 1.0-2
pve-zsync: 1.6-15
qemu-server: 5.0-22
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.6-pve1~bpo9
If I create the disk on lvm-storage I can start the VM.
After that, i can also migrate the disk to zfs-storage and both links will be created (from the unused an the new one):
Code:
 ls -lsa /dev/zvol/rpool/data/
total 0
0 drwxr-xr-x 2 root root 120 Mar 21 15:00 .
0 drwxr-xr-x 3 root root  80 Mar 16 17:27 ..
0 lrwxrwxrwx 1 root root  14 Mar 21 15:00 vm-206-disk-1 -> ../../../zd224
0 lrwxrwxrwx 1 root root  14 Mar 21 15:00 vm-206-disk-2 -> ../../../zd240
Code:
cat /etc/pve/qemu-server/206.conf
balloon: 0
bootdisk: scsi0
cores: 2
ide2: local:iso/SW_DVD9_Windows_Svr.ISO,media=cdrom
memory: 8192
name: vtest
net0: virtio=BE:79:CE:72:82:AF,bridge=vmbr0,tag=5
numa: 0
ostype: win10
scsi0: local-zfs:vm-206-disk-2,size=10G
scsihw: virtio-scsi-pci
smbios1: uuid=194ab9c7-8487-40c2-bc4f-578087c24587
sockets: 2
unused0: local-zfs:vm-206-disk-1
unused1: sasraid:vm-206-disk-1
As zfs-related stuff I have installed additional things
Code:
ii  znapzend                             0.17.0-1                       amd64        High performance open source ZFS backup with mbuffer and ssh support
ii  pve-zsync                            1.6-15                         all          Proxmox VE storage management library
Udo
 
Hi Udo,

this is the same issue like you reported with the VM restore.
I'm working on it.
The problem is Udev is to slow.
 
There are some cache problems what I do not understand yet, but I'm on the way ;-)
Is there only one pool on this Host?
Can you send me the output of

zpool status

Did you run a restore on this host before this happens?
 
There are some cache problems what I do not understand yet, but I'm on the way ;-)
Is there only one pool on this Host?
Hi Wolfgang,
no there are three pools in this case.
Can you send me the output of

zpool status
of course
Code:
# zpool status
  pool: pve01pool
 state: ONLINE
  scan: scrub repaired 0B in 0h8m with 0 errors on Sun Mar 11 00:32:47 2018
config:

        NAME                              STATE     READ WRITE CKSUM
        pve01pool                         ONLINE       0     0     0
          mirror-0                        ONLINE       0     0     0
            wwn-0x55cd2e414ec65535-part2  ONLINE       0     0     0
            wwn-0x55cd2e414ec655b4-part2  ONLINE       0     0     0

errors: No known data errors

  pool: pve02pool
 state: ONLINE
  scan: scrub repaired 0B in 0h49m with 0 errors on Sun Mar 11 01:13:36 2018
config:

        NAME                                                     STATE     READ WRITE CKSUM
        pve02pool                                                ONLINE       0     0     0
          mirror-0                                               ONLINE       0     0     0
            wwn-0x5000c500acb8e9ae-part1                         ONLINE       0     0     0
            wwn-0x5000c500acb8f6f5-part1                         ONLINE       0     0     0
          mirror-1                                               ONLINE       0     0     0
            wwn-0x5000c500acb96a62-part1                         ONLINE       0     0     0
            wwn-0x5000c500acb8f39b-part1                         ONLINE       0     0     0
        logs
          mirror-2                                               ONLINE       0     0     0
            scsi-SATA_INTEL_SSDSC2KG01_BTYM74230F3V1P9DGN-part1  ONLINE       0     0     0
            scsi-SATA_INTEL_SSDSC2KG01_BTYM74230F761P9DGN-part1  ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.
  scan: scrub repaired 0B in 0h1m with 0 errors on Sun Mar 11 00:25:22 2018
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sda2    ONLINE       0     0     0
            sdb2    ONLINE       0     0     0

errors: No known data errors
Did you run a restore on this host before this happens?
no, I've updated this host on friday last week with reboot and tried to create an new VM, which fails on two different pools (pve01pool + rpool/data - both ssd-mirrors).

Udo
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!