LVM and virtio-scsi - detects volume as complete drive

[Firm]

Hi,

I have successfully created a few VMs on PVE 4.0. Hardware raid (/dev/sda - Dell H730 RAID10) is partitioned like this:

NAME                     MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                        8:0    0   2.7T  0 disk
├─sda1                     8:1    0     1M  0 part
└─sda2                     8:2    0   2.7T  0 part
  ├─vg0-root             252:0    0  14.9G  0 lvm  /
  ├─vg0-vm--100--disk--1 252:1    0   100G  0 lvm
  ├─vg0-vm--101--disk--1 252:2    0   512G  0 lvm
  ├─vg0-vm--102--disk--1 252:3    0    16G  0 lvm
  ├─vg0-vm--102--disk--2 252:4    0    64G  0 lvm
  ├─vg0-vm--103--disk--1 252:5    0   512G  0 lvm
  ├─vg0-vm--104--disk--1 252:6    0    16G  0 lvm
  ├─vg0-vm--104--disk--2 252:7    0    64G  0 lvm
  └─vg0-vm--105--disk--1 252:8    0   100G  0 lvm
sdb                        8:16   0 446.6G  0 disk
sdc                        8:32   0 446.6G  0 disk
sdd                        8:48   0 446.6G  0 disk
sde                        8:64   0 446.6G  0 disk

Note: /dev/sdb, /dev/sdc, /dev/sdd, /dev/sde are RAID0 SSD-based single disks also configured through H730 controller.

VM 102 was created using virtio-blk driver all disks:

virtio0: lvm:vm-102-disk-1,size=16G
virtio1: lvm:vm-102-disk-2,size=64G
virtio2: /dev/sdb,size=457344M
virtio3: /dev/sdc,size=457344M

virtio2 and virtio3 were added manually as PVE UI doesn't allow attaching physical devices. VM (Debian 8.2) works fine although disks are shown as /dev/vdX (due to use of virtio-blk driver).

For VM 104 I used virtio-scsi driver for all disks:

scsihw: virtio-scsi-pci
scsi0: lvm:vm-104-disk-1,size=16G
scsi1: lvm:vm-104-disk-2,size=64G
scsi2: /dev/sdd,size=457344M
scsi3: /dev/sde,size=457344M

Note: I also tried to remove lvm: prefix and size= prefix - nothing changed.

So the problem 1 is: installer detects both /dev/sda and /dev/sdb as 3TB disks (as /dev/sda on pve host). How can I fix that?
The problem 2 is: I'm not sure if /dev/sdc inside vm104 really maps to /dev/sdd (inspired by problem 1). How can I check if it happened or not? hdparm/smartctl can't access inquiry data on disks.

Any ideas are welcome.

Regards,

 

[sigxcpu]

partition /dev/sdb on 1 and check the partition table on 2 to see if they are the same disk.

 

[stefws]

try to map devices through sata:

virtio0: lvm:vm-102-disk-1,size=16G
virtio1: lvm:vm-102-disk-2,size=64G
sata0: /dev/sdb,size=457344M
sata1: /dev/sdc,size=457344M

or all through sata:

sata0: lvm:vm-102-disk-1,size=16G
sata1: lvm:vm-102-disk-2,size=64G
sata2: /dev/sdb,size=457344M
sata3: /dev/sdc,size=457344M

 

[Firm]

partition /dev/sdb on 1 and check the partition table on 2 to see if they are the same disk.

Checked with hexdump on /dev/vdX devices - they happened to be different. So the problem 2 is not confirmed.

Regards,

 

[Firm]

VM with all sataX drives seems to boot normally and devices look ok (there an error is shown while booting but no traces in syslog).

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0    16G  0 disk
└─sda1   8:1    0    16G  0 part /
sdb      8:16   0    64G  0 disk
sdc      8:32   0 446.6G  0 disk
sdd      8:48   0 446.6G  0 disk

# ls -l /dev/disk/by-id/
total 0
lrwxrwxrwx 1 root root  9 Oct 28 11:45 ata-QEMU_DVD-ROM_QM00003 -> ../../sr0
lrwxrwxrwx 1 root root  9 Oct 28 11:45 ata-QEMU_HARDDISK_QM00005 -> ../../sda
lrwxrwxrwx 1 root root 10 Oct 28 11:45 ata-QEMU_HARDDISK_QM00005-part1 -> ../../sda1
lrwxrwxrwx 1 root root  9 Oct 28 11:45 ata-QEMU_HARDDISK_QM00007 -> ../../sdb
lrwxrwxrwx 1 root root  9 Oct 28 11:45 ata-QEMU_HARDDISK_QM00009 -> ../../sdc
lrwxrwxrwx 1 root root  9 Oct 28 11:45 ata-QEMU_HARDDISK_QM00011 -> ../../sdd

But with scsiX (SCSI Controller Type is set to VIRTIO) it recognizes disks with wrong order (but sizes are ok):

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0    16G  0 disk
└─sda1   8:1    0    16G  0 part /
sdb      8:16   0 446.6G  0 disk
sdc      8:32   0    64G  0 disk
sdd      8:48   0 446.6G  0 disk

# ls -l /dev/disk/by-id/
total 0
lrwxrwxrwx 1 root root  9 Oct 28 11:49 ata-QEMU_DVD-ROM_QM00003 -> ../../sr0
lrwxrwxrwx 1 root root  9 Oct 28 11:49 scsi-0QEMU_QEMU_HARDDISK_drive-scsi0 -> ../../sda
lrwxrwxrwx 1 root root 10 Oct 28 11:49 scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-part1 -> ../../sda1
lrwxrwxrwx 1 root root  9 Oct 28 11:49 scsi-0QEMU_QEMU_HARDDISK_drive-scsi1 -> ../../sdc
lrwxrwxrwx 1 root root  9 Oct 28 11:49 scsi-3644a84200d7362001dbcd101b29de295 -> ../../sdb
lrwxrwxrwx 1 root root  9 Oct 28 11:49 scsi-3644a84200d7362001dbcd223c3e0fe0f -> ../../sdd
lrwxrwxrwx 1 root root  9 Oct 28 11:49 wwn-0x644a84200d7362001dbcd101b29de295 -> ../../sdb
lrwxrwxrwx 1 root root  9 Oct 28 11:49 wwn-0x644a84200d7362001dbcd223c3e0fe0f -> ../../sdd

Unfortunately I can't reproduce Problem 1 anymore (which is mentioned in original post).

P.S. The naming order of devices is important since they are used as raw devices in service's configuration.

 

[spirit]

Hi, I have found the bug with qemu scsi disk + lvm on host

https://forum.proxmox.com/threads/problem-with-iscsi-virtio-ovmf.26192/#post-131418

I'll try to provide a patch this week