Migrating a real machine with NVME disk to PVE VM

[guerby]

Hi,

Today I had to migrate a physical machine running Fedora Core 32 with NVME root disk and UEFI boot to a virtual machine (OVMF) under PVE 6.3.

Unsurprisingly Fedora boot failed if I just copied (offline using an adapter) the whole /dev/nvme0n1 original block device to a ZFS volume and used that with volume as a default virtio-scsi disk.

After some searching about KVM and NVME I added to the VMID.conf the following line:

args: -drive file=/dev/mypool/vm-10000-disk-0,format=raw,if=none,id=NVME1 -device nvme,drive=NVME1,serial=nvme-1

And edited VMID.conf a bit more :
- removed the scsi0: line entirely
- removed the scsihw: virtio-scsi-pci line entirely
- removed scsi0 from boot: order=...

After going into the OVMF BIOS to select the right boot device the machine booted successfully FC32.

Now I want to add a regular virtio-scsi disk to this VM and since PVE doesn't know about "vm-10000-disk-0" (only mentionned in args) it looks like I could be in trouble with same name disk volumes.

I'm also wondering about future backups with this "hidden from PVE" disk.

What should I do to make PVE aware of this NVME device and associated volume and not risk issue during lifetime of this VM?

Thanks for your help!

PS: may be a "nvme" flag will be added at some point to PVE just like "SSD" currently is.

 

[fiona]

Hi,

Hi,

Today I had to migrate a physical machine running Fedora Core 32 with NVME root disk and UEFI boot to a virtual machine (OVMF) under PVE 6.3.

Unsurprisingly Fedora boot failed if I just copied (offline using an adapter) the whole /dev/nvme0n1 original block device to a ZFS volume and used that with volume as a default virtio-scsi disk.

After some searching about KVM and NVME I added to the VMID.conf the following line:

args: -drive file=/dev/mypool/vm-10000-disk-0,format=raw,if=none,id=NVME1 -device nvme,drive=NVME1,serial=nvme-1

And edited VMID.conf a bit more :
- removed the scsi0: line entirely
- removed the scsihw: virtio-scsi-pci line entirely
- removed scsi0 from boot: order=...

After going into the OVMF BIOS to select the right boot device the machine booted successfully FC32.

Now I want to add a regular virtio-scsi disk to this VM and since PVE doesn't know about "vm-10000-disk-0" (only mentionned in args) it looks like I could be in trouble with same name disk volumes.

PVE should see the volume, and avoid a direct naming conflict. You can check with pvesm list <storage ID for ZFS storage>

I'm also wondering about future backups with this "hidden from PVE" disk.

What should I do to make PVE aware of this NVME device and associated volume and not risk issue during lifetime of this VM?

But yes, for backups/replication/etc. it won't see the disk, so it's far from ideal.

Thanks for your help!

PS: may be a "nvme" flag will be added at some point to PVE just like "SSD" currently is.

You might want to subscribe to this feature request on our bug tracker.

Ping @oguz

 

[guerby]

Hi,



PVE should see the volume, and avoid a direct naming conflict. You can check with pvesm list <storage ID for ZFS storage>

Yes I confirm PVE sees both disks the one from args and the small one for OVMF.

But yes, for backups/replication/etc. it won't see the disk, so it's far from ideal.


You might want to subscribe to this feature request on our bug tracker.

Ping @oguz

Thanks!

 

[oguz]

hi,

PS: may be a "nvme" flag will be added at some point to PVE just like "SSD" currently is.

this is planned, but currently waiting on an upstream bug [0] to get fixed.
there are also other small things to do since qemu made nvme-related changes in the recent releases.

otherwise if you want, you can try out the patch on the list [1] for testing and let me know of any issue you find.

[0]: https://bugs.launchpad.net/qemu/+bug/1925496
[1]: https://lists.proxmox.com/pipermail/pve-devel/2020-May/043611.html