Migrating a real machine with NVME disk to PVE VM

G

guerby

Active Member
Proxmox Subscriber
Nov 22, 2020
88
18
28
51
Hi,

Today I had to migrate a physical machine running Fedora Core 32 with NVME root disk and UEFI boot to a virtual machine (OVMF) under PVE 6.3.

Unsurprisingly Fedora boot failed if I just copied (offline using an adapter) the whole /dev/nvme0n1 original block device to a ZFS volume and used that with volume as a default virtio-scsi disk.

After some searching about KVM and NVME I added to the VMID.conf the following line:

Code: 
args: -drive file=/dev/mypool/vm-10000-disk-0,format=raw,if=none,id=NVME1 -device nvme,drive=NVME1,serial=nvme-1

And edited VMID.conf a bit more :
- removed the scsi0: line entirely
- removed the scsihw: virtio-scsi-pci line entirely
- removed scsi0 from boot: order=...

After going into the OVMF BIOS to select the right boot device the machine booted successfully FC32.

Now I want to add a regular virtio-scsi disk to this VM and since PVE doesn't know about "vm-10000-disk-0" (only mentionned in args) it looks like I could be in trouble with same name disk volumes.

I'm also wondering about future backups with this "hidden from PVE" disk.

What should I do to make PVE aware of this NVME device and associated volume and not risk issue during lifetime of this VM?

Thanks for your help!

PS: may be a "nvme" flag will be added at some point to PVE just like "SSD" currently is.
 
Last edited:
Hi,
guerby said:
Hi,

Today I had to migrate a physical machine running Fedora Core 32 with NVME root disk and UEFI boot to a virtual machine (OVMF) under PVE 6.3.

Unsurprisingly Fedora boot failed if I just copied (offline using an adapter) the whole /dev/nvme0n1 original block device to a ZFS volume and used that with volume as a default virtio-scsi disk.

After some searching about KVM and NVME I added to the VMID.conf the following line:

Code: 
args: -drive file=/dev/mypool/vm-10000-disk-0,format=raw,if=none,id=NVME1 -device nvme,drive=NVME1,serial=nvme-1

And edited VMID.conf a bit more :
- removed the scsi0: line entirely
- removed the scsihw: virtio-scsi-pci line entirely
- removed scsi0 from boot: order=...

After going into the OVMF BIOS to select the right boot device the machine booted successfully FC32.
Click to expand...

guerby said:
Now I want to add a regular virtio-scsi disk to this VM and since PVE doesn't know about "vm-10000-disk-0" (only mentionned in args) it looks like I could be in trouble with same name disk volumes.
Click to expand...
PVE should see the volume, and avoid a direct naming conflict. You can check with pvesm list <storage ID for ZFS storage>

guerby said:
I'm also wondering about future backups with this "hidden from PVE" disk.

What should I do to make PVE aware of this NVME device and associated volume and not risk issue during lifetime of this VM?
Click to expand...
But yes, for backups/replication/etc. it won't see the disk, so it's far from ideal.

guerby said:
Thanks for your help!

PS: may be a "nvme" flag will be added at some point to PVE just like "SSD" currently is.
Click to expand...
You might want to subscribe to this feature request on our bug tracker.

Ping @oguz
 
  • Like
Reactions: guerby
hi,
guerby said:
PS: may be a "nvme" flag will be added at some point to PVE just like "SSD" currently is.
Click to expand...
this is planned, but currently waiting on an upstream bug [0] to get fixed.
there are also other small things to do since qemu made nvme-related changes in the recent releases.

otherwise if you want, you can try out the patch on the list [1] for testing and let me know of any issue you find.

[0]: https://bugs.launchpad.net/qemu/+bug/1925496
[1]: https://lists.proxmox.com/pipermail/pve-devel/2020-May/043611.html
 
You must log in or register to reply here.
Top Bottom