Hey all, when installing PVE on my Minisforum MS-01, I selected to use a RAIDZ1 mirror for my two NVMes as the root. While passing through the GPU to a Docker VM works perfectly fine, I of course lose display functionality on PVE itself. This should be perfectly fine most of the time, but if something ever goes badly, I'm concerned this will make troubleshooting nigh impossible if I can't SSH into it for whatever reason.
Because of this, I'm trying to enable SR-IOV / GVT, so I can passthrough a virtual GPU to the Docker VM for hardware transcoding in Jellyfin / Emby / Plex, while retaining display on the host, which I can see through my JetKVM.
I've followed this guide as a starting point: https://www.derekseaman.com/2024/07...u-vt-d-passthrough-with-intel-alder-lake.html
One of the key phaases is running:
This is supposed to cause booting into shim on reboot, which should let me enroll the MOK for DKMS. I can verify that the key is a new key to be enrolled by running:
I believe this is because me using ZFS as the root is causing systemd-boot to be used instead of Grub.
I found this page: https://pve.proxmox.com/wiki/Host_Bootloader#sysboot_secure_boot
I have attempted to follow the steps therein.
This shows me an EFI System Partition on each NVMe as I would expect based on the page above So I ran:
As the page says to run this command once for each EFI System Partition.
After doing this:
This would seem to indicate that the proxmox boot entry is first, so I would expect it to boot into shimx64.efi upon reboot. It doesn't. It still boots into systemd-boot, but there's a new entry called EFI Default Loader or something similar, and if I can manage to hit that in a couple of seconds, it does open shim, which let me enroll the key.
Upon reboot, I do see virtual GPUs now when I run:
which is progress. However, they don't seem to be able to select from the Mapped device dropdown in the Proxmox GUI, only the Raw device dropdown.
Moreover, if I check the efibootmgr order again, I get:
So the systemd-boot is back to being the default order instead of the proxmox entry. Is this the expected behavior, or have I messed up something?
Thanks in advance for any help.
Because of this, I'm trying to enable SR-IOV / GVT, so I can passthrough a virtual GPU to the Docker VM for hardware transcoding in Jellyfin / Emby / Plex, while retaining display on the host, which I can see through my JetKVM.
I've followed this guide as a starting point: https://www.derekseaman.com/2024/07...u-vt-d-passthrough-with-intel-alder-lake.html
One of the key phaases is running:
Bash:
mokutil --import /var/lib/dkms/mok.pub
Bash:
mokutil --list-newI believe this is because me using ZFS as the root is causing systemd-boot to be used instead of Grub.
I found this page: https://pve.proxmox.com/wiki/Host_Bootloader#sysboot_secure_boot
I have attempted to follow the steps therein.
Bash:
root@pve-01:~# lsblk -o +FSTYPE
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS FSTYPE
sr0 11:0 1 1024M 0 rom
zd0 230:0 0 100G 0 disk
├─zd0p1 230:1 0 512M 0 part vfat
├─zd0p2 230:2 0 98.5G 0 part ext4
└─zd0p3 230:3 0 976M 0 part swap
zd16 230:16 0 1M 0 disk
nvme0n1 259:0 0 1.8T 0 disk
├─nvme0n1p1 259:1 0 1007K 0 part
├─nvme0n1p2 259:2 0 1G 0 part vfat
└─nvme0n1p3 259:4 0 1.8T 0 part zfs_member
nvme1n1 259:3 0 1.8T 0 disk
├─nvme1n1p1 259:5 0 1007K 0 part
├─nvme1n1p2 259:6 0 1G 0 part vfat
└─nvme1n1p3 259:7 0 1.8T 0 part zfs_member
Bash:
proxmox-boot-tool init /dev/nvme1n1p2 grub
proxmox-boot-tool init /dev/nvme0n1p2 grubAfter doing this:
Bash:
root@pve-01:~# efibootmgr -v
BootCurrent: 0007
Timeout: 3 seconds
BootOrder: 0000,0007,0006,0013,0014,0001,0002,0004,0005
Boot0000* proxmox HD(2,GPT,507e41d3-4307-4965-a70d-308544d7cb0f,0x800,0x200000)/File(\EFI\proxmox\shimx64.efi)
Boot0001* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-V PciRoot(0x0)/Pci(0x1c,0x0)/Pci(0x0,0x0)/MAC(5847ca770a20,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0002* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-LM PciRoot(0x0)/Pci(0x1d,0x0)/Pci(0x0,0x0)/MAC(5847ca770a21,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0004* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-V PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)/Pci(0x3,0x0)/Pci(0x0,0x0)/MAC(0276c6014a21,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0005* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-V PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)/Pci(0x7,0x0)/Pci(0x0,0x0)/MAC(0276c6014e09,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0006* Linux Boot Manager HD(2,GPT,4e50f1ec-fabf-4c32-9d20-56c36a62a011,0x800,0x200000)/File(\EFI\systemd\systemd-bootx64.efi)
Boot0007* Linux Boot Manager HD(2,GPT,507e41d3-4307-4965-a70d-308544d7cb0f,0x800,0x200000)/File(\EFI\systemd\systemd-bootx64.efi)
Boot0013* UEFI OS HD(2,GPT,507e41d3-4307-4965-a70d-308544d7cb0f,0x800,0x200000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
Boot0014* UEFI OS HD(2,GPT,4e50f1ec-fabf-4c32-9d20-56c36a62a011,0x800,0x200000)/File(\EFI\BOOT\BOOTX64.EFI)..BOThis would seem to indicate that the proxmox boot entry is first, so I would expect it to boot into shimx64.efi upon reboot. It doesn't. It still boots into systemd-boot, but there's a new entry called EFI Default Loader or something similar, and if I can manage to hit that in a couple of seconds, it does open shim, which let me enroll the key.
Upon reboot, I do see virtual GPUs now when I run:
Bash:
lspci | grep VGAMoreover, if I check the efibootmgr order again, I get:
Bash:
root@pve-01:~# efibootmgr -v
BootCurrent: 0007
Timeout: 3 seconds
BootOrder: 0007,0006,0013,0014,0000,0001,0002,0004,0005
Boot0000* proxmox HD(2,GPT,507e41d3-4307-4965-a70d-308544d7cb0f,0x800,0x200000)/File(\EFI\proxmox\shimx64.efi)
Boot0001* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-V PciRoot(0x0)/Pci(0x1c,0x0)/Pci(0x0,0x0)/MAC(5847ca770a20,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0002* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-LM PciRoot(0x0)/Pci(0x1d,0x0)/Pci(0x0,0x0)/MAC(5847ca770a21,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0004* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-V PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)/Pci(0x3,0x0)/Pci(0x0,0x0)/MAC(0276c6014a21,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0005* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-V PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)/Pci(0x7,0x0)/Pci(0x0,0x0)/MAC(0276c6014e09,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0006* Linux Boot Manager HD(2,GPT,4e50f1ec-fabf-4c32-9d20-56c36a62a011,0x800,0x200000)/File(\EFI\systemd\systemd-bootx64.efi)
Boot0007* Linux Boot Manager HD(2,GPT,507e41d3-4307-4965-a70d-308544d7cb0f,0x800,0x200000)/File(\EFI\systemd\systemd-bootx64.efi)
Boot0013* UEFI OS HD(2,GPT,507e41d3-4307-4965-a70d-308544d7cb0f,0x800,0x200000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
Boot0014* UEFI OS HD(2,GPT,4e50f1ec-fabf-4c32-9d20-56c36a62a011,0x800,0x200000)/File(\EFI\BOOT\BOOTSo the systemd-boot is back to being the default order instead of the proxmox entry. Is this the expected behavior, or have I messed up something?
Thanks in advance for any help.