MS-01 i9-13900H Xe iGPU Full Passthrouh

M

markc

Active Member
Sep 12, 2020
58
13
28
70
Gold Coast, Australia
spiderweb.com.au
My Minisforum MS-01 finally arrived after 4 months so I've spent nearly 24 hous trying to see if I can get iGPU passthrough to work, no luck so far. I'll detail my settings and results so far and see if anyone can comment. Hopefully this thread will end up with a working example that can apply to all MS-01s with Xe gfx. I'm only interested in a Linux guest (CachyOS FWIW) but I'm happy to test Ubuntu/Debian/Win10 if I have any success with a CachyOS guest VM.

First up, my current PVE version, fastfetch and grub output...

Code: 
~ pveversion
pve-manager/8.2.2/9355359cd7afbae4 (running kernel: 6.8.4-2-pve)

~ fastfetch --logo none
OS: Debian GNU/Linux bookworm 12.5 x86_64
Host: Venus Series
Kernel: 6.8.4-2-pve
Uptime: 1 hour, 23 mins
Packages: 744 (dpkg)
Shell: bash 5.2.15
Terminal: /dev/pts/0
CPU: 13th Gen Intel(R) Core(TM) i9-13900H (20) @ 5.40 GHz
GPU: Intel Iris Xe Graphics
Memory: 19.97 GiB / 94.03 GiB (21%)
Swap: Disabled
Disk (/): 2.72 GiB / 1.74 TiB (0%) - zfs
Disk (/rpool): 128.00 KiB / 1.74 TiB (0%) - zfs
Local IP (vmbr0): 192.168.20.25/24 *
Locale: en_US.UTF-8

~ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-6.8.4-2-pve root=ZFS=/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet
intel_iommu=on iommu=pt
pcie_acs_override=downstream,multifunction initcall_blacklist=sysfb_init
vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1
modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu,snd_hda_intel,snd_hda_codec_hdmi,i915,xe

I don't recall doing anything in the BIOS of the host. This is the IOMMU output...

Code: 
~ dmesg | grep -e DMAR -e IOMMU
[    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[    0.019771] ACPI: DMAR 0x0000000042767000 000088 (v02 INTEL  EDK2     00000002      01000013)
[    0.019797] ACPI: Reserving DMAR table memory at [mem 0x42767000-0x42767087]
[    0.112877] DMAR: IOMMU enabled
[    0.238012] DMAR: Host address width 39
[    0.238013] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.238018] DMAR: dmar0: reg_base_addr fed90000 ver 4:0 cap 1c0000c40660462 ecap 29a00f0505e
[    0.238019] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.238022] DMAR: dmar1: reg_base_addr fed91000 ver 5:0 cap d2008c40660462 ecap f050da
[    0.238023] DMAR: RMRR base: 0x0000004c000000 end: 0x000000503fffff
[    0.238025] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.238026] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.238026] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.239561] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.668194] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics
[    0.750606] DMAR: No ATSR found
[    0.750607] DMAR: No SATC found
[    0.750608] DMAR: IOMMU feature fl1gp_support inconsistent
[    0.750608] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.750609] DMAR: IOMMU feature nwfs inconsistent
[    0.750610] DMAR: IOMMU feature dit inconsistent
[    0.750610] DMAR: IOMMU feature sc_support inconsistent
[    0.750611] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.750611] DMAR: dmar0: Using Queued invalidation
[    0.750614] DMAR: dmar1: Using Queued invalidation
[    0.751192] DMAR: Intel(R) Virtualization Technology for Directed I/O

This the current guest config and keep in mind I have already thrown nearly every thing at the wall to see what might stick so these settings may have already changed...

Code: 
~ qm config 100
agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0;net0
cores: 4
cpu: host
efidisk0: local-zfs:vm-100-disk-1,efitype=4m,size=1M
hostpci0: 0000:00:02.0,pcie=1,rombar=0,x-vga=1
machine: q35
memory: 16384
meta: creation-qemu=8.1.2,ctime=1709278650
name: cos
net0: virtio=BC:24:11:C4:66:DC,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: local-zfs:vm-100-disk-0,discard=on,iothread=1,size=32G
scsihw: virtio-scsi-single
smbios1: uuid=da2668c8-7e82-4840-8d7e-b8b74111eb71
sockets: 1
tags: CachyOS
vga: none
vmgenid: df113d7e-2251-4b75-bf28-d65acfd721f5

This is promising inside the guest as there is now no renderD128 device on the host and a lsof /dev/dri/renderD128 shows a heap of Plasma desktop processes using this device so the desktop is happy on the inside of the guest but the video output is not reaching the HDMI monitor.

Code: 
~ ll /dev/dri*
total 0
drwxr-xr-x 2 root root         80 May  6 12:23 by-path/
crw-rw---- 1 root video  226,   0 May  6 12:23 card0
crw-rw-rw- 1 root render 226, 128 May  6 12:23 renderD128

But here we go, it looks like the i915 kernel driver is crashing inside the guest...

Code: 
~ sudo dmesg -w (trimmed output)
[    4.128907] i915 0000:01:00.0: [drm] [ENCODER:235:DDI A/PHY A] failed to retrieve link info, disabling eDP
[    4.129061] ------------[ cut here ]------------
[    4.129062] i915 0000:01:00.0: Platform does not support port C
[    4.129083] WARNING: CPU: 1 PID: 502 at drivers/gpu/drm/i915/display/intel_display.c:7473 assert_port_valid+0x58/0x70 [i915]
[    4.129192] Modules linked in: intel_rapl_msr intel_rapl_common intel_uncore_frequency_common intel_pmc_core intel_vsec pmt_telemetry pmt_class kvm_intel kvm i915(+) irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic gf128mul ghash_clmulni_intel snd_hda_intel sha512_ssse3 sha256_ssse3 snd_intel_dspcfg sha1_ssse3 snd_intel_sdw_acpi aesni_intel snd_hda_codec crypto_simd drm_buddy snd_hda_core cryptd i2c_algo_bit snd_hwdep ttm vfat rapl drm_display_helper fat snd_pcm cec snd_timer video snd soundcore psmouse lpc_ich wmi pcspkr i2c_i801 i2c_smbus joydev mousedev mac_hid lz4 lz4_compress pkcs8_key_parser i2c_dev crypto_user fuse loop dm_mod nfnetlink zram qemu_fw_cfg ip_tables x_tables zfs(POE) hid_generic usbhid spl(OE) virtio_net serio_raw atkbd net_failover libps2 failover virtio_scsi vivaldi_fmap virtio_console virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev i8042 intel_agp intel_gtt serio
[    4.129225] CPU: 1 PID: 502 Comm: (udev-worker) Tainted: P           OE      6.8.7-2-cachyos #1 e216a10ae2e41484fd3701f8809ebda5efe42ea6
[    4.129227] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 4.2023.08-4 02/15/2024
[    4.129228] RIP: 0010:assert_port_valid+0x58/0x70 [i915]
[    4.129310] Code: 7f 08 8d 6e 41 4c 8b 67 50 4d 85 e4 75 03 4c 8b 27 e8 6c 2c 5c ef 89 e9 4c 89 e2 48 c7 c7 40 68 38 c1 48 89 c6 e8 78 a5 ba ee <0f> 0b 89 d8 5b 5d 83 e0 01 41 5c c3 cc cc cc cc 0f 1f 84 00 00 00
[    4.129312] RSP: 0018:ffffaf0f01f27668 EFLAGS: 00010282
[    4.129313] RAX: 0000000000000000 RBX: 000000000000001e RCX: 0000000000000027
[    4.129314] RDX: ffff89c8efca1e88 RSI: 0000000000000001 RDI: ffff89c8efca1e80
[    4.129315] RBP: 0000000000000043 R08: 00000000ffffffea R09: 00000000ffffefff
[    4.129315] R10: ffffffffb1e5b840 R11: 0000000000000003 R12: ffff89c5816a42a0
[    4.129316] R13: ffff89c596759ab8 R14: 0000000000000002 R15: ffff89c581727000
[    4.129317] FS:  00007f4888aa6540(0000) GS:ffff89c8efc80000(0000) knlGS:0000000000000000
[    4.129318] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    4.129318] CR2: 00007f4887db4cbf CR3: 000000010ad04000 CR4: 0000000000750ef0
[    4.129321] PKRU: 55555554
[    4.129321] Call Trace:
[    4.129322]  <TASK>
[    4.129323]  ? __warn+0x81/0x1b0
[    4.129327]  ? assert_port_valid+0x58/0x70 [i915 f31f984425d8753430389b59802635a073912311]
TRIMMED OUTPUT
[    4.130687]  entry_SYSCALL_64_after_hwframe+0x78/0x80
[    4.130690] RIP: 0033:0x7f488956f48d
[    4.130704] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 73 c8 0c 00 f7 d8 64 89 01 48
[    4.130705] RSP: 002b:00007fff6be91c18 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[    4.130707] RAX: ffffffffffffffda RBX: 0000641df0992ba0 RCX: 00007f488956f48d
[    4.130708] RDX: 0000000000000004 RSI: 00007f488967f376 RDI: 0000000000000030
[    4.130709] RBP: 00007f488967f376 R08: 0000000000000070 R09: 00007fff6be91c60
[    4.130710] R10: 0000000000000050 R11: 0000000000000246 R12: 0000000000020000
[    4.130710] R13: 0000641df098f290 R14: 0000000000000000 R15: 0000641df0993a00
[    4.130712]  </TASK>
[    4.130712] ---[ end trace 0000000000000000 ]---
[    4.158040] i915 0000:01:00.0: [drm] GT0: GuC firmware i915/adlp_guc_70.bin version 70.20.0
[    4.158043] i915 0000:01:00.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
[    4.172775] i915 0000:01:00.0: [drm] GT0: HuC: authenticated for all workloads
[    4.173651] i915 0000:01:00.0: [drm] GT0: GUC: submission enabled
[    4.173651] i915 0000:01:00.0: [drm] GT0: GUC: SLPC enabled
[    4.174132] i915 0000:01:00.0: [drm] GT0: GUC: RC enabled
[    4.174984] i915 0000:01:00.0: [drm] Protected Xe Path (PXP) protected content support initialized
[    4.175460] [drm] Initialized i915 1.6.0 20230929 for 0000:01:00.0 on minor 0
[    4.175571] i915 display info: display version: 13
[    4.175574] i915 display info: cursor_needs_physical: no
[    4.175574] i915 display info: has_cdclk_crawl: yes
[    4.175575] i915 display info: has_cdclk_squash: no
[    4.175576] i915 display info: has_ddi: yes
[    4.175576] i915 display info: has_dp_mst: yes
[    4.175577] i915 display info: has_dsb: yes
[    4.175577] i915 display info: has_fpga_dbg: yes
[    4.175577] i915 display info: has_gmch: no
[    4.175578] i915 display info: has_hotplug: yes
[    4.175579] i915 display info: has_hti: no
[    4.175579] i915 display info: has_ipc: yes
[    4.175580] i915 display info: has_overlay: no
[    4.175580] i915 display info: has_psr: yes
[    4.175581] i915 display info: has_psr_hw_tracking: no
[    4.175581] i915 display info: overlay_needs_physical: no
[    4.175581] i915 display info: supports_tv: no
[    4.175582] i915 display info: has_hdcp: yes
[    4.175582] i915 display info: has_dmc: yes
[    4.175583] i915 display info: has_dsc: yes
[    5.055782] i915 0000:01:00.0: [drm] Cannot find any crtc or sizes

If anyone has had any success with full iGPU passthrough with any MS-01 then I'd appreciate any hints.
 
Last edited:
Apologies for the bump but there should be 100+'s more MS-01s in the hands of Proxmox users by now and I am still hoping to hear of any success by anyone trying iGPU FULL passthrough with any MS-01?

_gabriel, my i915 was blacklisted on the host. FWIW.
 
I've had some major stability issues even with the latest microcode. How's the stability on the MS-01 so far?
 
zeptrey said:
I've had some major stability issues even with the latest microcode. How's the stability on the MS-01 so far?
Click to expand...

Mine seems fine with no crashes or lockups. Up 10 days atm and just checked dmesg output with nothing dramatic there either. Uptodate packages with pve-firmware 3.11 and proxmox-kernel-6.8 6.8.4-3. It's under reasonable load. I have 1TB in the 3x2 slot for the PVE host, a 2TB in the 3x4 slot for Ceph and a 4TB drive (all Crucial) in the 4x4 slot passed through to a PBS LXC container. Backups are my main concern so I gave PBS the fastest NVMe slot.

So... has no one yet managed to get full iGPU passthrough to work... at all?

What about with an ARC/Intel card in the PCIe slot?

At worst, are there any confirmed nVidia cards that will handle full GPU passthrough to a LINUX guest VM?
 
Last edited:
I found a Post online on Derek Seaman's Tech Blog is suggesting that (GPU) PCIe pass trough For
Alderlake / Raptor lake model iGPUs is not working due to a kernel issue with the 6.8.x-x-pve.

On this post a procedure is suggested to mitigate the issue, by downgrading
the kernel to 6.5.13-3 resulting in successful pass trough.

I did not test this, but I am considering to try it,

Obviously this is not something to try on a production system.
 
Last edited:
hein.waz said:
I found a Post online on Derek Seaman's Tech Blog is suggesting that (GPU) PCIe pass trough For
Alderlake / Raptor lake model iGPUs is not working due to a kernel issue with the 6.8.x-x-pve.

On this post a procedure is suggested to mitigate the issue, by downgrading
the kernel to 6.5.13-3 resulting in successful pass trough.

I did not test this, but I am considering to try it,

Obviously this is not something to try on a production system.
Click to expand...
I tried the above, and it seems to work okay... Still getting these errors on boot from dmesg -H:
Code: 
[  +0.000052] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.6 on minor 6
[  +0.000140] pci 0000:00:02.7: [8086:a7a0] type 00 class 0x030000
[  +0.000016] pci 0000:00:02.7: DMAR: Skip IOMMU disabling for graphics
[  +0.000043] pci 0000:00:02.7: Adding to iommu group 29
[  +0.000005] pci 0000:00:02.7: vgaarb: bridge control possible
[  +0.000001] pci 0000:00:02.7: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[  +0.000003] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[  +0.000003] i915 0000:00:02.1: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  +0.000002] i915 0000:00:02.2: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  +0.000003] i915 0000:00:02.3: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  +0.000002] i915 0000:00:02.4: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  +0.000003] i915 0000:00:02.5: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none
[  +0.000002] i915 0000:00:02.6: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
[  +0.000038] i915 0000:00:02.7: enabling device (0000 -> 0002)
[  +0.000013] i915 0000:00:02.7: Running in SR-IOV VF mode
[  +0.000116] i915 0000:00:02.7: [drm] *ERROR* GT0: IOV: Unable to confirm version 1.9 (0000000000000000)
[  +0.000049] i915 0000:00:02.7: [drm] *ERROR* GT0: IOV: Found interface version 0.1.9.0
[  +0.000279] i915 0000:00:02.7: [drm] VT-d active for gfx access
[  +0.000017] i915 0000:00:02.7: [drm] Using Transparent Hugepages
[  +0.000299] i915 0000:00:02.7: [drm] *ERROR* GT0: IOV: Unable to confirm version 1.9 (0000000000000000)
[  +0.000056] i915 0000:00:02.7: [drm] *ERROR* GT0: IOV: Found interface version 0.1.9.0
[  +0.000178] i915 0000:00:02.7: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  +0.000002] i915 0000:00:02.7: HuC firmware PRELOADED
[  +0.001823] i915 0000:00:02.7: [drm] Protected Xe Path (PXP) protected content support initialized
[  +0.000005] i915 0000:00:02.7: [drm] PMU not supported for this GPU.
[  +0.000091] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.7 on minor 7
[  +0.000203] i915 0000:00:02.0: Enabled 7 VFs

But...at at the end it says it's enabled? And I can see inside of my Plex LXC that I can use the iGPU to transcode with. HOWEVER, Windows did not like using it at all. Windows 11 kept crashing and 10 would just shutdown the gpu. SO YMMV.
 
Last edited:
  • Like
Reactions: okiedokie, ydnah2 and markc
ReplacedRiver47 said:
Hey guys, so I just figured this one out, after messing with it for a couple days.
Click to expand...

Congratulations, well done. So after a month, is your VM still working okay? Any gotchas or points you would add to the setup procedure?

FWIW, so far, I've made a start on translating the gangqizai/igd repository, so I can more easily follow along. I intend to update the README and repo as I test it out and will fine tune it specifically for the MS-01...

https://github.com/markc/ms01-igpu-passthrough
 
  • Like
Reactions: okiedokie
hi @markc @ReplacedRiver47 I wanted to follow up and see if your respective systems are stable and operating properly with HDMI output. I've had a difficult time locating a guide with full passthrough using the Intel igpu, and this is the closest I've found. Have either of you found success with Windows 11?

Thanks @markc for creating the github translation.
 
jun3280 said:
hi @markc @ReplacedRiver47 I wanted to follow up and see if your respective systems are stable and operating properly with HDMI output. I've had a difficult time locating a guide with full passthrough using the Intel igpu, and this is the closest I've found. Have either of you found success with Windows 11?

Thanks @markc for creating the github translation.
Click to expand...
It works (almost) just fine.
I have a 12900H version (shouldn't be any difference) and one of the VMs is a Kodi machine with HDMI output for both video and audio. AV output also working on any of the TB ports
The only (kinda unsolvable) annoyance is that once the VM reboots, it can no longer see HDMI audio outputs. Host restart fixes that until next VM reboot. As far as I understand, this is HW limitation of the audio device that does not support PCIe reset. This prevents it to be released and reinitialized on VM reboot.
As a side note, seems like there is no need for audio rom. It works exactly the same with or without it.
 
BDYB said:
It works (almost) just fine.
I have a 12900H version (shouldn't be any difference) and one of the VMs is a Kodi machine with HDMI output for both video and audio. AV output also working on any of the TB ports
The only (kinda unsolvable) annoyance is that once the VM reboots, it can no longer see HDMI audio outputs. Host restart fixes that until next VM reboot. As far as I understand, this is HW limitation of the audio device that does not support PCIe reset. This prevents it to be released and reinitialized on VM reboot.
As a side note, seems like there is no need for audio rom. It works exactly the same with or without it.
Click to expand...

Thanks for your reply @BDYB. I may get an Intel box to try this on. Curious, will full passthrough, there's the igpu is fully locked down to just one VM right? Or can you share the gpu with other VMs - I've read people using it in Jellyfin?
 
jun3280 said:
Thanks for your reply @BDYB. I may get an Intel box to try this on. Curious, will full passthrough, there's the igpu is fully locked down to just one VM right? Or can you share the gpu with other VMs - I've read people using it in Jellyfin?
Click to expand...
Full passthrough is exactly what the name suggest - some VM completely owns it.
There is also GPU sharing. Intel fully allows to split GPU functions across multiple devices, but there will be no audio/video output from VMs on that GPU.
 
  • Like
Reactions: jun3280
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back