Intel IGP (Gemini Lake) Passthrough (q35) fails to boot on Ubuntu 18.04.3 LTS – i915 conflict detected with stolen region

[n1nj4888]

Hi There,

I currently have a headless Intel NUC7PJYH and can passthrough the Intel IGP to a VM running Ubuntu 18.04.3LTS as long as the machine type is set to “Default (i440fx)”. I don’t use the passthrough Intel IGP for graphics output, I just pass it through as a secondary graphics card (the primary is set to “Default (memory=128)”) to use the Intel Quicksync functionality of the IGP and this seems to work ok…

I wanted to test the setup out but using the newer q35 machine type and passing the same Intel IGP through to a cloned VM where the only change was the machine type from i440fx to q35… With the previous i440fx VM stopped, when starting this new VM, the boot process hangs at the following point (see attached screenshot). I also tried booting the Ubuntu 18.04.3 LTS ISO and that hangs similarly when starting the Ubuntu Setup at the same prompt: [drm:i915_gem_init_stolen [i915]] *ERROR* conflict detected with stolen region: ….



I’ve tried using both machine type “q35” from the GUI and machine type “pc-q35-3.1” (from the console) and both seem to result in the same hang message at boot.

The VM configuration is as below.

agent: 1
bios: ovmf
boot: dcn
bootdisk: scsi0
cores: 4
cpu: host
efidisk0: DataStore:113/vm-113-disk-1.qcow2,size=4M
hostpci0: 00:02.0
ide2: none,media=cdrom
machine: q35
memory: 8192
name: pve-vm-q35-passthrough-test
net0: virtio=<MAC>,bridge=vmbr0
numa: 0
ostype: l26
scsi0: DataStore:113/vm-113-disk-0.qcow2,discard=on,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=<UUID>
sockets: 1
vga: memory=128
vmgenid: <VMGENID>

I currently have the following set on the Proxmox 6 Host:

cat /etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on

And the following set - Although these were not required for the Intel IGP Passthrough to work using the i440fx machine type:

cat /etc/modprobe.d/blacklist.conf
blacklist snd_hda_intel
blacklist snd_hda_codec_hdmi
blacklist i915

cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=8086:3184

Any pointers how to fix this or what I might be doing wrong?

 

[n1nj4888]

Anyone any ideas on this? I've since tried the following combinations:

1. Adding

   root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on video=efifb:off

   to the kernel command line and running

   pve-efiboot-tool refresh

   before rebooting
2. Changing the machine type between q35 and pc-q35-3.1 using

   qm set VMID -machine pc-q35-3.1

3. Adding the "kernel_irqchip=on" arguement using the following - Both with the machine type set to q35 and pc-q35-3.1

4. qm set 113 -args '-machine type=q35,kernel_irqchip=on'

5. Changing the PCI device to PCI Express in the GUI (it worked previously under the i440fx machine type as PCI)
6. Toggling the PCI / PCIe Device "All Functions" to on/off in the GUI

Still I'm getting Ubuntu 18.04.3LTS hanging at the initial boot screens with the following output to the noVNC console:




Current VMID.conf is as follows:

agent: 1
bios: ovmf
boot: dcn
bootdisk: scsi0
cores: 4
cpu: host
efidisk0: DataStore:113/vm-113-disk-1.qcow2,size=4M
hostpci0: 00:02.0
ide2: none,media=cdrom
machine: pc-q35-3.1
memory: 8192
name: pve-vm-q35-passthrough-test
net0: virtio=<MAC>,bridge=vmbr0
numa: 0
ostype: l26
scsi0: DataStore:113/vm-113-disk-0.qcow2,discard=on,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=<UUID>
sockets: 1
vga: memory=128
vmgenid: <VMGENID>

... So, I'm stumped! Would appreciate any insight into how others have got the Gemini Lake IGP passed through under PVE 6.0-6 using the q35 / pc-q35-3.1Machine Type!

Thanks

 

[n1nj4888]

Hi Guys,

Anyone got any suggestions on how to fix this since I'm stumped?

The VM (Ubuntu 18.04.3 LTS Server) fails to boot (hangs at the following screen in the console) when I set the Machine Type to Q35:

agent: 1
bios: ovmf
boot: dcn
bootdisk: scsi0
cores: 4
cpu: host
efidisk0: DataStore:113/vm-113-disk-1.qcow2,size=4M
hostpci0: 00:02.0
ide2: none,media=cdrom
machine: q35
memory: 8192
name: pve-vm-q35
net0: virtio=<MAC>,bridge=vmbr0
numa: 0
ostype: l26
scsi0: DataStore:113/vm-113-disk-0.qcow2,discard=on,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=<UUID>
sockets: 1
vga: memory=128
vmgenid:<VMGENID>

I've also tried changing the Machine Type to the following different options but it still doesn't boot...
qm set ID -machine pc-q35-3.1
qm set ID -args '-machine type=q35,kernel_irqchip=on'


The VM does boot if I delete the VM conf arguments above and revert the Machine Type back to i440fx?

The following is the similar dmesg output in that case where the VM boots with a i440fx Machine Type - Note there is no "Conflict detected with stolen region" and the "owns=io+mem" rather than "owns=none" here - and the DMC firmware i915/glk_dmc_ver1_04.bin (v1.4) is successfully loaded...

[    2.309068] i915 0000:00:10.0: GuC: No firmware known for this platform!
[    2.310139] [drm] HuC: No firmware known for this platform!
[    2.313554] checking generic (c0000000 160000) vs hw (800000000 10000000)
[    2.313556] [drm] Replacing VGA console driver
[    2.330027] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    2.330988] [drm] Driver supports precise vblank timestamp query.
[    2.332690] i915 0000:00:10.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0x0cfc
[    2.332769] hidraw: raw HID events driver (C) Jiri Kosina
[    2.334386] [drm] Failed to find VBIOS tables (VBT)
[    2.337096] i915 0000:00:10.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
[    2.340530] [drm] Finished loading DMC firmware i915/glk_dmc_ver1_04.bin (v1.4)
[    2.343913] [drm] Applying Increase DDI Disabled quirk

 

[n1nj4888]

I've tested this on both Coffee Lake (NUC8i5BEH) and Gemini Lake (NUC7PJYH) and both exhibit the same issue - The Ubuntu 18.04.3 LTS VM boots using i440fx Machine Type but fails to boot on both NUCs using either q35 or pc-q35-3.1 Machine Types

I've raised this as a bug at: https://bugzilla.proxmox.com/show_bug.cgi?id=2381

 

[n1nj4888]

Hi Guys,

Some progress... I've just tried booting two different Ubuntu Server 19.10 VMs with the Intel IGP passed-through, one with q35 machine type and the other with i440fx machine type - Like Ubuntu 18.04.3 LTS, the i440fx seems to work fine but the q35 VM console hangs at the following screen




After about 20-30seconds though, I'm able to SSH to the VM (which I wasn't able to do with the Ubuntu 18.04.3 LTS q35 VM) but I still get the i915 conflict detected with stolen region error message and the VM console does not get passed the screen above...

Anything I'm doing wrong with the passthrough here for the q35 VM?

The VMID.conf is as follows;

agent: 1,fstrim_cloned_disks=1
bios: ovmf
bootdisk: scsi0
cores: 4
efidisk0: DataStore:121/vm-121-disk-1.qcow2,size=128K
hostpci0: 00:02.0
ide2: DataStore:iso/ubuntu-19.10-live-server-amd64.iso,media=cdrom
machine: q35
memory: 8192
name: pve-vm-ubuntu-server-19-10-q35
net0: virtio=<MACADDRESS>,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
parent: Snapshot4
scsi0: DataStore:121/vm-121-disk-0.qcow2,discard=on,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=<UUID>
sockets: 1
vmgenid: <VMGENID>

And here is some sample output of "drm" messages from dmesg from the VM:

# dmesg | grep drm
[    1.030376] bochs-drm 0000:00:01.0: remove_conflicting_pci_framebuffers: bar 0: 0xc0000000 -> 0xc0ffffff
[    1.031360] bochs-drm 0000:00:01.0: remove_conflicting_pci_framebuffers: bar 2: 0xc324b000 -> 0xc324bfff
[    1.032361] fb0: switching to bochsdrmfb from EFI VGA
[    1.032869] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console
[    1.032968] [drm] Found bochs VGA, ID 0xb0c5.
[    1.032970] [drm] Framebuffer size 16384 kB @ 0xc0000000, mmio @ 0xc324b000.
[    1.033115] [drm] Initialized bochs-drm 1.0.0 20130925 for 0000:00:01.0 on minor 0
[    1.033640] fbcon: bochs-drmdrmfb (fb0) is primary device
[    1.067067] bochs-drm 0000:00:01.0: fb0: bochs-drmdrmfb frame buffer device
[    1.076418] [drm] Found 128MB of eDRAM
[    1.076746] [drm:i915_gem_init_stolen [i915]] *ERROR* conflict detected with stolen region: [mem 0x7c000000-0x7fffffff]
[    1.104763] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    1.105276] [drm] Driver supports precise vblank timestamp query.
[    1.128538] [drm:gen9_set_dc_state [i915]] *ERROR* DC state mismatch (0x0 -> 0x2)
[    1.131687] [drm] Finished loading DMC firmware i915/kbl_dmc_ver1_04.bin (v1.4)
[    2.234051] [drm] failed to retrieve link info, disabling eDP
[    3.216907] [drm] Initialized i915 1.6.0 20190619 for 0000:06:10.0 on minor 1
[    3.254161] [drm] Cannot find any crtc or sizes
[    3.289569] [drm] Cannot find any crtc or sizes
[    3.327230] [drm] Cannot find any crtc or sizes

Thanks!

 

[Rayures]

having the same issue.

Q35, GVT-g, Ubuntu 20.04 LTS boot hangs in PVE Console and cannot ever SSH into the machine.

but when i set 'display' to none, Ubuntu does boot and i can ssh into the machine (like 10seconds after start).
did not test if i.e. quicksync is working.

quest dmesg:

root@test:~$ dmesg | grep i915
[    1.002603] WARNING: CPU: 1 PID: 159 at drivers/gpu/drm/i915/gt/uc/intel_uc.c:34 __intel_uc_reset_hw+0x66/0x70 [i915]
[    1.002606] Modules linked in: i915(+) aesni_intel(+) video i2c_algo_bit crypto_simd drm_kms_helper cryptd glue_helper psmouse virtio_net i2c_i801 syscopyarea sysfillrect sysimgblt fb_sys_fops ahci(+) libahci lpc_ich virtio_scsi net_failover failover drm
[    1.002662] RIP: 0010:__intel_uc_reset_hw+0x66/0x70 [i915]
[    1.002728]  __uc_sanitize+0x38/0x40 [i915]
[    1.002761]  intel_uc_sanitize+0x14/0x20 [i915]
[    1.002794]  intel_gt_sanitize+0x1c/0x90 [i915]
[    1.002827]  i915_gem_sanitize+0x52/0x90 [i915]
[    1.002861]  i915_gem_init_mmio+0xe/0x10 [i915]
[    1.002888]  i915_driver_mmio_probe+0xc4/0x310 [i915]
[    1.002914]  i915_driver_probe+0x223/0x5d0 [i915]
[    1.002967]  i915_pci_probe+0x5a/0x140 [i915]
[    1.003041]  i915_init+0x61/0x6c [i915]
[    1.003396] i915 0000:01:00.0: vgaarb: deactivate vga console
[    1.039898] i915 0000:01:00.0: iGVT-g active, disabling use of stolen memory
[    1.059388] i915 0000:01:00.0: BAR 6: can't assign [??? 0x00000000 flags 0x20000000] (bogus alignment)
[    1.065277] i915 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[    1.113546] [drm] Finished loading DMC firmware i915/kbl_dmc_ver1_04.bin (v1.4)
[    1.212882] [drm] Initialized i915 1.6.0 20190822 for 0000:01:00.0 on minor 0
[    1.570495] fbcon: i915drmfb (fb0) is primary device
[    1.608717] i915 0000:01:00.0: fb0: i915drmfb frame buffer device

vm.conf:

agent: 1
balloon: 1024
bios: ovmf
bootdisk: scsi0
cores: 2
cpu: host
efidisk0: ZFSdata:vm-121-disk-1,size=1M
hostpci0: 00:02.0,mdev=i915-GVTg_V5_8,pcie=1
ide2: none,media=cdrom
machine: q35
memory: 2048
name: test
net0: virtio=72:88:76:D9:71:B5,bridge=vmbr0,firewall=1
numa: 1
ostype: l26
scsi0: ZFSdata:vm-121-disk-0,discard=on,iothread=1,size=128G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=80f7e114-b8b2-4188-a7c2-3cdaeae881e1
sockets: 1
vga: none
vmgenid: 98160115-3975-4ea3-84c3-334283a77479