[SOLVED] Intel GVT-g: can't assign more than one 5_4 mdev

[kvn1351]

Hi, I've been stuck on this for almost an entire day now...

The documentation in regards to GVT-g isn't that abundant. I expected to be able to use at least two i915-GVTg_V5_4 vGPUs. Instead, I'm only able to use 1
V5_4 or 2 V5_8. The resulting performance in my two VMs is thus absolutely terrible.

I've also seen that others have more supported mdev types (like on the arch page):

# ls /sys/devices/pci${GVT_DOM}/$GVT_PCI/mdev_supported_types
i915-GVTg_V5_1
i915-GVTg_V5_2
i915-GVTg_V5_4
i915-GVTg_V5_8

Instead, all I get are these:

root@prox:~# ls /sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types
i915-GVTg_V5_4
i915-GVTg_V5_8

So what am I missing here? Am I really just supposed to have one single V5_4 available?

Info​

CPU: Intel 10400

root@prox:~# apt show qemu-system-x86
Package: qemu-system-x86
Version: 1:3.1+dfsg-8+deb10u8

Kernel Version: Linux 5.4.106-1-pve #1 SMP PVE 5.4.106-1 (Fri, 19 Mar 2021 11:08:47 +0100)

I've been following these guides/docs:
https://cetteup.com/216/how-to-use-...rdware-accelerated-streaming-in-a-proxmox-vm/
https://wiki.archlinux.org/index.php/Intel_GVT-g

root@prox:~# cat /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
#GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on i915.enable_gvt=1 drm.debug=0"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on i915.enable_dc=0 i915.enable_guc=0 i915.enable_gvt=1 kvm.ignore_msrs=1 kvm.report_ignored_msrs=0"
GRUB_CMDLINE_LINUX=""

# Disable os-prober, it might add menu entries for each guest
GRUB_DISABLE_OS_PROBER=true

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

root@prox:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

# Modules required for PCI passthrough
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

# Modules required for Intel GVT
kvmgt
exngt
vfio-mdev

root@prox:~# ls /sys/bus/pci/devices/0000:00:02.0
00000000-0000-0000-0000-000000000100  current_link_speed  drm            index          local_cpus            numa_node  resource2         subsystem_vendor
ari_enabled                           current_link_width  enable         iommu          max_link_speed        power      resource2_wc      uevent
boot_vga                              d3cold_allowed      firmware_node  iommu_group    max_link_width        remove     resource4         vendor
broken_parity_status                  device              gvt_firmware   irq            mdev_supported_types  rescan     revision
class                                 dma_mask_bits       i2c-0          label          modalias              reset      rom
config                                driver              i2c-1          link           msi_bus               resource   subsystem
consistent_dma_mask_bits              driver_override     i2c-2          local_cpulist  msi_irqs              resource0  subsystem_device

00000000-0000-0000-0000-000000000100 is the UUID of the only 5_4 vGPU I'm able to assign.

I'd be very grateful for any kind of help.

Have a nice day,
Kev

 

[dcsapak]

what is the reason you expected to use 2 V5_4 devices? these values are given by the kernel driver/hardware and is not really something we can change
what can play a role is the assigned video memory (in the bios)

also a newer kernel (for testing 5.11 is available) could help here

 

[kvn1351]

I found the issue. It was video memory-related indeed. I fixed it increasing the aperture size. Since my BIOS didn't have this option I had to find the offset and set it with a modified GRUB shell.

I've documented the proceedings here:
https://github.com/intel/gvt-linux/issues/131#issuecomment-815362316

Thanks.