Nvidia PCIE Passthough Video/Audio issues

[sourceminer]

Hey guys,
I have a windows 10 vm that is using PCIE Passthough. The video works, but once in a while the windows machine will start to boot but then video disappears. I have tried the updated Drivers for the card and windows default drivers. Also another interesting issue is that audio though the HDMI starts to crackle then completely distort. Anyone else having these issues?

Here is the VM config:

agent: 1
bios: ovmf
boot: cdn
bootdisk: sata0
cores: 4
hostpci0: 06:00.0;06:00.1,pcie=1,x-vga=on
ide2: none,media=cdrom
machine: q35
memory: 8000
name: Windows10
net0: virtio=3A:62:32:62:61:66,bridge=vmbr0
numa: 0
onboot: 1
ostype: win8
sata0: Raid6-VMData:vm-103-disk-1,cache=writeback,size=144G
scsihw: virtio-scsi-pci
smbios1: uuid=873e1bb7-6925-47c4-b693-02e8507fcc6f
sockets: 2
unused0: local:103/vm-103-disk-1.qcow2
usb0: host=6-2

pveversion

root@proxmox:/etc/pve/qemu-server# pveversion -v
proxmox-ve: 4.2-54 (running kernel: 4.4.10-1-pve)
pve-manager: 4.2-15 (running version: 4.2-15/6669ad2c)
pve-kernel-4.2.6-1-pve: 4.2.6-36
pve-kernel-4.4.10-1-pve: 4.4.10-54
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-42
qemu-server: 4.0-81
pve-firmware: 1.1-8
libpve-common-perl: 4.0-68
libpve-access-control: 4.0-16
libpve-storage-perl: 4.0-55
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-19
pve-container: 1.0-68
pve-firewall: 2.0-29
pve-ha-manager: 1.0-32
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve9~jessie

Grub Option:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

root@proxmox:/etc/pve/qemu-server# lspci -n -s 6:
06:00.0 0300: 10de:1380 (rev a2)
06:00.1 0403: 10de:0fbc (rev a1)

Also getting this error:
DMAR: Allocating domain for dcdbas failed
I see someone else having this without a response.
https://forum.proxmox.com/threads/i...ar-allocating-domain-for-dcdbas-failed.28398/

Found this article which suggests changing grub to intel_iommu=pt
https://centoros.wordpress.com/2015/07/22/iommu-dmar-fault/
** This didnt help. Caused the VM not to boot at all.

 

[sourceminer]

Is there paid support? Kind of disappointed in this forum. Most of the questions I ask dont seem to have any responses. Does anyone monitor?
Any direction at all?

I still have the issue although USB Logitech Speakers are working better, but still has some signs of jitter, and I cannot tell why.

 

[tom]

First, update to latest stable version, working on outdated installations make no sense.

See http://pve.proxmox.com/wiki/Downloads#Update_a_running_Proxmox_Virtual_Environment_4.x_to_latest_4.3

 

[sourceminer]

Thanks for the response Tom, I was thinking about this but looking at the release notes didn't bring out anything that might be related.
I will update tonight.

For what its worth I did find this which improved the Audio coming from HDMI. Its actually usable now, however still some cracking sounds, like there is a buffer overrun.

This might be helpful for others.
http://vfio.blogspot.com/2014/09/vfio-interrupts-and-how-to-coax-windows.html
- Specifically I used the Tool http://vfio.blogspot.com/2014/09/vfio-interrupts-and-how-to-coax-windows.html
The Tool was nice as it shows you what devices to enable MSI Mode ( I enabled MSI on the Video and HD Audio).

After doing this HDMI like I said was now usable, but still not optimal.

 

[sourceminer]

Ok so I updated my version to 4.3 (of course no real change).
Still have audio chop.

root@proxmox:~# pveversion -v
proxmox-ve: 4.3-71 (running kernel: 4.4.21-1-pve)
pve-manager: 4.3-10 (running version: 4.3-10/7230e60f)
pve-kernel-4.2.6-1-pve: 4.2.6-36
pve-kernel-4.4.21-1-pve: 4.4.21-71
pve-kernel-4.4.10-1-pve: 4.4.10-54
lvm2: 2.02.116-pve3
corosync-pve: 2.4.0-1
libqb0: 1.0-1
pve-cluster: 4.0-47
qemu-server: 4.0-94
pve-firmware: 1.1-10
libpve-common-perl: 4.0-80
libpve-access-control: 4.0-19
libpve-storage-perl: 4.0-68
pve-libspice-server1: 0.12.8-1
vncterm: 1.2-1
pve-docs: 4.3-14
pve-qemu-kvm: 2.7.0-6
pve-container: 1.0-81
pve-firewall: 2.0-31
pve-ha-manager: 1.0-35
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 2.0.5-1
lxcfs: 2.0.4-pve2
criu: 1.6.0-1
novnc-pve: 0.5-8
smartmontools: 6.5+svn4324-1~pve80
zfsutils: 0.6.5.8-pve13~bpo80

 

[efeu]

The sound problem is a KVM or specially VFIO Problem with interrupts, for some devices your mentioned MSI Trick helps, this problem is known for a long time, but no real progress, just some workarounds (which did not help me, so I forgot them, sry). What always helps is buying a dedicated sound card (pci/pcie) and passthrough it to vm or a dedicated usb controller (pci/pcie) and passthrough it to vm and use usb soundcard/headset.
Passing through USB Ports or the whole onboard USB Controller will not help (I tried a lot, cause I have/had the same problem, but it's not with proxmox, it's KVM/VFIO Problem with onboard devices).

It seems that a lot of mainboards (specially consumer boards) have not fully implemented IOMMU and/or vt-d and this causes the problem.

This is my grub cmdline:
quiet intel_iommu=on pcie_acs_override=downstream

If u use NVidia Card, your missing the kvm hide options in your kvm config, here is mine (should always work for NV):
args: -cpu host,kvm=off,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=Nvidia43FIX

Since pve 4.3 (have had no problems with lower versions) I also have reboot problems with my Win10 VM. If i do a:
qm shutdown xxx / shutdown in the VM
qm start xxx
the VM hangs even before uefi

then after a few times doing this:
qm stop xxx
qm start xxx

The VM boots again, no idea why, no error or something. Its realy annoying.

 

[spirit]

If u use NVidia Card, your missing the kvm hide options in your kvm config, here is mine (should always work for NV):
args: -cpu host,kvm=off,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=Nvidia43FIX

This is already done by proxmox automatically when x-vga=on is enabled on hostpci.

 

[efeu]

Wow, thank you verry much, there must have been a bug in the past, cause I always had the x-vga=on option, but got the NVidia Error 43 with it, then I always added my mentioned args line.

No shutdown/start hang atm.

Seems to be working now. Makes me happy

Regards

 

[sourceminer]

OK so just to be clear the only options that I seem to be missing would be that of the GRUB: pcie_acs_override=downstream
Spirit mentioned that my config with the x-vga=on option turned on was sufficient. Thanks everyone, I will add that update to my GRUB to see if anything changes. EFEU, I thought about getting a dedicated sound card, but wasnt sure if that was still going to produce the same results, as I am not using any local onboard resources. So far my USB speakers sound worse than the HDMI audio passthough of the N-Vidia.

I will keep this post updated as to the results.
Also seems like it would be a good time to update some of the online wiki documentation as there are some discrepancies.

 

[sourceminer]

Sorry for the lack of response, I have been super busy the last week. So I have had the chance to reboot my host with the updated Grub adding the pcie_acs_override=downstream without any improvement. Still lots of audio jitter.

 

[mcflym]

What kind of NVIDIA card do you have?

Edit: OK, i saw it .. 750ti

 

[sourceminer]

Just a Quick update. Simple fix may have been that I didnt have NUMA enabled. So far I have had 0 issues with the Audio since enabling.

 

[trent]

Hi,

I dont know how to start a topic but if you guys would allow me, please.
We are planning to virtualize 3 windows 7 pc,s, and each vm would have a physical video card for s-video output. I am wondering if this kind of set up would work and any possible issues would be encountered.

Any response would be greatly appreciated.

Thanks.

 

[sourceminer]

Should work fine as long as you have a recommended video card. Mine seems to be working great.