[SOLVED] PCI/GPU passthrough and GUI malfunction

tredok · Nov 11, 2019

Hello,

I want to test how GPU Pasthrough works. If everything is ok, then I will buy à GPU. But, I've got a problem.

I've install a graphic card on my motherboard (there is an integrated GPU). It is well detected, the system boot. Everything works... except I can't access to Proxmox at distance.

* the GUI doesn't work
* I can't connect to SSH

The problem occurs before I configure the PCI passthrough. So, I've configured Proxmox, I've stopped before the creation of the VM. Nothing changes.

At fisrt sight, I though it was an network error. I've searched information :

* I can ping Proxmox
* I've got one VM, a server. It is accessible via a domain name. It works perfectly
* GPU and network are not in the same IOMMU group

Then, I thought it was maybe an hardware issue, so I've tried to change :

* another alimentation
* the second PCIe port
* another PCIe device (SATA controler)

Every devices connect is well detected, the lspci command give all information and they seems correct.

For me, that's an mystery ^^

Do you have an idea ? Any command or log I could watch for ?

dcsapak · Nov 11, 2019

tredok said:
. except I can't access to Proxmox at distance.

what do you mean exactly by this? does the webpage not load? you cannot connect via ssh ?

tredok · Nov 11, 2019

Exactly. Without the PCI devices, the webpage and SSH connection works. With a pci device, it doesn't.

If I unplug the PCI device, it works ^^

dcsapak · Nov 11, 2019

i would check if the address (and thus the name) of the nic changes, e.g. from enp3s0 to enp4s0.
check with 'ip addr' and 'ip link'

tredok · Nov 11, 2019

There are differences : enp2s0 and enp3s0.

If I understand, enp2s0 config says to use vmbr0, but not enp3s0 ?

Without PCIe

Code:

root@grosveinard:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
    link/ether 90:2b:34:66:fe:be brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 90:2b:34:66:fe:be brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.253/24 brd 192.168.1.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::922b:34ff:fe66:febe/64 scope link
       valid_lft forever preferred_lft forever
8: tap101i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr101i0 state UNKNOWN group default qlen 1000
    link/ether e2:34:e5:62:ee:a6 brd ff:ff:ff:ff:ff:ff
9: fwbr101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether da:c9:60:9a:51:7b brd ff:ff:ff:ff:ff:ff
10: fwpr101p0@fwln101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether ea:53:b1:1b:76:a5 brd ff:ff:ff:ff:ff:ff
11: fwln101i0@fwpr101p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000
    link/ether da:c9:60:9a:51:7b brd ff:ff:ff:ff:ff:ff

root@grosveinard:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 90:2b:34:66:fe:be brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 90:2b:34:66:fe:be brd ff:ff:ff:ff:ff:ff
4: tap101i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr101i0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 2e:d6:36:2b:91:09 brd ff:ff:ff:ff:ff:ff
5: fwbr101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 0a:1a:01:fe:d5:b4 brd ff:ff:ff:ff:ff:ff
6: fwpr101p0@fwln101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether c2:76:f4:bb:6a:09 brd ff:ff:ff:ff:ff:ff
7: fwln101i0@fwpr101p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP mode DEFAULT group default qlen 1000
    link/ether 0a:1a:01:fe:d5:b4 brd ff:ff:ff:ff:ff:ff

With PCIe

Code:

root@grosveinard:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defau$
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default q$
    link/ether 90:2b:34:66:fe:be brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP gro$
    link/ether 52:09:3b:2a:70:0c brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.253/24 brd 192.168.1.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::f45b:46ff:fe45:df1/64 scope link
       valid_lft forever preferred_lft forever
4: tap101i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fas$

root@grosveinard:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAUL$
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT gr$
    link/ether 90:2b:34:66:fe:be brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mod$
    link/ether 52:09:3b:2a:70:0c brd ff:ff:ff:ff:ff:ff
4: tap101i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fas$
    link/ether 56:34:57:72:fe:49 brd ff:ff:ff:ff:ff:ff
5: fwbr101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP$
    link/ether a2:f3:ea:a0:be:60 brd ff:ff:ff:ff:ff:ff
6: fwpr101p0@fwln101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueu$
    link/ether 52:09:3b:2a:70:0c brd ff:ff:ff:ff:ff:ff
7: fwln101i0@fwpr101p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueu$
    link/ether a2:f3:ea:a0:be:60 brd ff:ff:ff:ff:ff:ff

tredok · Nov 12, 2019

I've found this two thread which seem to give a solution. I'wll try tonight ^^

* https://forum.proxmox.com/threads/networking-error-with-gpu-installed.43638/
* https://forum.proxmox.com/threads/changing-video-card-broke-networking.40848/

tredok · Nov 13, 2019

I've follow the instructions, but it doesn't work :

* connect pci device
* edit /etc/network/interfaces
* change the NIC
* reboot

Nothing happen. When I check with ip link or ip addr, the status is down. I've found the command line to up, nothing change.

Now, I've got a new problem : I reverted the changes, and uninstall the sévices, but the interface is still down :/

jim.bond.9862 · Nov 13, 2019

Your problem may be that your nic and pci you use for gpu are on the same bus.
Try moving the gpu to different pci slot.

tredok · Nov 13, 2019

Allready test. Different slot and different device.

I'll investigate the bus, I'll tell you ^^

jim.bond.9862 · Nov 14, 2019

I only suggested because I had same issue on my older MB.
I was trying to passthrough a nic and it kept disabling my sata add on card with disks ,
Turned out all my pci slots shared the same bus.

tredok · Nov 14, 2019

Do you identify the bus with lspci or another command ?

Edit : there is a schema on my motherboard manual.

jim.bond.9862 · Nov 14, 2019

tredok said:
Do you identify the bus with lspci or another command ?

Edit : there is a schema on my motherboard manual.

I don't remember all I did at the time, but lspci was one of the until for sure. It took a lot of reading,googling and head scratching to get to the answer.
Luckily it was not missions critical, just playing around.

tredok · Nov 14, 2019

I've tested a new install with an unsued drive and all PCi device connect (graphic card, Sata controler and M.2).

Everything works (except M.2, but the device seems out of order). The bus and NIC are the same in this test than my current proxmox.

I don't understand ^^

tredok · Nov 17, 2019

Well, I've solved the problem, but I don't understand how :/

I've change the /etc/network/interfaces and added all NIC founded. When I changed the configuration (add a GPU card, a SATA controler and M.2), the NIC change to enp2s0 to enp4s0.

I've made an 'ip addr' to identify actual NIC (wich give me enp3s0)

I've added three lines :
iface enp2s0 inet manual
iface enp3s0 inet manual
iface enp4s0 inet manual

And modify the bridge_ports line with actual NIC (enp3s0).

Then, I reboot and everything works.

Search

Search

[SOLVED] PCI/GPU passthrough and GUI malfunction

tredok

Member

dcsapak

Proxmox Staff Member

tredok

Member

dcsapak

Proxmox Staff Member

tredok

Member

tredok

Member

tredok

Member

jim.bond.9862

Renowned Member

tredok

Member

jim.bond.9862

Renowned Member

tredok

Member

Attachments

jim.bond.9862

Renowned Member

tredok

Member

tredok

Member