[SOLVED] passthrough of usb-controller causes ethernet to stop

Discussion in 'Proxmox VE: Installation and configuration' started by liebero3, Sep 15, 2018.

  1. liebero3

    liebero3 New Member

    Joined:
    Apr 27, 2017
    Messages:
    16
    Likes Received:
    0
    Hello,

    I'm a novice in virtualisation and don't know any further with a problem I'm having.

    I have a working windows 10 vm with gpu passthrough. But if I pass through a second pci, no matter if I choose the onboard asmedia or a pci usb card (both of them have their own iommu group) the ethernet of my host stops working right in the moment I start the vm.

    Does anyone have any idea, what could cause this behaviour? I'm willing to post every log you like, but since I don't have any idea, what the problem could be, I don't know what log could help you.

    My cpu is a ryzen 1700 and my mainboard is the gigabyte x370 gaming 5.

    Thanks in advance,
    Till
     
  2. efeu

    efeu Member
    Proxmox VE Subscriber

    Joined:
    Nov 6, 2015
    Messages:
    74
    Likes Received:
    6
    I have the exact same setup and I use the CPU internal USB controller for passthrough.
     
  3. liebero3

    liebero3 New Member

    Joined:
    Apr 27, 2017
    Messages:
    16
    Likes Received:
    0
    I am pretty sure it is a fault on my side. But I have no clue where.

    The moment I start the vm with the usb controller passed through the ethernet loses connection. Its still working in a way, its sending whois requests to the router. But the moment the vm starts the lights on the ethernet port go out and restarting the network service does not help, just a restart brings back network connection.

    Do you use the m2 port? I have a ssd there and the gpu i am passing through is in the x8 slot, the second from the bottom. Maybe its a problem of lane capacity, because ethernet, 2nd gpu and nvme are connected? I read in the manual, that they are all connected
     
  4. liebero3

    liebero3 New Member

    Joined:
    Apr 27, 2017
    Messages:
    16
    Likes Received:
    0
    Sry for doubleposting, but I found time to gather some informations, which might help helping me.

    Here is my lspci output

    Code:
    00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1450
    00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Device 1451
    00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
    00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453
    00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453
    00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
    00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
    00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453
    00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453
    00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
    00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
    00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454
    00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
    00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454
    00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59)
    00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
    00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1460
    00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1461
    00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1462
    00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1463
    00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1464
    00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1465
    00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1466
    00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1467
    01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd Device a804
    02:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43b9 (rev 02)
    02:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43b5 (rev 02)
    02:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b0 (rev 02)
    03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02)
    03:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02)
    03:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02)
    03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02)
    04:00.0 USB controller: ASMedia Technology Inc. Device 1343
    05:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)
    06:00.0 Ethernet controller: Qualcomm Atheros Device e0b1 (rev 10)
    08:00.0 VGA compatible controller: NVIDIA Corporation G86 [GeForce 8400 GS] (rev a1)
    09:00.0 VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 710B] (rev a1)
    09:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1)
    0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a
    0a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Device 1456
    0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 145c
    0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455
    0b:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
    0b:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Device 1457
    
    This is the last seconds till I start the vm from tcpdump:

    Code:
    13:46:17.281762 IP 185.54.150.28.https > pve.fritz.box.45676: Flags [R.], seq 232938313, ack 2584655648, win 0, length 0
    13:46:17.281896 IP pve.fritz.box.59643 > fritz.box.domain: 29021+ PTR? 28.150.54.185.in-addr.arpa. (44)
    13:46:17.427666 IP fritz.box.domain > pve.fritz.box.59643: 29021 NXDomain 0/1/0 (107)
    13:46:18.365849 c0:25:06:ec:35:00 (oui Unknown) > 00:b0:52:00:00:01 (oui Unknown), ethertype Unknown (0x88e1), length 60:
            0x0000:  0068 a000 b052 2c0d 1277 0000 0000 0000  .h...R,..w......
            0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:18.365859 c0:25:06:ec:35:00 (oui Unknown) > Broadcast, ethertype Unknown (0x88e1), length 60:
            0x0000:  0000 a000 b052 2c0d 1277 0000 0000 0000  .....R,..w......
            0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:18.365861 c0:25:06:ec:35:00 (oui Unknown) > Broadcast, ethertype Unknown (0x8912), length 60:
            0x0000:  0170 a000 0000 1f84 0da3 97a2 5553 bef1  .p..........US..
            0x0010:  fcf9 796b 5214 13e9 e200 0000 0000 0000  ..ykR...........
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:20.367953 c0:25:06:ec:35:00 (oui Unknown) > 00:b0:52:00:00:01 (oui Unknown), ethertype Unknown (0x88e1), length 60:
            0x0000:  0068 a000 b052 2c0d 1277 0000 0000 0000  .h...R,..w......
            0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:20.367965 c0:25:06:ec:35:00 (oui Unknown) > Broadcast, ethertype Unknown (0x88e1), length 60:
            0x0000:  0000 a000 b052 2c0d 1277 0000 0000 0000  .....R,..w......
            0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:20.367968 c0:25:06:ec:35:00 (oui Unknown) > Broadcast, ethertype Unknown (0x8912), length 60:
            0x0000:  0170 a000 0000 1f84 0da3 97a2 5553 bef1  .p..........US..
            0x0010:  fcf9 796b 5214 13e9 e200 0000 0000 0000  ..ykR...........
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:22.369032 c0:25:06:ec:35:00 (oui Unknown) > 00:b0:52:00:00:01 (oui Unknown), ethertype Unknown (0x88e1), length 60:
            0x0000:  0068 a000 b052 2c0d 1277 0000 0000 0000  .h...R,..w......
            0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:22.369043 c0:25:06:ec:35:00 (oui Unknown) > Broadcast, ethertype Unknown (0x88e1), length 60:
            0x0000:  0000 a000 b052 2c0d 1277 0000 0000 0000  .....R,..w......
            0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:22.369045 c0:25:06:ec:35:00 (oui Unknown) > Broadcast, ethertype Unknown (0x8912), length 60:
            0x0000:  0170 a000 0000 1f84 0da3 97a2 5553 bef1  .p..........US..
            0x0010:  fcf9 796b 5214 13e9 e200 0000 0000 0000  ..ykR...........
            0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
    13:46:23.178611 IP dsp.adfarm1.adition.com.https > pve.fritz.box.41906: Flags [P.], seq 57193017:57193048, ack 3850339201, win 245, length 31
    13:46:23.178642 IP pve.fritz.box.41906 > dsp.adfarm1.adition.com.https: Flags [.], ack 31, win 342, length 0
    13:46:23.178646 IP dsp.adfarm1.adition.com.https > pve.fritz.box.41906: Flags [F.], seq 31, ack 1, win 245, length 0
    13:46:23.178751 IP pve.fritz.box.48952 > fritz.box.domain: 31213+ PTR? 118.159.114.85.in-addr.arpa. (45)
    13:46:23.192451 IP fritz.box.domain > pve.fritz.box.48952: 31213 1/0/0 PTR dsp.adfarm1.adition.com. (82)
    13:46:23.221877 IP pve.fritz.box.41906 > dsp.adfarm1.adition.com.https: Flags [.], ack 32, win 342, length 0
    13:46:23.395706 IP pve.fritz.box.41906 > dsp.adfarm1.adition.com.https: Flags [R.], seq 1, ack 32, win 342, length 0
    tcpdump: pcap_loop: The interface went down
    663 packets captured
    5633 packets received by filter
    4970 packets dropped by kernel
    
    And here is my configuration file of the windows vm:

    Code:
    bios: ovmf
    bootdisk: scsi0
    cores: 8
    efidisk0: local-lvm:vm-100-disk-2,size=128K
    hostpci0: 09:00,x-vga=on,pcie=1
    hostpci1: 04:00,pcie=1
    ide0: local:iso/virtio-win-0.1.160.iso,media=cdrom,size=315276K
    ide2: local:iso/Win10_1803_German_x64.iso,media=cdrom
    machine: q35
    memory: 8192
    net0: virtio=42:69:13:3C:BA:24,bridge=vmbr0
    numa: 0
    ostype: win10
    scsi0: local-lvm:vm-100-disk-1,cache=writethrough,size=64G
    scsihw: virtio-scsi-pci
    smbios1: uuid=bad0258f-81ba-4fe1-900d-372707a286cc
    sockets: 1
    
    Anyone any ideas what could be wrong?
     
  5. dcsapak

    dcsapak Proxmox Staff Member
    Staff Member

    Joined:
    Feb 1, 2016
    Messages:
    2,792
    Likes Received:
    251
    can you list your iommu groups? e.g. with this script
    Code:
    #!/bin/bash
    shopt -s nullglob
    for d in /sys/kernel/iommu_groups/*/devices/*; do
       n=${d#*/iommu_groups/*}; n=${n%%/*}
       printf 'IOMMU Group %s ' "$n"
       lspci -nns "${d##*/}"
    done;
    
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. liebero3

    liebero3 New Member

    Joined:
    Apr 27, 2017
    Messages:
    16
    Likes Received:
    0
    You were right, it was that easy. I mentioned, that I am a novice? :)

    I actually thought, that the Groups were the part before the colon in the lspci output.

    I changed to the other usb-controller in Group 20 and everything works perfect!

    Thanks a lot!


    Code:
    IOMMU Group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1452]
    IOMMU Group 10 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1452]
    IOMMU Group 11 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1454]
    IOMMU Group 12 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 59)
    IOMMU Group 12 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
    IOMMU Group 13 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1460]
    IOMMU Group 13 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1461]
    IOMMU Group 13 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1462]
    IOMMU Group 13 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1463]
    IOMMU Group 13 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1464]
    IOMMU Group 13 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1465]
    IOMMU Group 13 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1466]
    IOMMU Group 13 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1467]
    IOMMU Group 14 01:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd Device [144d:a804]
    IOMMU Group 15 02:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b9] (rev 02)
    IOMMU Group 15 02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b5] (rev 02)
    IOMMU Group 15 02:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b0] (rev 02)
    IOMMU Group 15 03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b4] (rev 02)
    IOMMU Group 15 03:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b4] (rev 02)
    IOMMU Group 15 03:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b4] (rev 02)
    IOMMU Group 15 03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b4] (rev 02)
    IOMMU Group 15 04:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:1343]
    IOMMU Group 15 05:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
    IOMMU Group 15 06:00.0 Ethernet controller [0200]: Qualcomm Atheros Device [1969:e0b1] (rev 10)
    IOMMU Group 16 08:00.0 VGA compatible controller [0300]: NVIDIA Corporation G86 [GeForce 8400 GS] [10de:0422] (rev a1)
    IOMMU Group 17 09:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208 [GeForce GT 710B] [10de:128b] (rev a1)
    IOMMU Group 17 09:00.1 Audio device [0403]: NVIDIA Corporation GK208 HDMI/DP Audio Controller [10de:0e0f] (rev a1)
    IOMMU Group 18 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device [1022:145a]
    IOMMU Group 19 0a:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Device [1022:1456]
    IOMMU Group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1453]
    IOMMU Group 20 0a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:145c]
    IOMMU Group 21 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device [1022:1455]
    IOMMU Group 22 0b:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
    IOMMU Group 23 0b:00.3 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Device [1022:1457]
    IOMMU Group 2 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1453]
    IOMMU Group 3 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1452]
    IOMMU Group 4 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1452]
    IOMMU Group 5 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1453]
    IOMMU Group 6 00:03.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1453]
    IOMMU Group 7 00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1452]
    IOMMU Group 8 00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:1452]
    IOMMU Group 9 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1454]
    
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice