Why Not IOMMU_Groups...NO Result?

Nollimox

Member
Mar 9, 2023
268
21
18
IP addr:
root@nollicomm:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:40:a0:90 brd ff:ff:ff:ff:ff:ff
3: enp1s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:40:a0:91 brd ff:ff:ff:ff:ff:ff
4: enp1s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:40:a0:92 brd ff:ff:ff:ff:ff:ff
5: enp1s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:40:a0:93 brd ff:ff:ff:ff:ff:ff
6: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
link/ether e4:b9:7a:ec:99:53 brd ff:ff:ff:ff:ff:ff
altname enp0s31f6
7: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether e4:b9:7a:ec:99:53 brd ff:ff:ff:ff:ff:ff
inet 10.8.27.36/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::e6b9:7aff:feec:9953/64 scope link
valid_lft forever preferred_lft forever
8: vmbr1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 06:e5:46:41:22:d1 brd ff:ff:ff:ff:ff:ff
9: vmbr2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 06:f8:bc:13:37:fc brd ff:ff:ff:ff:ff:ff
root@nollicomm:~#

Yet, running this always produce no groups: why?
root@nollicomm:~# find /sys/kernel/iommu_groups/ -type l
root@nollicomm:~#

See...

GNU nano 7.2 /etc/network/interfaces
iface enp1s0f1 inet manual

iface enp1s0f2 inet manual

iface enp1s0f3 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.8.27.36/24
gateway 10.8.27.1
bridge-ports eno1
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0

auto vmbr2
iface vmbr2 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0

There's no other answer...this must be a bug...it like the virtual environment host doesn't want the virtual machine to have network interface...yet, I cannot update or upgrade but I ping google...
root@nollicomm:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=111 time=110 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=111 time=60.5 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=111 time=154 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=111 time=76.5 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=111 time=74.7 ms
64 bytes from 8.8.8.8: icmp_seq=6 ttl=111 time=72.5 ms
64 bytes from 8.8.8.8: icmp_seq=7 ttl=111 time=70.6 ms
^C
--- 8.8.8.8 ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 6012ms
rtt min/avg/max/mdev = 60.502/88.426/153.735/30.314 ms
root@nollicomm:~#

Why we never get: Kernel driver in use: vfio-pci
Kernel driver in use: pcieport

Why denied permission...root user here:
root@nollicomm:~# echo 4 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs
-bash: /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs: Permission denied
root@nollicomm:~#
 
Last edited:
Yet, running this always produce no groups: why?
root@nollicomm:~# find /sys/kernel/iommu_groups/ -type l
root@nollicomm:~#
Probably because iOMMU is not fully enabled? What is the output of cat /proc/cmdline. Is IOMMU and/or AMD-Vi and/or Intel VT-d enabled in the motherboard BIOS? Does your CPU and motherboard support it? What CPU and motherboard are you using (and does it run the latest BIOS)?
There's no other answer...this must be a bug...it like the virtual environment host doesn't want the virtual machine to have network interface...yet, I cannot update or upgrade but I ping google...
What is the output of apt update? Why all the network stuff when your question appears to be about IOMMU? Are you trying to do PCI(e) passthrough of a network controller?
Why we never get: Kernel driver in use: vfio-pci
Kernel driver in use: pcieport
Because vfio-pci does not bind to PCI(e) Bridges, it binds to PCI(e) devices below a bridge. Sometimes a 'softdep' is necessary to make sure vfio-pci loads before the actual driver.
Why denied permission...root user here:
root@nollicomm:~# echo 4 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs
-bash: /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs: Permission denied
Maybe the device does not support it or SR-IOV is not (fully) enabled.

Can you maybe explain a bit about what you are trying to achieve? Can we start with getting IOMMU enabled so we can looks at the IOMMU groups (using for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done) and possibilities for passthrough?
 
Why all the network stuff when your question appears to be about IOMMU?
Thank you for responding...the reason for all the network in associated IOMMU, is for pfSense...there are two PCIe devices that needs to passthrough. I had gotten over-board and tried to add the sound too. In this process, I screwed the platform (Dell Precision 3630MT) over thinking and in a rush. However, I have learned and to appreciate Proxmox's resiliency.

I had exported the ifreload.txt attached and was surprised it identify the computer as Dell OptiPlex 5060 MT. However, examining the document revealed the hardware is confused. It made no mention of the PCIE.

So, I'll flash the bios with the latest firmware (did this earlier prior to my rumbustious going overboard). Wished, as Mac platform has, a keyboard method to clear and flush hardware cache. After the firmware reset, I think it should work with just:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
 

Attachments

  • ifreload.txt
    10.1 KB · Views: 0

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!