Configure secondary USB to ethernet NIC for use in LXC container

[Jedis]

Hello,

I am trying to set up monitoring with Wireshark in a Proxmox LXC container running Rocky Linux 9. Some notes on my config are below:

1) I am using a Ubiquiti USW-Lite-16-PoE switch.
2) Physical port mapping on switch: My PC is plugged into port 1, Proxmox's onboard NIC is plugged into port 4, and the USB to ethernet NIC is plugged into port 5.
3) Port 5 is set to mirror mode and to mirror my PC on switch port 1.
4) The USB to ethernet NIC is detected in Proxmox and shows up under System > Network > enx0061c834a9ac

/etc/network/interfaces:

auto enx0061c834a9ac
iface enx0061c834a9ac inet manual

5) I then set up a Linux Bridge:

Name: vmbr9
Autostart: yes
VLAN aware: yes
Bridge ports: enx0061c834a9ac

/etc/network/interfaces:

auto vmbr9
iface vmbr9 inet manual
    bridge-ports enx0061c834a9ac
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

6) I then added a virtual network device to the LXC container:

Name: eth1
Bridge: vmbr9
IPv4: DHCP
Firewall: no

7) I configured PROMISC on eth1 in the container. Can verify it is seeing packets. NOTE: It does NOT get an IP address when the port is set in mirror mode on the switch. This appears to be a limitation with Ubiquiti switches.

ifconfig

eth1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether <redacted>  txqueuelen 1000  (Ethernet)
        RX packets 56807  bytes 9291705 (8.8 MiB)
        RX errors 0  dropped 6  overruns 0  frame 0
        TX packets 3131  bytes 941878 (919.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

At this point, Wireshark is not seeing any packets coming across with no filters on eth1.

I decided to remove all of this Linux bridge config stuff and try to pass through the USB to ethernet NIC directly to the container:

1) Looked on the Proxmox host's output of the lsusb command:

$ lsusb
Bus 004 Device 002: ID 0b95:1790 ASIX Electronics Corp. AX88179 Gigabit Ethernet
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 002: ID 8087:0026 Intel Corp. AX201 Bluetooth
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

$ ls -lsa /dev/bus/usb/004/002
0 crw-rw-r-- 1 root root 189, 385 Apr  1 20:53 /dev/bus/usb/004/002

$ ls -lsa /dev/serial/by-id
ls: cannot access '/dev/serial/by-id': No such file or directory

2) So the NIC is on bus 4, device 2. These mappings change when the Proxmox host reboots. Looking at solutions, I found folks suggesting to use /dev/serial/by-id but unfortunately, that path does not exist - I am guessing because the USB device doesn't also register itself as a serial device. So, sticking with 004/002 just as a proof of concept for now.

3) Went to the LXC container config > Resources > Add. Entered in /dev/bus/usb/004/002 and set UID/GID to 100000 so it aligns with root in the container. Validated I now see this in the config file:

$ cat /etc/pve/lxc/101.conf
arch: amd64
cores: 4
dev0: /dev/bus/usb/004/002,gid=100000,uid=100000
features: keyctl=1,nesting=1
hostname: monitor
memory: 4096
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=redacted,ip=dhcp,type=veth
onboot: 1
ostype: centos
rootfs: local-lvm:vm-101-disk-0,size=15G
swap: 4096
unprivileged: 1
lxc.cgroup2.devices.allow: c 116:* rwm
lxc.mount.entry: /dev/snd dev/snd none bind,create=dir
lxc.idmap: u 0 100000 65536
lxc.idmap: g 0 100000 63
lxc.idmap: g 63 29 1
lxc.idmap: g 64 100064 65472

Note that the lxc.idmap , lxc.cgroup2.devices.allow, and lxc.mount.entry are for the USB sound card.

4) eth1 does not show up under ifconfig after a reboot of the container. I removed the device added through the Proxmox web UI.

5) I then tried modifying the config file directly in the same manner I did to get the sound card working a year ago. After modifications, and a reboot of the container, I see this:

$ cat /etc/pve/lxc/101.conf
arch: amd64
cores: 4
dev0: /dev/bus/usb/004/002,gid=100000,uid=100000
features: keyctl=1,nesting=1
hostname: monitor
memory: 4096
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=redacted,ip=dhcp,type=veth
onboot: 1
ostype: centos
rootfs: local-lvm:vm-101-disk-0,size=15G
swap: 4096
unprivileged: 1
lxc.cgroup2.devices.allow: c 116:* rwm
[B]lxc.cgroup2.devices.allow: c 189:* rwm[/B]
lxc.mount.entry: /dev/snd dev/snd none bind,create=dir
[B]lxc.mount.entry: /dev/bus/usb/004/002 dev/bus/usb/004/002 none bind,optional,create=file[/B]
lxc.idmap: u 0 100000 65536
lxc.idmap: g 0 100000 63
lxc.idmap: g 63 29 1
lxc.idmap: g 64 100064 65472

6) At this point, I can see the USB NIC here:

$ ls -lsa /dev/bus/usb/004/002
0 crw-rw-r-- 1 nobody nobody 189, 385 Apr  1 20:53 /dev/bus/usb/004/002

But the permissions are not set correctly: nobody:nobody. Also, eth1 does not show up under ifconfig in the container. I am guessing it's because the permissions are not correct. I am not sure how to edit the existing idmaps without messing up the sound card.

I've also tried, unsuccessfully, in trying to set up an OVS Bridge instead of a Linux Bridge. It was unsuccessful like the Linux Bridge attempt.

Any tips or help on how to get this USB NIC functional would be appreciated!

My old Netgear router allowed the Proxmox on-board ethernet NIC to still send/receive traffic and participate in the network when the port was set to mirror mode. However, the Ubiquiti switch sets the port to listen-only and does not get an IP address. My workaround to that was to add the USB NIC and set that one to mirror mode so that the on-board NIC for Proxmox was unchanged.

Assuming all of this gets resolved, there's also still the issue of the USB bus and device numbers changing on a reboot. Saw some folks try to work around that with bash scripts, but wondering if there's a more elegant way to that bit.

Thanks so much for any leads and help!

 

[Jedis]

Got this working with ChatGPT. Also resolved the issue of the USB device ID changing. I'm passing through all USB devices. In this case, it's only the sound card and the USB to ethernet adapter plugged into USB. YMMV if you have other devices. It survived a reboot of the Proxmox host and container.

I reset everything back to base - removed the changes in the container and in the Proxmox host's LXC container config file. Removed the Linux Bridge.

Add the following to the LXC container config:

lxc.cgroup2.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/bus/usb dev/bus/usb none bind,optional,create=dir
lxc.net.1.type: phys
lxc.net.1.link: enx0061c834a9ac
lxc.net.1.flags: up

The lxc.net.1. config appears to be the missing piece to getting the adapter to show up in the LXC container!

$ ifconfig
enx0061c834a9ac: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether 00:24:9b:0d:d5:f4  txqueuelen 1000  (Ethernet)
        RX packets 52797  bytes 18040621 (17.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 437  bytes 60668 (59.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The permissions seem to still show as nobody:nobody in the container, but it is working! Not sure how important that part is to fix.

$ ls -lsa /dev/bus/usb/004/002 
0 crw-rw-r-- 1 nobody nobody 189, 385 Apr  3 12:22 /dev/bus/usb/004/002