USB-C 2.5gbe ethernet dongle install

[ace308]

I am upgrading my 1gb on board ethernet connection with a Cable Matters USB-C 2.5gb ethernet dongle with Realtek drivers. My network router uses a 1gb eth connection to a 2.5gb switch, which connects to all my devices together (PC, NAS Server, Synology etc)

Plugging in the USB-C device, PVE recognizes it under the network settings and the drivers seems to be correct.

I have added another 'linux bridge - vmbr1' with the bridge port of the USB device: enxf44dad01f3da and given it an IP address 192.168.0.100/24.



My question is, how do I update my PVE network settings to log into Proxmox with the 2.5gb connection and not the onboard ethernet connection. I must be missing one more step.

When I try to connect to connect via the dongle on either 192.168.0.99:8006 or 192.168.0.100:8006, there is no connection "this site cannot be reached". I verified this on the router, it does not see that computer/connection. I had a static ip set and even removed that to see if it did anything, and even bypassed the 2.5gb switch to go directly to the router but still no connection via the dongle. I noticed using 'ethtool enxf44dad01f3da', the link detected says 'no'.

I assume I need to remove the gateway on vmbr0 and add it to vmbr1, but because I use the GUI, I must be connected in order to access the network settings or shell, it gives me an error when I try to add it via the GUI because you can only have 1 gateway per node. I was able to add the gateway manually via the command line but then it has 2 gateways. I assume I need to plug in a keyboard and monitor and edit /etc/network/interfaces via the command line.

Is there a way to keep both eth ports active in case something happens to the dongle, I can log back in with the onboard eth connection without having to connect?

auto lo
iface lo inet loopback

iface enp7s0 inet manual

iface enxf44dad01f3da inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.0.99/24
        gateway 192.168.0.1
        bridge-ports enp7s0
        bridge-stp off
        bridge-fd 0

iface wlp6s0 inet manual

auto vmbr1
iface vmbr1 inet static
        address 192.168.0.100/24
        bridge-ports enxf44dad01f3da
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*

root@pve:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether f0:2f:74:b0:a6:16 brd ff:ff:ff:ff:ff:ff
4: wlp6s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b0:a4:60:5d:9d:d2 brd ff:ff:ff:ff:ff:ff
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether f0:2f:74:b0:a6:16 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.99/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::f22f:74ff:feb0:a616/64 scope link
       valid_lft forever preferred_lft forever
6: vmbr1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether f4:4d:ad:01:f3:da brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 scope global vmbr1
       valid_lft forever preferred_lft forever
    inet6 fe80::f64d:adff:fe01:f3da/64 scope link
       valid_lft forever preferred_lft forever
8: enxf44dad01f3da: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether f4:4d:ad:01:f3:da brd ff:ff:ff:ff:ff:ff

root@pve:~# ethtool enxf44dad01f3da
Settings for enxf44dad01f3da:
        Supported ports: [ TP    MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
                                2500baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
                                2500baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
                                             2500baseT/Full
        Link partner advertised pause frame use: Symmetric
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Auto-negotiation: on
        Port: MII
        PHYAD: 32
        Transceiver: internal
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00007fff (32767)
                               drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol
        Link detected: no

 

[neoplayer]

Hi

1. Eliminate vmbr1
2. Create a Linux Bond and add both enxf44dad01f3da and enp7s0 as slaves. Decide and select which mode is better for your use case. *Don't apply the configuration yet.
3. Change the vmbr0 bridge port to the bond name.
4. Apply configuration.
* It will help to read the documentation about this subject Linux Bond

Small example of what it should look like:

iface bond0 inet manual
    bond-slaves enp7s0 enxf44dad01f3da
    bond-miimon 100
    bond-mode balance-alb
  
auto vmbr0
iface vmbr0 inet static
    address 192.168.0.99/24
    gateway 192.168.0.1
    bridge-ports bond0
    bridge-stp off
    bridge-fd 0

Hope it helps!

 

[Moya]

Thank you for these instructions, I had a similar use case. Not the best at networking either so this was very helpful.
With this config will this enable Port Truncate like on my QNAP NAS? I have 2 x 2.5 gbps ports combined with the same protocol to 5 gbps is this the same principle for the above 'bond'? Many thanks

 

[neoplayer]

I'm not sure I understand your question corretly, if you are asking about vlan commucation with the rest of your network you can add this line to the bridge configuration or activate it using the PVE GUI.'''

auto vmbr0
iface vmbr0 inet static
    address 192.168.0.99/24
    gateway 192.168.0.1
    bridge-ports bond0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

 

[Moya]

Sorry I meant if it was possible to use 2 usb adapters of 2.5gbps and us port trunking to get to 5gbps via balance-alb?

 

[neoplayer]

Sorry I meant if it was possible to use 2 usb adapters of 2.5gbps and us port trunking to get to 5gbps via balance-alb?

Not really, remenber that one of the limitations of balance-alb is unlike LACP it can't truly aggregate bandwidth. Each connection is assigned to a single interface and won't utilize the combined bandwidth of all interfaces, but it could increase the performance of your setup if it uses concurrent connections regularly.

 

[mow]

Try balance-rr.
With 3 1G interfaces, I got 943MBit with balance-alb and 1,76GBit with balance-rr in iperf.
balance-alb distributes the routes by source&target IP, so connections to the same host will always go through the same interface as already established connections, so -alb is only helpful for concurrent connections from/to different IPs, but not when you want more bandwidth between e.g. pve and pbs.

 

[neoplayer]

Try balance-rr.
With 3 1G interfaces, I got 943MBit with balance-alb and 1,76GBit with balance-rr in iperf.
balance-alb distributes the routes by source&target IP, so connections to the same host will always go through the same interface as already established connections, so -alb is only helpful for concurrent connections from/to different IPs, but not when you want more bandwidth between e.g. pve and pbs.

The common problem with the setups that uses balance-rr is that it causes packets belonging to the same TCP connection to be sent over different interfaces. This disrupts the expected order of packets, leading to retransmissions and decreased performance for protocols like TCP that rely on in-order delivery.

 

[mow]

The common problem with the setups that uses balance-rr is that it causes packets belonging to the same TCP connection to be sent over different interfaces. This disrupts the expected order of packets, leading to retransmissions and decreased performance for protocols like TCP that rely on in-order delivery.

TCP was designed with out-of-order transmission and opportunistic routing in mind. As long as you don't exceed the receive window, it'll be OK.