7.2 update Aquantia NIC not working anymore

T

Thomas Stone

Renowned Member
Proxmox Subscriber
Mar 8, 2016
66
5
73
Hello:

I did this update a couple days ago, which I guess is Proxmox 7.1 to 7.2:

Code: 
Start-Date: 2022-05-05  09:20:50
Commandline: apt dist-upgrade
Install: pve-kernel-5.15.30-2-pve:amd64 (5.15.30-3, automatic), pve-kernel-5.15:amd64 (7.2-1, automatic), libvirglrenderer1:amd64 (0.8.2-5, automatic)
Upgrade: pve-docs:amd64 (7.1-2, 7.2-2), syncthing:amd64 (1.19.2, 1.20.1), proxmox-widget-toolkit:amd64 (3.4-7, 3.4-10), libpve-rs-perl:amd64 (0.6.0, 0.6.1), pve-firmware:amd64 (3.3-6, 3.4-1), pve-qemu-kvm:amd64 (6.1.1-2, 6.2.0-5), libproxmox-acme-perl:amd64 (1.4.1, 1.4.2), libpve-cluster-api-perl:amd64 (7.1-3, 7.2-1), pve-ha-manager:amd64 (3.3-3, 3.3-4), lxcfs:amd64 (4.0.11-pve1, 4.0.12-pve1), libpve-storage-perl:amd64 (7.1-1, 7.2-2), libpve-guest-common-perl:amd64 (4.1-1, 4.1-2), pve-cluster:amd64 (7.1-3, 7.2-1), proxmox-ve:amd64 (7.1-1, 7.2-1), lxc-pve:amd64 (4.0.11-1, 4.0.12-1), novnc-pve:amd64 (1.3.0-2, 1.3.0-3), proxmox-backup-file-restore:amd64 (2.1.5-1, 2.1.8-1), qemu-server:amd64 (7.1-4, 7.2-2), libpve-access-control:amd64 (7.1-7, 7.1-8), pve-container:amd64 (4.1-4, 4.2-1), libproxmox-acme-plugins:amd64 (1.4.1, 1.4.2), pve-i18n:amd64 (2.6-2, 2.7-1), proxmox-backup-client:amd64 (2.1.5-1, 2.1.8-1), smartmontools:amd64 (7.2-pve2, 7.2-pve3), pve-manager:amd64 (7.1-12, 7.2-3), libpve-common-perl:amd64 (7.1-5, 7.1-6), firefox-esr:amd64 (91.8.0esr-1~deb11u1, 91.9.0esr-1~deb11u1), pve-kernel-helper:amd64 (7.1-14, 7.2-2), libpve-cluster-perl:amd64 (7.1-3, 7.2-1)
End-Date: 2022-05-05  09:22:35

and after a reboot my Aquantia NIC no longer works. The bridge is still showing as up. It is set to a static IP, but it can't talk to anyone! It's a 10G NIC but connected to a 1G switch. Is this a known issue? Is there a known solution?

lspci shows it:

Code: 
2d:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)

It's on the ASRock motherboard of my whitebox AMD Ryzen 2700x based system.

Code: 
uname -a
Linux pvea 5.15.30-2-pve #1 SMP PVE 5.15.30-3 (Fri, 22 Apr 2022 18:08:27 +0200) x86_64 GNU/Linux

What else can I tell you? Thanks!
 
ah, I see this has been mentioned by others, and has been patched: kernel.org

Would love to see this merged into Proxmox kernel asap!

Here's where to see the error:

Code: 
# cat syslog.1 | grep aquantia
May  5 09:24:47 pvea kernel: [   13.325232] UBSAN: array-index-out-of-bounds in drivers/net/ethernet/aquantia/atlantic/aq_nic.c:484:48
May  5 09:24:47 pvea kernel: [   13.398137] UBSAN: array-index-out-of-bounds in drivers/net/ethernet/aquantia/atlantic/aq_nic.c:515:49
May  5 09:42:24 pvea kernel: [ 1073.587728] UBSAN: array-index-out-of-bounds in drivers/net/ethernet/aquantia/atlantic/aq_nic.c:1262:48
May  5 11:13:59 pvea kernel: [   13.570643] UBSAN: array-index-out-of-bounds in drivers/net/ethernet/aquantia/atlantic/aq_nic.c:484:48
May  5 11:13:59 pvea kernel: [   13.630094] UBSAN: array-index-out-of-bounds in drivers/net/ethernet/aquantia/atlantic/aq_nic.c:515:49
 
Hmm. Some developments: Perhaps for the first time ever this machine had locked up on me. Power off then reboot, I booted into the older 5.13 kernel that was offered in the boot menu, and still the aquantia NIC isn't working. I guess I'm confused if the driver is in the kernel or if now I'm still using the newest driver (broken) but the older kernel and that's why it's not working.

[edit]

I guess I can answer my own question, if the driver is at drivers/net/ethernet/aquantia/atlantic/aq_nic.c then clearly it is not in the kernel.
 
Last edited:
I'm now running the testing kernel 5.15.35.3 (I think!):

Code: 
# uname -a
Linux pvea 5.15.35-1-pve #1 SMP PVE 5.15.35-3 (Wed, 11 May 2022 07:57:51 +0200) x86_64 GNU/Linux

and the NIC is not magically working again. However, syslog is no longer showing array-index-out-of-bounds anymore! So, progress. Maybe there's something I need to do to kickstart it again.

any clues here:

Code: 
# ip link | grep vmbr1
4: enp45s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP mode DEFAULT group default qlen 1000
8: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
10: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr1 state UNKNOWN mode DEFAULT group default qlen 1000

Code: 
# ip addr | grep vmbr1
4: enp45s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP group default qlen 1000
8: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 192.168.1.3/24 scope global vmbr1
10: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr1 state UNKNOWN group default qlen 1000
 
Hello again.. I'm now on 5.15.39-1-pve #1 SMP PVE 5.15.39-1 (Wed, 22 Jun 2022 17:22:00 +0200) x86_64 GNU/Linux kernel and still can't ping any IPs on the same subnet (or any) through this NIC. Should it work? Can I help with getting it fixed by posting any logs?

I'm about ready to throw some hardware at the problem, but I'd rather use the on-board NIC if possible since it did work before on Proxmox! Thanks.
 
Code: 
# ip link
2: enp42s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 70:85:c2:82:52:ad brd ff:ff:ff:ff:ff:ff
3: enp47s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master vmbr2 state UP mode DEFAULT group default qlen 1000
    link/ether 00:1b:21:bd:96:10 brd ff:ff:ff:ff:ff:ff
4: enp45s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP mode DEFAULT group default qlen 1000
    link/ether 70:85:c2:82:52:af brd ff:ff:ff:ff:ff:ff
5: enp47s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:1b:21:bd:96:12 brd ff:ff:ff:ff:ff:ff
6: wlp39s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 68:ec:c5:bf:1a:fd brd ff:ff:ff:ff:ff:ff
7: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 70:85:c2:82:52:ad brd ff:ff:ff:ff:ff:ff
8: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 70:85:c2:82:52:af brd ff:ff:ff:ff:ff:ff
9: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:1b:21:bd:96:10 brd ff:ff:ff:ff:ff:ff

Code: 
# ip addr
2: enp42s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether 70:85:c2:82:52:ad brd ff:ff:ff:ff:ff:ff
3: enp47s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master vmbr2 state UP group default qlen 1000
    link/ether 00:1b:21:bd:96:10 brd ff:ff:ff:ff:ff:ff
4: enp45s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP group default qlen 1000
    link/ether 70:85:c2:82:52:af brd ff:ff:ff:ff:ff:ff
5: enp47s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:1b:21:bd:96:12 brd ff:ff:ff:ff:ff:ff
6: wlp39s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 68:ec:c5:bf:1a:fd brd ff:ff:ff:ff:ff:ff
7: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 70:85:c2:82:52:ad brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.2/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::7285:c2ff:fe82:52ad/64 scope link
       valid_lft forever preferred_lft forever
8: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 70:85:c2:82:52:af brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.3/24 scope global vmbr1
       valid_lft forever preferred_lft forever
    inet6 fe80::7285:c2ff:fe82:52af/64 scope link
       valid_lft forever preferred_lft forever
9: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether 00:1b:21:bd:96:10 brd ff:ff:ff:ff:ff:ff
    inet 172.16.16.2/24 scope global vmbr2
       valid_lft forever preferred_lft forever
    inet6 fe80::21b:21ff:febd:9610/64 scope link
       valid_lft forever preferred_lft forever

I can ping 172.16.16.1 and 10.0.1.1 but not hosts on 192.168.1.1 (subnet). Well, I can ping itself at 192.168.1.3.

enp45s0 / vmbr1 is the aquantia.

Code: 
2d:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)

Code: 
2d:00.0 0200: 1d6a:d107 (rev 02)
    Subsystem: 1849:d107
    Kernel driver in use: atlantic
    Kernel modules: atlantic

It is on the motherboad with CPU AMD Ryzen 2700x, ASRock I think.
 
Last edited:
Thomas Stone said:
Code: 
# ip link
2: enp42s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 70:85:c2:82:52:ad brd ff:ff:ff:ff:ff:ff
3: enp47s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master vmbr2 state UP mode DEFAULT group default qlen 1000
    link/ether 00:1b:21:bd:96:10 brd ff:ff:ff:ff:ff:ff
4: enp45s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP mode DEFAULT group default qlen 1000
    link/ether 70:85:c2:82:52:af brd ff:ff:ff:ff:ff:ff
5: enp47s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:1b:21:bd:96:12 brd ff:ff:ff:ff:ff:ff
6: wlp39s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 68:ec:c5:bf:1a:fd brd ff:ff:ff:ff:ff:ff
7: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 70:85:c2:82:52:ad brd ff:ff:ff:ff:ff:ff
8: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 70:85:c2:82:52:af brd ff:ff:ff:ff:ff:ff
9: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:1b:21:bd:96:10 brd ff:ff:ff:ff:ff:ff

Code: 
# ip addr
2: enp42s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether 70:85:c2:82:52:ad brd ff:ff:ff:ff:ff:ff
3: enp47s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master vmbr2 state UP group default qlen 1000
    link/ether 00:1b:21:bd:96:10 brd ff:ff:ff:ff:ff:ff
4: enp45s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP group default qlen 1000
    link/ether 70:85:c2:82:52:af brd ff:ff:ff:ff:ff:ff
5: enp47s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:1b:21:bd:96:12 brd ff:ff:ff:ff:ff:ff
6: wlp39s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 68:ec:c5:bf:1a:fd brd ff:ff:ff:ff:ff:ff
7: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 70:85:c2:82:52:ad brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.2/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::7285:c2ff:fe82:52ad/64 scope link
       valid_lft forever preferred_lft forever
8: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 70:85:c2:82:52:af brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.3/24 scope global vmbr1
       valid_lft forever preferred_lft forever
    inet6 fe80::7285:c2ff:fe82:52af/64 scope link
       valid_lft forever preferred_lft forever
9: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP group default qlen 1000
    link/ether 00:1b:21:bd:96:10 brd ff:ff:ff:ff:ff:ff
    inet 172.16.16.2/24 scope global vmbr2
       valid_lft forever preferred_lft forever
    inet6 fe80::21b:21ff:febd:9610/64 scope link
       valid_lft forever preferred_lft forever

I can ping 172.16.16.1 and 10.0.1.1 but not hosts on 192.168.1.1 (subnet). Well, I can ping itself at 192.168.1.3.

enp45s0 / vmbr1 is the aquantia.

Code: 
2d:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)

Code: 
2d:00.0 0200: 1d6a:d107 (rev 02)
    Subsystem: 1849:d107
    Kernel driver in use: atlantic
    Kernel modules: atlantic

It is on the motherboad with CPU AMD Ryzen 2700x, ASRock I think.
Click to expand...

Hi Thomas,

I have an AMD Ryzen 5950x on a x570 motherboard (MSI). The Aquantia nic is at:
24:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02) Subsystem: Micro-Star International Co., Ltd. [MSI] AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] Kernel driver in use: atlantic Kernel modules: atlantic

I think you need to look at your /etc/network/interfaces file. Mine contains this (not the whole file):

auto enp36s0 iface enp36s0 inet manual mtu 9000 #10Gb Atlantia Nic auto vmbr0 iface vmbr0 inet manual bridge-ports enp36s0 enp43s0 bridge-stp off bridge-fd 0 bridge-vlan-aware yes bridge-vids 2-4094 mtu 9000 # bridge 10Gb and 1Gb Nics auto vlan100 iface vlan100 inet static address 192.168.101.32/24 gateway 192.168.101.254 mtu 9000 vlan-raw-device vmbr0

So my management interface is on a separate vlan (100) but you could just give your bridge an ip address. I wouldn't edit the file directly; it can be done through the web interface but if you post yours, I can take a look.
 
The only thing I noticed is that there was no MTU listed for vmbr1 / enp45s0 so I updated that. I also then did ifdown ifup in the terminal and tried changing something in the GUI and changing it back, applying... Still not working. Anything look wrong to you?

Code: 
auto lo
iface lo inet loopback

iface enp42s0 inet manual
    mtu 1500

iface enp45s0 inet manual
    mtu 1500

iface enp47s0f0 inet manual
    mtu 9000

iface enp47s0f1 inet manual
    mtu 9000

auto vmbr0
iface vmbr0 inet static
    address 10.0.1.2/24
    bridge-ports enp42s0
    bridge-stp off
    bridge-fd 0
    mtu 1500
#Home Lan Bridge

auto vmbr1
iface vmbr1 inet static
    address 192.168.1.3/24
    bridge-ports enp45s0
    bridge-stp off
    bridge-fd 0
    mtu 1500
#10GAq WifiLan

auto vmbr2
iface vmbr2 inet static
    address 172.16.16.2/24
    gateway 172.16.16.1
    bridge-ports enp47s0f0
    bridge-stp off
    bridge-fd 0
    mtu 9000
#SFPTop StuLan

iface vmbr3 inet static
    address 172.16.15.2/24
    bridge-ports enp47s0f1
    bridge-stp off
    bridge-fd 0
    mtu 9000
#SFPBottom
 
Thomas Stone said:
The only thing I noticed is that there was no MTU listed for vmbr1 / enp45s0 so I updated that. I also then did ifdown ifup in the terminal and tried changing something in the GUI and changing it back, applying... Still not working. Anything look wrong to you?

Code: 
auto lo
iface lo inet loopback

iface enp42s0 inet manual
    mtu 1500

iface enp45s0 inet manual
    mtu 1500

iface enp47s0f0 inet manual
    mtu 9000

iface enp47s0f1 inet manual
    mtu 9000

auto vmbr0
iface vmbr0 inet static
    address 10.0.1.2/24
    bridge-ports enp42s0
    bridge-stp off
    bridge-fd 0
    mtu 1500
#Home Lan Bridge

auto vmbr1
iface vmbr1 inet static
    address 192.168.1.3/24
    bridge-ports enp45s0
    bridge-stp off
    bridge-fd 0
    mtu 1500
#10GAq WifiLan

auto vmbr2
iface vmbr2 inet static
    address 172.16.16.2/24
    gateway 172.16.16.1
    bridge-ports enp47s0f0
    bridge-stp off
    bridge-fd 0
    mtu 9000
#SFPTop StuLan

iface vmbr3 inet static
    address 172.16.15.2/24
    bridge-ports enp47s0f1
    bridge-stp off
    bridge-fd 0
    mtu 9000
#SFPBottom
Click to expand...
I think for each nic you need an 'auto' entry. For example:

Code: 
auto enp42s0
iface enp42s0 inet manual
    mtu 1500

This is the 'autostart' option. You have it on your bridges but not on your nics. Try that.
 
Thanks Jonathan... the other two bridges are working fine without that setting. I tried it just in case, but it made no difference. Maybe I'll change the cable! lol (there are activity lights on the back of the motherboard for the nic).
 
Argh! Checking the cable was a good idea! It was not the cable I thought it was, and then I could see no activity, though there was a link to the switch. But the line through the wall to another switch was not making full contact anymore so that was the problem. I do maintain that at some point a software issue broke this, but if that's true, then by the time it did get fixed in software, I had a hardware problem too.

Thanks Jonathan, sorry to waste your time.
 
Thomas Stone said:
Argh! Checking the cable was a good idea! It was not the cable I thought it was, and then I could see no activity, though there was a link to the switch. But the line through the wall to another switch was not making full contact anymore so that was the problem. I do maintain that at some point a software issue broke this, but if that's true, then by the time it did get fixed in software, I had a hardware problem too.

Thanks Jonathan, sorry to waste your time.
Click to expand...
HI Thomas,

I was going to suggest checking the cables if that didn't work but I am glad you got it sorted!
 
You must log in or register to reply here.
Top Bottom