Willing to upgrade - recommended hardware?

M

Marc Finns

New Member
May 31, 2024
16
3
3
Hi there, I am one of those migrating from free ESXI to Proxmox, and of course I have a question!
I had a fairly complex VLAN setup on a node running for ages on a NUC8i7BEH. I managed to fully recreate it and migrate all my VMs to the latest Proxmox.
However, the system is unusable in practice. Under heavy network traffic, I get the "netdev watchdog transmit queue 0 timed out" error and the node stalls or become unreachable for a while. It does not even survive an internet SpeedTest!
I was under the impression this is an issue related to the Intel i219-V NIC and I am willing to upgrade to a different hardware.
However, reading the forum and other internet sources, it appears as the issue is a broader one. Clearly, I don't want to buy a random gear and end up with the same issue.
My question is whether there is a hardware setup that does not have this issue.
Thanks for your help.
 

Attachments

  • 1.jpeg
    1.jpeg
    743.9 KB · Views: 18
What's your budget, and do you care about keeping things "low power" (electricity wise) or not?

Because if you want "absolutely guaranteed to work, and work well" and don't care about the power draw, then it's hard to go wrong with ex-enterprise gear (in tower form factor, not the noisy rack stuff).

Random example: https://www.ebay.com/itm/335417610998, though there are heaps of things like it. :)
 
Looking around online, one of the suggested workarounds to fix that problem is turning off TCP Segmentation Offloading:

Bash: 
# ethtool -K eno1 tso off

That'll only last until the next boot, but maybe try it out and see if it works (at all) first?
 
Last edited:
justinclift said:
What's your budget, and do you care about keeping things "low power" (electricity wise) or not?

Because if you want "absolutely guaranteed to work, and work well" and don't care about the power draw, then it's hard to go wrong with ex-enterprise gear (in tower form factor, not the noisy rack stuff).

Random example: https://www.ebay.com/itm/335417610998, though there are heaps of things like it. :)
Click to expand...
Thanks for taking the time. In fact I care about energy, this thing is on 24/7. I would prefer something recent and low power. NUC-style budget I guess. There is plenty of miniPCs out there… the tricky bit is the NIC I guess…
 
justinclift said:
Looking around online, one of the suggested workarounds to fix that problem is turning off TCP Segmentation Offloading:

Bash: 
# ethtool -K eno1 tso off

That'll only last until the next boot, but maybe try it out and see if it works (at all) first?
Click to expand...
Thanks for the tip. I did try but it only mitigates the issue and it impacts performance…
 
  • Like
Reactions: justinclift
Marc Finns said:
the tricky bit is the NIC I guess
Click to expand...
Ahhh. Do any of the NUC type of things you'd consider have a PCIe interface you can plug a network card into?

There are quite a few options for network cards if you're ok to look at Ebay. Random examples:
Anyway, there's a bunch of options if a PCIe card can be worked in. :)
 
On a different approach, maybe just grab one of the more decent USB to network adapters that are around instead? Should be much cheaper, and also pretty easy to find.

Some of those are pretty reliable, if you do your research. :)
 
justinclift said:
Ahhh. Do any of the NUC type of things you'd consider have a PCIe interface you can plug a network card into?

There are quite a few options for network cards if you're ok to look at Ebay. Random examples:
Anyway, there's a bunch of options if a PCIe card can be worked in. :)
Click to expand...
There are indeed with a pcie slot. Is there a clear understanding of which chips don’t have this issue? I see that also the Realtek have problems. Even if I go for a swappable nic, it would be annoying to try blindly…
 
If it helps, Mellanox cards of the ConnectX-3 and Connect-X 4 series are very widely used (both in homelabs and in industry), and don't tend to have weird issues like what you hit.
 
  • Like
Reactions: Marc Finns
Discovered something potentially relevant with my recommendation of Mellanox ConnectX-3 series cards today.

It looks like they only have support for vlans 0 to 126, and trying to use higher vlan numbers doesn't work with that specific series.

Your initial post mentions you have a complex vlan setup, so that might be an actual problem for your use case.
 
justinclift said:
Discovered something potentially relevant with my recommendation of Mellanox ConnectX-3 series cards today.

It looks like they only have support for vlans 0 to 126, and trying to use higher vlan numbers doesn't work with that specific series.

Your initial post mentions you have a complex vlan setup, so that might be an actual problem for your use case.
Click to expand...
Thanks. That would be sufficient I guess. It is complex, but still a home setup…
 
justinclift said:
On a different approach, maybe just grab one of the more decent USB to network adapters that are around instead? Should be much cheaper, and also pretty easy to find.

Some of those are pretty reliable, if you do your research. :)
Click to expand...
That’s indeed my backup solution. Unfortunately I need WOL support, perhaps I could use the built in NIC just for wol…
You mentioned that some are reliable, any specific name/chipset? I was reluctant to go USB specifically because of reliability. Thanks
 
Last edited:
Marc Finns said:
You mentioned that some are reliable, any specific name/chipset?
Click to expand...
I'm not the right person for recommending specific models any more. I used to use some with a homelab setup when I was living in the UK, but that was prior to Brexit :mad: and I've subsequently returned to Australia and am using ex-enterprise server gear now.

I think the ones I was using were some Axxios based chipset back then, but it was several years ago now and I don't remember the details.
 
Last edited:
  • Like
Reactions: Marc Finns
UPDATE: I bought a Startech usb32000spt dual USB dongle, based on the ASIX AX88179A Chipset.
It does work, even with LAG.
However, I am experiencing random kernel freezes. Not sure if this is related to the dongle or to something else.
Nothing in the logs, it just stops. Not sure what to do to investigate... I might be forced to revert to ESXI!
 
Gah. That sucks.

Hmmm, are you using kernel 6.8.x or 6.5.x at the moment? If it's 6.8.x, then maybe try dropping back to 6.5.x and see if the stability improves? (instructions here under "Kernel 6.8" heading)

Lots of people have been having weird issues after the recent update to kernel 6.8, so maybe it's just that. Hopefully. :)
 
Last edited:
justinclift said:
Gah. That sucks.

Hmmm, are you using kernel 6.8.x or 6.5.x at the moment? If it's 6.8.x, then maybe try dropping back to 6.5.x and see if the stability improves? (instructions here under "Kernel 6.8" heading)

Lots of people have been having weird issues after the recent update to kernel 6.8, so maybe it's just that. Hopefully. :)
Click to expand...
Thanks for the hint, in fact these are the exact symptoms of my issue.
However, the instructions you kindly posted are to avoid the kernel update prior to a proxmox upgrade.
My case is a bit different, as I started from a fresh install of proxmox 8.2.2. I tried to downgrade the kernel to 6.5.x but no luck.
The system would not boot, with the following message:

Loading Linux 6.5.11-8-pve ... error: bad shim signature.
Loading initial ramdisk
.. .
error: you need to load the kernel first.
Press any key to continue....

Is the downgrade possible? What am I doing wrong?
Thanks + apologies if the question turns out to be stupid...!
 
Marc Finns said:
Thanks for the hint, in fact these are the exact symptoms of my issue.
However, the instructions you kindly posted are to avoid the kernel update prior to a proxmox upgrade.
My case is a bit different, as I started from a fresh install of proxmox 8.2.2. I tried to downgrade the kernel to 6.5.x but no luck.
The system would not boot, with the following message:

Loading Linux 6.5.11-8-pve ... error: bad shim signature.
Loading initial ramdisk
.. .
error: you need to load the kernel first.
Press any key to continue....

Is the downgrade possible? What am I doing wrong?
Thanks + apologies if the question turns out to be stupid...!
Click to expand...
UPDATE: I bypassed the issue by disabling secure boot, but it would be interesting to understand if there is a better way...
 
  • Like
Reactions: justinclift
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back