One 10Gb nic better then 3x dual port 1Gb nic?

[helloha]

I was wondering if it's better to replace my 3 pcie cards with one 10Gb nic and to use virtual interfaces.

I'm mainly wondering about this because it can save some power. Every pcie card takes about 7-15 watt.

And I have 10Gb running with my bare metal Freenas boxes anyway.

Would there be any downsides in terms of performance?

I'm running a E3-1270v2 with 32GB of ram.

I mainly use it for home use. Several VM's for Pfsense, Nextcloud, Uifi controller...

thx for any feedback!
K.

 

[talos]

I would recommend using a single 10g, less power usage and a single connection can use as much bandwith this physical connection provide. In my home setup i have two Proxmox hosts and one OPNsense Firewall connected via 10g, i do a cross backup between these two hosts where 10g helps a lot.

As for connecting 10g equipment, this is my experience:

* 10BASE-T, cheap but nics/onboard PHYs can get quiete hot and draw lots of power
* SFP+ Optical, getting very hot without active room cooling
* DAC Cables, cheap, low heat and what i use for my setup

 

[helloha]

I would recommend using a single 10g, less power usage and a single connection can use as much bandwith this physical connection provide. In my home setup i have two Proxmox hosts and one OPNsense Firewall connected via 10g, i do a cross backup between these two hosts where 10g helps a lot.

As for connecting 10g equipment, this is my experience:

* 10BASE-T, cheap but nics/onboard PHYs can get quiete hot and draw lots of power
* SFP+ Optical, getting very hot without active room cooling
* DAC Cables, cheap, low heat and what i use for my setup

Thanks for the feedback, yes I’ve been running 10g sfp+ for several years now since it was cheap on the used market. Used it with a direct connnection between freenas and my workstation. Made the jump for a switch since unifi released an affordable 10g aggregation switch recently.

So more kill a watt tests… I’ll post the results.

 

[Dunuin]

I was wondering if it's better to replace my 3 pcie cards with one 10Gb nic and to use virtual interfaces.

Would there be any downsides in terms of performance?

Latency could get worse with just one NIC. With 3 NICs you could use one exclusivly for the connection to your NAS and the other ones for services and internet. So even if you run a multiple terabyte backup from the PVE to NAS the services and internet won't be slowed down because the other NICs haven't got much to do.
If you just got one 10Gbit NIC and your NAS is capable to utilize the full 10Gbit bandwidth your connection is saturated and all services/internet will be slower and the latency will increase because the link is already flooded with packets.

 

[testcb00]

Latency could get worse with just one NIC. With 3 NICs you could use one exclusivly for the connection to your NAS and the other ones for services and internet. So even if you run a multiple terabyte backup from the PVE to NAS the services and internet won't be slowed down because the other NICs haven't got much to do.
If you just got one 10Gbit NIC and your NAS is capable to utilize the full 10Gbit bandwidth your connection is saturated and all services/internet will be slower and the latency will increase because the link is already flooded with packets.

Hi, Dunuin. I would like to know more about it.

I am planning to use a 4-Port Intel XL710 10GbE SFP+ PCIe NIC as my 1U server NIC.
I will passthrough 2 Port to pfSense VM, one Port to TrueNAS VM and one Port as SR-IOV PF for other Linux VM.

Both the 4 Port of XL710 will connect to a 10GbE Switch. My PC, other IoT device and WAN will connect to the Switch.
Under this condition, if the NAS and my PC will not saturate the 10GbE link, will I still face the latency problem?

 

[Dunuin]

Sounds fine. I just wouldn't share the same NIC for pfsense and the NAS.