Uprgrade from PVE 5 to PVE 6 results in a slower network connection with VM pfsense having interface E1000

[xekhz]

Hello,
I think this is my first post dont beat me up.

I have always managed to solve my problems searching advice in this forum, and until now, zero problems,
But today i gave up.
Let me explain

I have been using proxmox since V5 , i had a cluster with 3 nodes in pve5, when pve6 was released i started upgrandig the servers and evaluating the result.

Problem:
Uprgrade from PVE 5 to PVE 6 results in a slower network connection with VM pfsense having interface E1000

Context
I have a PVE5 (https://postimg.cc/Ln40wg70) with pfsense 2.4.4 when i execute iperf3 from a remote node to the VM pfsense i get this values.

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 475 MBytes 399 Mbits/sec 2634 sender
[ 5] 0.00-10.01 sec 473 MBytes 397 Mbits/sec receiver

The path is; starting pve6 node -> switch -> switch -> pv5 node -> VM pfsense

in the PVE6 (https://postimg.cc/5QXQQ3yR) cluster i also have a pfsense 2.4.4 but doing nothing

running the same command i get this
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 339 MBytes 285 Mbits/sec 0 sender
[ 5] 0.00-10.02 sec 336 MBytes 281 Mbits/sec receiver

The path is; starting pve6 node -> VM pfsense

Conclusion:
Having both pfsense alike, same version and hardware.
Performance in a much shorter network path is ~120 Mbits average slower...
What cloud be the problem ?

 

[Stefan_R]

This might be a regression in QEMU... E1000 was never designed for high-throughput though, why not use virtio network interfaces? pfSense natively supports them and they provide *much* better latency and throughput.

 

[xekhz]

Hello,
I have tried, But when i enable virto.. the systems statrs lagging.
I have two wans interfaces, with vlan tag and E1000 connected to the ISP router
And one interface E1000 ( lets name it LAN ) connected to a mikrotik switch. When i change the LAN interface to VIRTIO the network starts lagging

 

[Stefan_R]

When i change the LAN interface to VIRTIO the network starts lagging

What do you mean by "lagging"? Latency?

Also, with virtio it might be necessary to disable any hardware offloading in pfsense, so within the VM.

 

[xekhz]

Not lattency, the pfsense vm, becames unresponsive and the interface offline.

 

[spirit]

Not lattency, the pfsense vm, becames unresponsive and the interface offline.

do you have disabled hardware checksum offload ?

https://docs.netgate.com/pfsense/en...ksum offload,checksum offload and click save.

 

[xekhz]

I disabled all of this :
Hardware Checksum Offloading
Hardware TCP Segmentation Offloading
Hardware Large Receive Offloading

 

[xekhz]

In this print screen ( https://ibb.co/0sRZfw3 ) it is visible what happens to the PVE6 cluster.
Node5 grey. ( made a pvestatd restart and did not work )
Node4 , storage also grey, and cant connect via IP address
As you can see i added the virtio interface. In pfsense i changed the assignements from the E1000 to the Virtio

 

[Stefan_R]

In this print screen ( https://ibb.co/0sRZfw3 ) it is visible what happens to the PVE6 cluster.
Node5 grey. ( made a pvestatd restart and did not work )
Node4 , storage also grey, and cant connect via IP address

That is an entirely different issue from the pfSense one though? Or are you routing your cluster traffic through the pfSense VM? (if that's the case, stop doing that, please )

 

[xekhz]

That is an entirely different issue from the pfSense one though? Or are you routing your cluster traffic through the pfSense VM? (if that's the case, stop doing that, please )

yes i was doing that...
What is your advice?
thanks.

 

[bobmc]

Ideally your proxmox host should have at least two physical nics - dedicate one of these nics to pfsense WAN traffic (create a vmbr1 for this) and when you create your pfsense VM assign a network interface on vmbr1 for WAN traffic (and plug your internet router into this port either directly or via a switch) then assign any LAN interfaces on the VM to vmbr0. I would also use virtio wherever possible - E1000 should only be used where compatibility is needed.

 

[xekhz]

thanks will do that.