Odd problem with VirtIO NIC

[imrazor]

I recently set up pfSense in a Proxmox VM, and while it works as an OpenVPN endpoint, it has displayed a couple of odd issues when attempting to connect from the VM to the host (ie, to the Proxmox GUI.) When the VirtIO NIC is selected, I cannot connect to the Proxmox host (though, weirdly, I can ping it.) If I select the Realtek or Intel virtual NIC, I have no problem connecting across OpenVPN to the VM, then to the Proxmox GUI.

I'm not sure if the problem lies with pfSense's virtio NIC driver (FreeBSD, right?) or with KVM's NIC emulation. The Realtek driver seemed to occasionally drop connections, but so far the Intel driver seems stable. However, this probably increases overhead on the pfSense VM and the Proxmox host. Any thoughts on this?

 

[bobmc]

I've seen some odd behaviour using pfSense under Proxmox recently which seems to be a dependent on both hardware and software versions. I've had one installation providing ipsec VPN links running rock solid for almost a year on some repurposed hardware and as a result recently setup a new site as a copy of the original install but with new server grade hardware running Proxmox 5.2. The new system is completely solid (two months now) but after upgrading the original site to Proxmox 5.2 the VPN connection is now very flakey (perfect for VPN into the pfSense console but access to any other containers or KVM's or to the proxmox host is either very slow or times out)

The major difference hardware wise, is that one system uses Intel i210 NICs while the other uses Intel E1000's (both are running VirIO drivers)

Two things to check
1. Did you enable 'Disable Hardware Checksum Offload' in the pfSense GUI
2. Are you on the latest kernel in Proxmox - (4.15.18-7-pve)

 

[imrazor]

No, checksum offloading was enabled. I've turned it off. On my Proxmox host (actually full install of Debian 9 + Proxmox kernel & packages) uname -r reports:
4.15.18-7-pve
So it looks like I'm on the latest kernel, but that pfSense option may be causing problems. I'm working remotely right now, but when I get a chance I'll change back to the virtio NIC and see what happens.

 

[imrazor]

Turning off checksum offloading in pfSense seemed to work. If it helps to narrow the problem down, the host's adapter is an onboard Realtek 8168FB.

So is KVM trying to pass through the hardware checksum offloading to the host's physical NIC, but failing in odd ways? Would putting in a genuine Intel E1000 adapter into the host allow this to function properly? Is turning off checksum offloading likely to put significantly more load on the VM?

 

[dcsapak]

So is KVM trying to pass through the hardware checksum offloading to the host's physical NIC, but failing in odd ways? Would putting in a genuine Intel E1000 adapter into the host allow this to function properly? Is turning off checksum offloading likely to put significantly more load on the VM?

with virtio the checksum offloading would probably be done by your hardware (depending on the network /drivers/etc. of your host) if you deactivate this, then yes the guest cpu has to do this

with e1000 or realtek the qemu process has to do this, at least i am not aware that qemu is capable of offloading it then

so without virtio and offloading it always has to be done by the cpu, but you can test the load on the cpu e.g. with iperf

see also https://www.netgate.com/docs/pfsense/virtualization/virtio-driver-support.html

 

[bobmc]

Well the system that is having issues for me, is using an (pci) dual Intel E1000 so I wouldn't say that would be the answer to your problems. Having said that, I've not had great experiences with Realtek NIC's in the past.

Assuming the Realtek is providing the WAN interface for pfSense, have you though about trying hardware pass-through?

 

[imrazor]

Well the system that is having issues for me, is using an (pci) dual Intel E1000 so I wouldn't say that would be the answer to your problems. Having said that, I've not had great experiences with Realtek NIC's in the past.

Assuming the Realtek is providing the WAN interface for pfSense, have you though about trying hardware pass-through?

Right now the VM is bridged to the physical NIC. I suppose I could buy an additional network card to passthrough, but last time I tried passthrough with Proxmox things got rather complicated. I managed to get passthrough working at the time, but if my current config remains stable and the load is reasonable I'd rather not do it again.