This how-to-fix post to inform people on how Suricata crashes with OPNSense on Proxmox (any version) can be remediated.
The advisories here may not be suitable for production environments, I trust you know this already.
Context
VM-hardware has Q35 chipset and uses virtio network interfaces.
The OPNSense host has qemu-guest-agent installed.
Indicator
Jan 28 12:39:45 opnsense kernel: 385.664273 [2197] netmap_buf_size_validate error: large MTU (8192) needed but <interfacename> does not support NS_MOREFRAG
Assumption
This indicates MTU inconsistency when MTU is set >1500 on the bridge and this is 'broken' in-between the bridge and the IPS.
for large MTU only virtio appears to work with ProxMox and OPNSense
this also implies to install and start qemu-guest in the opnsense firewall for optimal performance and stability
Recommended is to check if
MTU on the bridge is >1500
configure : within Proxmox
check and set the VM-hardware network-interface(s) to 1 so these adopt the MTU of the connected network.
configure : within OPNSense
[ for Interfaces ] check and/or clear MTU settings for the monitored interfaces
[ for Suricata] under the 'advanced' section of the IPS service : check and/or clear default packet size (MTU) setting
setting the MTU here can affect detection reliability and 'drop' or 'conflate' frames on inspection
if all else fails, consider setting value for MTU=interface-MTU-22
also take note to NOT add VLAN interfaces to Suricata for monitoring, use the physical interface only
important know that on non-enterprise network cards there may not be support for 'real' Jumbo frames which permits MTU >1500
Look up the specifications for the network interface cards (NIC) and do not set the MTU higher than the hardware supports, even if the MTU on the connecting switch is set to a much higher value.
[ for SYSTEM: SETTINGS: TUNABLES ] manually create the key dev.netmap.bufsize with value = <MTUvalue>
this to work around issues with some NIC where MTU is not working well, so hard-set it here with this key
configure : optionally for OPNSense
[ for SYSTEM: SETTINGS: TUNABLES ] manually create the key dev.netmap.admode with value = 1
this to avoid flapping between native and emulation state for the network interface
[ for Suricata] you can try set the MTU-22 as size for stability
Considerations
when the value for the MTU is cleared for an interface this defaults to 1500
consider this may severely impact IPS performance and/or accuracy
performance is best when setting physical interface to max MTU supported by card and connected switch
next set the bridge to value=phyisicalMTU-34
Resources
https://docs.opnsense.org/manual/ips.html
https://man.freebsd.org/cgi/man.cgi...eBSD+12.1-RELEASE+and+Ports#SUPPORTED_DEVICES
https://man.freebsd.org/cgi/man.cgi?vtnet
The advisories here may not be suitable for production environments, I trust you know this already.
Context
VM-hardware has Q35 chipset and uses virtio network interfaces.
The OPNSense host has qemu-guest-agent installed.
Indicator
Jan 28 12:39:45 opnsense kernel: 385.664273 [2197] netmap_buf_size_validate error: large MTU (8192) needed but <interfacename> does not support NS_MOREFRAG
Assumption
This indicates MTU inconsistency when MTU is set >1500 on the bridge and this is 'broken' in-between the bridge and the IPS.
for large MTU only virtio appears to work with ProxMox and OPNSense
this also implies to install and start qemu-guest in the opnsense firewall for optimal performance and stability
Recommended is to check if
MTU on the bridge is >1500
configure : within Proxmox
check and set the VM-hardware network-interface(s) to 1 so these adopt the MTU of the connected network.
configure : within OPNSense
[ for Interfaces ] check and/or clear MTU settings for the monitored interfaces
[ for Suricata] under the 'advanced' section of the IPS service : check and/or clear default packet size (MTU) setting
setting the MTU here can affect detection reliability and 'drop' or 'conflate' frames on inspection
if all else fails, consider setting value for MTU=interface-MTU-22
also take note to NOT add VLAN interfaces to Suricata for monitoring, use the physical interface only
important know that on non-enterprise network cards there may not be support for 'real' Jumbo frames which permits MTU >1500
Look up the specifications for the network interface cards (NIC) and do not set the MTU higher than the hardware supports, even if the MTU on the connecting switch is set to a much higher value.
[ for SYSTEM: SETTINGS: TUNABLES ] manually create the key dev.netmap.bufsize with value = <MTUvalue>
this to work around issues with some NIC where MTU is not working well, so hard-set it here with this key
configure : optionally for OPNSense
[ for SYSTEM: SETTINGS: TUNABLES ] manually create the key dev.netmap.admode with value = 1
this to avoid flapping between native and emulation state for the network interface
[ for Suricata] you can try set the MTU-22 as size for stability
Considerations
when the value for the MTU is cleared for an interface this defaults to 1500
consider this may severely impact IPS performance and/or accuracy
performance is best when setting physical interface to max MTU supported by card and connected switch
next set the bridge to value=phyisicalMTU-34
Resources
https://docs.opnsense.org/manual/ips.html
https://man.freebsd.org/cgi/man.cgi...eBSD+12.1-RELEASE+and+Ports#SUPPORTED_DEVICES
https://man.freebsd.org/cgi/man.cgi?vtnet
Last edited: