Networking between PFSense VM and PVE went down. Internet was up.

davidg1982

Active Member
May 26, 2017
14
2
43
41
Chicago, Illinois USA
I am baffled by this one. I run a one machine PVE environment running OVS with everything going through PFSense via two vm bridges.

Everything has been working great, but this morning something unusual happened. Network connectivity between my VMs and PVE were down. Connectivity between PFSense (KVM) and PVE was down. Connectivity between PFSense and the Internet was up.
I read through a bunch of logs and couldn't find anything strange except for some ipv4 martian source messages.

So, I am here like Dr.House (TV Show) looking for possible explanations why the networking between hosts would go down, but my PFSense connectivity would stay up.

/etc/network/interfaces:
Code:
auto lo
iface lo inet loopback

#allow-ovs vmbr0

allow-vmbr0 eno1
iface eno1 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr0

auto vmbr0
iface vmbr0 inet manual
        ovs_type OVSBridge
        ovs_ports eno1

auto vmbr1
iface vmbr1 inet static
        address  192.168.0.2
        netmask  255.255.255.0
        gateway  192.168.0.1
        ovs_type OVSBridge
        ovs_ports vlan10 vlan20 vlan30 vlan40

allow-vmbr1 vlan10
iface vlan10 inet manual
        ovs_type OVSIntPort
        ovs_bridge vmbr1
        ovs_options tag=10

allow-vmbr1 vlan20
iface vlan20 inet manual
        ovs_type OVSIntPort
        ovs_bridge vmbr1
        ovs_options tag=20

allow-vmbr1 vlan30
iface vlan30 inet manual
        ovs_type OVSIntPort
        ovs_bridge vmbr1
        ovs_options tag=30

allow-vmbr1 vlan40
iface vlan40 inet manual
        ovs_type OVSIntPort
        ovs_bridge vmbr1
        ovs_options tag=40

type of card being used
Code:
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

pveversion -v
Code:
proxmox-ve: 5.4-2 (running kernel: 4.15.18-17-pve)
pve-manager: 5.4-10 (running version: 5.4-10/9603c337)
pve-kernel-4.15: 5.4-5
pve-kernel-4.15.18-17-pve: 4.15.18-43
pve-kernel-4.15.18-16-pve: 4.15.18-41
pve-kernel-4.15.18-14-pve: 4.15.18-39
pve-kernel-4.15.18-12-pve: 4.15.18-36
pve-kernel-4.15.18-10-pve: 4.15.18-32
pve-kernel-4.15.18-9-pve: 4.15.18-30
pve-kernel-4.15.18-7-pve: 4.15.18-27
pve-kernel-4.15.18-5-pve: 4.15.18-24
pve-kernel-4.15.17-1-pve: 4.15.17-9
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-11
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-53
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
openvswitch-switch: 2.7.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-37
pve-container: 2.0-39
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

ethtool -k eno1
Code:
Features for eno1:
Cannot get device udp-fragmentation-offload settings: Operation not supported
rx-checksumming: on
tx-checksumming: on
    tx-checksum-ipv4: on
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: on
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: off [fixed]
scatter-gather: on
    tx-scatter-gather: on
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
    tx-tcp-segmentation: on
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp-mangleid-segmentation: off
    tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
 
Last edited:
DId you find an explanation?
Connectivity between PFSense (KVM) and PVE was down.
I don't understand this, which interface was down?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!