Problem Statement
PVE Host cannot ping or communicate to VM on same host, only when that VM is in a specific vlan (that is shared with the host). Issue persists across hosts, but always only the one specific vlan has issues - guests in another vlan (also with a vlanX interface on the host) work fine. Communication to/from this VM from another host in the network is fine. Only host<>VM on same localhost machine has issues
Environment
Host physical network setup:
In summary
I have another VM in the same host, in vmbr0, tagged with vlan60, and I can ping this VM from the host no issues. So seems to be specific to vlan 2. Issue is also an entire loss of connectivity. No TCP/UDP/ICMP works at all.
Finally, I can confirm that tcpdump inside the guest VM (simply running stock Debian 12) shows no traffic entering the VM, and tap*i0 interface on the host also shows no traffic entering the interface. Feels like a host-level routing issue.
I thought originally it may be an issue with the default gateway behavior, so I tried moving the default gateway to vlan60 (10.1.60.1) - no change. Can still ping the vlan60 VM, but not the vlan2 VM.
Thoughts? I've got these things split out to spread my storage traffic off to its own NIC, though since they are all 40Gbps links, there *likely* isn't a huge impact if I put them all on the same NIC (or bonded the two 40Gbps links and run it all off that). I'm also unsure if this would resolve the issue, since I have no clue why vlan2 doesn't work, but vlan60 does. I'd also just like to figure out why this isn't working, so I can know the logic on how to deploy going forward.
PVE Host cannot ping or communicate to VM on same host, only when that VM is in a specific vlan (that is shared with the host). Issue persists across hosts, but always only the one specific vlan has issues - guests in another vlan (also with a vlanX interface on the host) work fine. Communication to/from this VM from another host in the network is fine. Only host<>VM on same localhost machine has issues
Environment
Host physical network setup:
- NICs:
- enp9s0: Will be used for VM traffic, trunked)
- enp13s0: Will be used for Management traffic (vlan2)
- enp14s0: Will be used for Storage (ceph, etc) traffic (vlan60)
- enp10s0,enp1s0f0,enp1s0f1: Unused
Code:
root@pve03:~# cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!
auto lo
iface lo inet loopback
auto enp1s0f0
iface enp1s0f0 inet manual
#Ethernet 1 - unused
auto enp1s0f1
iface enp1s0f1 inet manual
#Ethernet 2 - unused
auto enp10s0
iface enp10s0 inet manual
#VIC2 - unused
auto enp9s0
iface enp9s0 inet manual
ovs_type OVSPort
ovs_bridge vmbr0
#VIC VM Network
auto enp13s0
iface enp13s0 inet manual
ovs_type OVSPort
ovs_bridge vmbr2
#VIC mgmt
auto enp14s0
iface enp14s0 inet manual
ovs_type OVSPort
ovs_bridge vmbr60
#VIC Storage
auto vlan2
iface vlan2 inet static
address 10.1.0.33/24
gateway 10.1.0.1
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=2
#Management VLAN
auto vlan60
iface vlan60 inet static
address 10.1.60.33/24
ovs_type OVSIntPort
ovs_bridge vmbr60
ovs_options tag=60
#Storage VLAN
auto vmbr2
iface vmbr2 inet manual
ovs_type OVSBridge
ovs_ports enp13s0 vlan2
#Management Network Bridge
auto vmbr60
iface vmbr60 inet manual
ovs_type OVSBridge
ovs_ports enp14s0 vlan60
#Storage Network Bridge
auto vmbr0
iface vmbr0 inet manual
ovs_type OVSBridge
ovs_ports enp9s0
#VM Network Bridge
Code:
root@pve03:~# ip -c r
default via 10.1.0.1 dev vlan2 proto kernel onlink
10.1.0.0/24 dev vlan2 proto kernel scope link src 10.1.0.33
10.1.60.0/24 dev vlan60 proto kernel scope link src 10.1.60.33In summary
- vmbr0 bridges enp9s0 and is where all VMs live (with vlans tagged in the .conf accordingly)
- vmbr2 bridges enp13s0 and vlan2 (the host's management "NIC")
- vmbr60 bridges enp13s0 and vlan60 (the host's storage "NIC")
I have another VM in the same host, in vmbr0, tagged with vlan60, and I can ping this VM from the host no issues. So seems to be specific to vlan 2. Issue is also an entire loss of connectivity. No TCP/UDP/ICMP works at all.
Finally, I can confirm that tcpdump inside the guest VM (simply running stock Debian 12) shows no traffic entering the VM, and tap*i0 interface on the host also shows no traffic entering the interface. Feels like a host-level routing issue.
I thought originally it may be an issue with the default gateway behavior, so I tried moving the default gateway to vlan60 (10.1.60.1) - no change. Can still ping the vlan60 VM, but not the vlan2 VM.
Thoughts? I've got these things split out to spread my storage traffic off to its own NIC, though since they are all 40Gbps links, there *likely* isn't a huge impact if I put them all on the same NIC (or bonded the two 40Gbps links and run it all off that). I'm also unsure if this would resolve the issue, since I have no clue why vlan2 doesn't work, but vlan60 does. I'd also just like to figure out why this isn't working, so I can know the logic on how to deploy going forward.
Last edited: