I've got a rather complicated setup for home use, I'll try to give enough detail up front.
The problem is with vm105:
100(cloud) -- runs caddy as reverse proxy on docker, serves traffic through to other docker containers on itself, and other VMs and LXC containers across vlan15 and vlan20
105(haos) -- home assistant, can't get any traffic to respond to caddy, but from vm100 I can ping and I see multicast arp traffic from vm105
All VMs sit on vmbr0 with all vlan's enabled on the interface, and terminate on the proxmox network device for each VM.
I also have vmbr1 which is used for some VMs to communicate without relying on a physical network device.
PVE VM configs:
vm100:
vm105:
PVE network config:
PVE bridge setup:
PVE vlans:
I usually have firewall enabled for all guests, but disabled it for vm100 and vm105 for testing.
vm100 has IPs 10.15.1.220 and 10.20.1.220
vm105 has IP 10.20.1.228
Ping works:
But if I run "wget http://10.20.1.228:8123" from a separate terminal on vm100, nothing.
I know the traffic is leaving vm100, reaches vm105, then the return response comes back again all on vmbr0:
But the traffic never seems to reach vm100's tap device again:
The only traffic that shows up is some ARP and multicast coming from vm105.
As a test, I've done the same thing in reverse and I believe its the same result.
From vm105, running "curl http://10.20.1.220":
Nothing ever arrives on tap105i0.
I also have vlan10, which my PCs reside on. I can http://10.20.1.228:8123 from 10.10.1.1 and it works fine.
So my conclusion is that there's *something* blocking return traffic between the bridge and some tap devices.
The problem is with vm105:
100(cloud) -- runs caddy as reverse proxy on docker, serves traffic through to other docker containers on itself, and other VMs and LXC containers across vlan15 and vlan20
105(haos) -- home assistant, can't get any traffic to respond to caddy, but from vm100 I can ping and I see multicast arp traffic from vm105
All VMs sit on vmbr0 with all vlan's enabled on the interface, and terminate on the proxmox network device for each VM.
I also have vmbr1 which is used for some VMs to communicate without relying on a physical network device.
PVE VM configs:
vm100:
root@bedrock:~# cat /etc/pve/qemu-server/100.confname: cloudnet0: virtio=BC:24:11:3A:C7:69,bridge=vmbr0,tag=20net1: virtio=BC:24:11:32:00:07,bridge=vmbr0,tag=15vm105:
root@bedrock:~# cat /etc/pve/qemu-server/105.confname: haosnet0: virtio=02:8C:44:C6:3F:9C,bridge=vmbr0,tag=20PVE network config:
root@bedrock:~# cat /etc/network/interfaces...iface vmbr0 inet manual bridge-ports enp3s0 bridge-stp off bridge-fd 0 bridge-vlan-aware yes bridge-vids 10 15 20 25 30 50#VLAN Bridge for Guestsauto vmbr1iface vmbr1 inet static address 192.168.1.1/24 bridge-ports none bridge-stp off bridge-fd 0PVE bridge setup:
root@bedrock:~# brctl showbridge name bridge id STP enabled interfaces...vmbr0 8000.408d5c780643 no enp3s0 fwpr101p0 fwpr103p0 fwpr104p0 fwpr104p1 fwpr200p0 fwpr200p1 fwpr200p2 fwpr201p0 fwpr202p0 fwpr204p0 tap100i0 tap100i1 tap105i0...PVE vlans:
root@bedrock:~# bridge vlan showport vlan-idenp3s0 1 PVID Egress Untagged 10 15 20 25 30 50vmbr0 1 PVID Egress Untaggedvmbr1 1 PVID Egress Untagged...tap100i0 20 PVID Egress Untaggedtap100i1 15 PVID Egress Untagged...tap105i0 20 PVID Egress UntaggedI usually have firewall enabled for all guests, but disabled it for vm100 and vm105 for testing.
vm100 has IPs 10.15.1.220 and 10.20.1.220
vm105 has IP 10.20.1.228
Ping works:
root@cloud:~# tcpdump -nnvvS -i enp6s18 host 10.20.1.228tcpdump: listening on enp6s18, link-type EN10MB (Ethernet), snapshot length 262144 bytes09:10:03.687792 IP (tos 0x0, ttl 64, id 38461, offset 0, flags [DF], proto ICMP (1), length 84) 10.20.1.220 > 10.20.1.228: ICMP echo request, id 4, seq 1, length 6409:10:03.688111 IP (tos 0x0, ttl 63, id 57101, offset 0, flags [none], proto ICMP (1), length 84) 10.20.1.228 > 10.20.1.220: ICMP echo reply, id 4, seq 1, length 64But if I run "wget http://10.20.1.228:8123" from a separate terminal on vm100, nothing.
I know the traffic is leaving vm100, reaches vm105, then the return response comes back again all on vmbr0:
root@bedrock:~# tcpdump -i vmbr0 host 10.20.1.228 and host 10.20.1.220tcpdump: verbose output suppressed, use -v[v]... for full protocol decodelistening on vmbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes09:17:56.342721 IP 10.20.1.220.47944 > 10.20.1.228.8123: Flags [S], seq 3301836575, win 64240, options [mss 1460,sackOK,TS val 3035091219 ecr 0,nop,wscale 7], length 009:17:56.342867 IP 10.20.1.228.8123 > 10.20.1.220.47944: Flags [S.], seq 269643130, ack 3301836576, win 65160, options [mss 1460,sackOK,TS val 2921087751 ecr 3035091219,nop,wscale 7], length 009:17:57.359655 IP 10.20.1.228.8123 > 10.20.1.220.47944: Flags [S.], seq 269643130, ack 3301836576, win 65160, options [mss 1460,sackOK,TS val 2921088768 ecr 3035091219,nop,wscale 7], length 009:17:57.374254 IP 10.20.1.220.47944 > 10.20.1.228.8123: Flags [S], seq 3301836575, win 64240, options [mss 1460,sackOK,TS val 3035092251 ecr 0,nop,wscale 7], length 009:17:57.374390 IP 10.20.1.228.8123 > 10.20.1.220.47944: Flags [S.], seq 269643130, ack 3301836576, win 65160, options [mss 1460,sackOK,TS val 2921088782 ecr 3035091219,nop,wscale 7], length 009:17:59.407710 IP 10.20.1.228.8123 > 10.20.1.220.47944: Flags [S.], seq 269643130, ack 3301836576, win 65160, options [mss 1460,sackOK,TS val 2921090816 ecr 3035091219,nop,wscale 7], length 009:18:01.374173 ARP, Request who-has 10.20.1.228 tell 10.20.1.220, length 2809:18:01.374313 ARP, Reply 10.20.1.228 is-at 02:8c:44:c6:3f:9c (oui Unknown), length 28But the traffic never seems to reach vm100's tap device again:
root@bedrock:~# tcpdump -i tap100i0 src 10.20.1.228tcpdump: verbose output suppressed, use -v[v]... for full protocol decodelistening on tap100i0, link-type EN10MB (Ethernet), snapshot length 262144 bytes09:18:01.374310 ARP, Reply 10.20.1.228 is-at 02:8c:44:c6:3f:9c (oui Unknown), length 2809:18:23.075582 IP 10.20.1.228.33981 > 239.255.255.250.1900: UDP, length 324The only traffic that shows up is some ARP and multicast coming from vm105.
As a test, I've done the same thing in reverse and I believe its the same result.
From vm105, running "curl http://10.20.1.220":
root@bedrock:~# tcpdump -i vmbr0 host 10.20.1.228 and host 10.20.1.220tcpdump: verbose output suppressed, use -v[v]... for full protocol decodelistening on vmbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes09:27:51.495515 IP 10.20.1.228.54682 > 10.20.1.220.http: Flags [S], seq 3539920993, win 64240, options [mss 1460,sackOK,TS val 2921682903 ecr 0,nop,wscale 7], length 009:27:51.495701 IP 10.20.1.228.54682 > 10.20.1.220.http: Flags [S], seq 3539920993, win 64240, options [mss 1460,sackOK,TS val 2921682903 ecr 0,nop,wscale 7], length 009:27:51.495895 IP 10.20.1.220.http > 10.20.1.228.54682: Flags [S.], seq 3076438070, ack 3539920994, win 65160, options [mss 1460,sackOK,TS val 1513592998 ecr 2921682903,nop,wscale 7], length 009:27:52.495805 IP 10.20.1.228.54682 > 10.20.1.220.http: Flags [S], seq 3539920993, win 64240, options [mss 1460,sackOK,TS val 2921683904 ecr 0,nop,wscale 7], length 0Nothing ever arrives on tap105i0.
I also have vlan10, which my PCs reside on. I can http://10.20.1.228:8123 from 10.10.1.1 and it works fine.
So my conclusion is that there's *something* blocking return traffic between the bridge and some tap devices.
Last edited: