Hello!
I'm currently trying to set up a new Proxmox installation (from an existing Debian Bullseye), and it turned out I need to set up conntrack zones to make masquerading work correctly.
According to the wiki (https://pve.proxmox.com/wiki/Networ...ith_tt_span_class_monospaced_iptables_span_tt):
However, after applying this rule, the guest cannot ping the host. But after issuing a arping, the guest can successfully reach the host.
Some other curious things also happened:
- The first arping request receives two responses: one from the MAC of the vm's corresponding fwbr* interface, the other from vmbr0. This is presumably related to Linux's default arp_announce setting.
- The packet when traveling from tap to fwbr* immediately enters the iptables filter INPUT chain.
- When the ARP table (shown with `ip neighbour`) caches the MAC address from fwbr* interfaces, ping fails. If the MAC of vmbr0 is cached, then ping works.
It seems that somehow Linux kernel is making routing decisions based on the packet's MAC address besides IP addresses. This seems very strange to me, but I'm not familiar with Linux's networking internals. Is this the intended behavior of Proxmox? And is there a proper way to fix this?
Thank you very much!
---------
I've temporarily mitigated this issue by changing the arp_announce configuration:
iptables trace with failed pings (IN=fwbr100i0 -> FORWARD)
iptables trace with success pings (IN=fwbr100i0 -> FORWARD, IN=vmbr0 -> INPUT):
I'm currently trying to set up a new Proxmox installation (from an existing Debian Bullseye), and it turned out I need to set up conntrack zones to make masquerading work correctly.
According to the wiki (https://pve.proxmox.com/wiki/Networ...ith_tt_span_class_monospaced_iptables_span_tt):
Code:
iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1However, after applying this rule, the guest cannot ping the host. But after issuing a arping, the guest can successfully reach the host.
Some other curious things also happened:
- The first arping request receives two responses: one from the MAC of the vm's corresponding fwbr* interface, the other from vmbr0. This is presumably related to Linux's default arp_announce setting.
- The packet when traveling from tap to fwbr* immediately enters the iptables filter INPUT chain.
- When the ARP table (shown with `ip neighbour`) caches the MAC address from fwbr* interfaces, ping fails. If the MAC of vmbr0 is cached, then ping works.
It seems that somehow Linux kernel is making routing decisions based on the packet's MAC address besides IP addresses. This seems very strange to me, but I'm not familiar with Linux's networking internals. Is this the intended behavior of Proxmox? And is there a proper way to fix this?
Thank you very much!
---------
I've temporarily mitigated this issue by changing the arp_announce configuration:
Code:
sysctl net.ipv4.conf.all.arp_announce=1
sysctl net.ipv4.conf.all.arp_ignore=1iptables trace with failed pings (IN=fwbr100i0 -> FORWARD)
Code:
Aug 21 21:22:15 hostname kernel: [ 2151.206048] TRACE: raw:PREROUTING:rule:2 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206054] TRACE: raw:PREROUTING:policy:3 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206069] TRACE: filter:INPUT:rule:1 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206079] TRACE: filter:PVEFW-INPUT:rule:1 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206083] TRACE: filter:PVEFW-HOST-IN:rule:4 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206093] TRACE: filter:PVEFW-smurfs:rule:4 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206098] TRACE: filter:PVEFW-smurfs:return:5 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206102] TRACE: filter:PVEFW-HOST-IN:return:11 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206106] TRACE: filter:PVEFW-INPUT:rule:2 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206110] TRACE: filter:PVEFW-INPUT:return:3 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9
Aug 21 21:22:15 hostname kernel: [ 2151.206113] TRACE: filter:INPUT:policy:2 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=ea:9a:d9:8d:77:2d:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3361 DF PROTO=ICMP TYPE=8 CODE=0 ID=2 SEQ=9iptables trace with success pings (IN=fwbr100i0 -> FORWARD, IN=vmbr0 -> INPUT):
Code:
Aug 21 21:23:22 hostname kernel: [ 2218.071300] TRACE: raw:PREROUTING:rule:2 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=36:a2:88:e1:9c:f8:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=21434 DF PROTO=ICMP TYPE=8 CODE=0 ID=4 SEQ=2
Aug 21 21:23:22 hostname kernel: [ 2218.071305] TRACE: raw:PREROUTING:policy:3 IN=fwbr100i0 OUT= PHYSIN=tap100i0 MAC=36:a2:88:e1:9c:f8:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=21434 DF PROTO=ICMP TYPE=8 CODE=0 ID=4 SEQ=2
Aug 21 21:23:22 hostname kernel: [ 2218.071312] TRACE: filter:FORWARD:rule:1 IN=fwbr100i0 OUT=fwbr100i0 PHYSIN=tap100i0 PHYSOUT=fwln100i0 MAC=36:a2:88:e1:9c:f8:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=21434 DF PROTO=ICMP TYPE=8 CODE=0 ID=4 SEQ=2
Aug 21 21:23:22 hostname kernel: [ 2218.071318] TRACE: raw:PREROUTING:policy:3 IN=vmbr0 OUT= PHYSIN=fwpr100p0 MAC=36:a2:88:e1:9c:f8:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=21434 DF PROTO=ICMP TYPE=8 CODE=0 ID=4 SEQ=2
Aug 21 21:23:22 hostname kernel: [ 2218.071324] TRACE: filter:INPUT:rule:1 IN=vmbr0 OUT= PHYSIN=fwpr100p0 MAC=36:a2:88:e1:9c:f8:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=21434 DF PROTO=ICMP TYPE=8 CODE=0 ID=4 SEQ=2
Aug 21 21:23:22 hostname kernel: [ 2218.071333] TRACE: filter:PVEFW-INPUT:rule:1 IN=vmbr0 OUT= PHYSIN=fwpr100p0 MAC=36:a2:88:e1:9c:f8:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=21434 DF PROTO=ICMP TYPE=8 CODE=0 ID=4 SEQ=2
Aug 21 21:23:22 hostname kernel: [ 2218.071337] TRACE: filter:PVEFW-HOST-IN:rule:3 IN=vmbr0 OUT= PHYSIN=fwpr100p0 MAC=36:a2:88:e1:9c:f8:06:83:e8:35:f6:0f:08:00 SRC=10.0.0.100 DST=10.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=21434 DF PROTO=ICMP TYPE=8 CODE=0 ID=4 SEQ=2