Firewall not matching

D

David Herselman

Renowned Member
Proxmox Subscriber
Jun 8, 2016
344
69
68
47
Johannesburg, South Africa
We have a cluster on which I would like to implement per VM firewalling. We enabled firewalling on the datacentre, node and guest basis and finally set the VM's network interface to also use the function.

Reviewing iptables shows packets matching the guest's IN chain but nothing in that chain is being matched.

Herewith applicable chains:
Code: 
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
5387M  629G PVEFW-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain PVEFW-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
26726 1086K DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
5282M  622G ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
  27M 1884M PVEFW-FWBR-IN  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in fwln+ --physdev-is-bridged
  26M 1717M PVEFW-FWBR-OUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out fwln+ --physdev-is-bridged
  77M 4964M            all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw */

Chain PVEFW-FWBR-IN (1 references)
 pkts bytes target     prot opt in     out     source               destination
  27M 1884M PVEFW-smurfs  all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID,NEW
22409 2413K tap105i0-IN  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out tap105i0 --physdev-is-bridged
  27M 1882M tap110i0-IN  all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out tap110i0 --physdev-is-bridged
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* PVESIG:lVmvmHpRKBUJookpkACTKl3FM0g */

Chain tap105i0-IN (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp spt:67 dpt:68
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
    0     0 GROUP-default-IN  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x80000000/0x80000000
    0     0 GROUP-sip_linux-IN  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x80000000/0x80000000
    0     0 PVEFW-Drop  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* PVESIG:p4ISMTQ5LnVj2VO4g3x70gL9ezc */


All chains have packet matches, except for the last one that should apply to the VM.


Network structure on this cluster is the standard Linux bridge with two bonds for VM traffic (bond0) and Ceph (bond1):
Code: 
auto lo
iface lo inet loopback

auto bond0
iface bond0 inet manual
        slaves eth0,eth1
        bond_miimon 100
        bond_mode active-backup
        mtu 9216

auto bond1
iface bond1 inet static
        address 10.254.1.4
        netmask  255.255.255.0
        slaves eth2,eth3
        bond_miimon 100
        bond_mode active-backup
        mtu 9216

auto eth0
iface eth0 inet manual
        bond-master bond0
        bond-primary eth0
        mtu 9216

auto eth1
iface eth1 inet manual
        bond-master bond0
        mtu 9216

auto eth2
iface eth2 inet manual
        bond-master bond1
        mtu 9216

auto eth3
iface eth3 inet manual
        bond-master bond1
        bond-primary eth3
        mtu 9216

auto eth4
iface eth4 inet manual

auto eth5
iface eth5 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.241.4
        netmask 255.255.255.0
        gateway 192.168.241.1
        bridge_ports bond0
        bridge_stp off
        bridge_fd 0
        mtu 9216

VM definition:
Code: 
agent: 1
boot: cdn
bootdisk: scsi0
cores: 2
cpu: SandyBridge,flags=+pcid
ide2: none,media=cdrom
localtime: 1
memory: 4096
name: 211-FreePBX
net0: virtio=5E:D8:F3:62:C5:90,bridge=vmbr0,firewall=1,tag=12
numa: 1
onboot: 1
ostype: l26
protection: 1
scsi0: rbd_hdd:vm-105-disk-1,cache=writeback,discard=on,size=100G
scsihw: virtio-scsi-pci
smbios1: uuid=cb38b843-e241-48f8-b681-8299a56bc8ac
sockets: 1
vga: cirrus


pveversion -v:
Code: 
proxmox-ve: 5.2-2 (running kernel: 4.15.18-1-pve)
pve-manager: 5.2-5 (running version: 5.2-5/eb24855a)
pve-kernel-4.15: 5.2-4
pve-kernel-4.15.18-1-pve: 4.15.18-15
ceph: 12.2.7-pve1
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-35
libpve-guest-common-perl: 2.0-17
libpve-http-server-perl: 2.0-9
libpve-storage-perl: 5.0-24
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-3
lxcfs: 3.0.0-1
novnc-pve: 1.0.0-1
proxmox-widget-toolkit: 1.0-19
pve-cluster: 5.0-28
pve-container: 2.0-24
pve-docs: 5.2-4
pve-firewall: 3.0-13
pve-firmware: 2.0-5
pve-ha-manager: 2.0-5
pve-i18n: 1.0-6
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.2-1
pve-xtermjs: 1.0-5
qemu-server: 5.0-29
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.9-pve1~bpo9
 
Having a first look it looks like an iptables bug which is not reproducible. For deeper investigation post the whole output of
Code: 
iptables-save
 
Hi Richard,

Apologies about not seeing this earlier, herewith the requested output:
Code: 
[root@kvm1c ~]# iptables-save
# Generated by iptables-save v1.6.0 on Tue Sep 11 17:13:32 2018
*filter
:INPUT ACCEPT [74773175:13446008020]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [71106382:11700562612]
:GROUP-default-IN - [0:0]
:GROUP-default-OUT - [0:0]
:GROUP-sip_linux-IN - [0:0]
:GROUP-sip_linux-OUT - [0:0]
:PVEFW-Drop - [0:0]
:PVEFW-DropBroadcast - [0:0]
:PVEFW-FORWARD - [0:0]
:PVEFW-FWBR-IN - [0:0]
:PVEFW-FWBR-OUT - [0:0]
:PVEFW-HOST-IN - [0:0]
:PVEFW-HOST-OUT - [0:0]
:PVEFW-INPUT - [0:0]
:PVEFW-OUTPUT - [0:0]
:PVEFW-Reject - [0:0]
:PVEFW-SET-ACCEPT-MARK - [0:0]
:PVEFW-logflags - [0:0]
:PVEFW-reject - [0:0]
:PVEFW-smurflog - [0:0]
:PVEFW-smurfs - [0:0]
:PVEFW-tcpflags - [0:0]
:tap105i0-IN - [0:0]
:tap105i0-OUT - [0:0]
:tap110i0-IN - [0:0]
:tap110i0-OUT - [0:0]
:tap111i0-IN - [0:0]
:tap111i0-OUT - [0:0]
-A INPUT -j PVEFW-INPUT
-A FORWARD -j PVEFW-FORWARD
-A OUTPUT -j PVEFW-OUTPUT
-A GROUP-default-IN -j MARK --set-xmark 0x0/0x80000000
-A GROUP-default-IN -p icmp -g PVEFW-SET-ACCEPT-MARK
-A GROUP-default-IN -s 192.168.241.0/24 -g PVEFW-SET-ACCEPT-MARK
-A GROUP-default-IN -m comment --comment "PVESIG:FAF/dK3+/B5KBZIymdy8qKJs3Yg"
-A GROUP-default-OUT -j MARK --set-xmark 0x0/0x80000000
-A GROUP-default-OUT -m comment --comment "PVESIG:m40rxGxRolSs4B26P5z+oKHDMIc"
-A GROUP-sip_linux-IN -j MARK --set-xmark 0x0/0x80000000
-A GROUP-sip_linux-IN -p tcp -m set --match-set PVEFW-0-voip_customers-v4 src -m tcp --dport 5060 -g PVEFW-SET-ACCEPT-MARK
-A GROUP-sip_linux-IN -p udp -m set --match-set PVEFW-0-voip_customers-v4 src -m udp --dport 5060 -g PVEFW-SET-ACCEPT-MARK
-A GROUP-sip_linux-IN -p udp -m set --match-set PVEFW-0-voip_customers-v4 src -m udp --dport 10000:20000 -g PVEFW-SET-ACCEPT-MARK
-A GROUP-sip_linux-IN -m comment --comment "PVESIG:WzNfniwbmlQ6NscahgysSUwfeG0"
-A GROUP-sip_linux-OUT -j MARK --set-xmark 0x0/0x80000000
-A GROUP-sip_linux-OUT -m comment --comment "PVESIG:RdQSYuPxngsluiQ9eebC+6m2K6A"
-A PVEFW-Drop -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Drop -j PVEFW-DropBroadcast
-A PVEFW-Drop -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Drop -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
-A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
-A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
-A PVEFW-Drop -m comment --comment "PVESIG:WDy2wbFe7jNYEyoO3QhUELZ4mIQ"
-A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
-A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
-A PVEFW-DropBroadcast -m comment --comment "PVESIG:NyjHNAtFbkH7WGLamPpdVnxHy4w"
-A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
-A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
-A PVEFW-FORWARD -m comment --comment "PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw"
-A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-FWBR-IN -m physdev --physdev-out tap105i0 --physdev-is-bridged -j tap105i0-IN
-A PVEFW-FWBR-IN -m physdev --physdev-out tap110i0 --physdev-is-bridged -j tap110i0-IN
-A PVEFW-FWBR-IN -m physdev --physdev-out tap111i0 --physdev-is-bridged -j tap111i0-IN
-A PVEFW-FWBR-IN -m comment --comment "PVESIG:Ij8EN4EQeCPywRPlpzQDIjwewvY"
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap105i0 --physdev-is-bridged -j tap105i0-OUT
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap110i0 --physdev-is-bridged -j tap110i0-OUT
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap111i0 --physdev-is-bridged -j tap111i0-OUT
-A PVEFW-FWBR-OUT -m comment --comment "PVESIG:xn7FrLiKkMKTfGxnDChBxV96NsU"
-A PVEFW-HOST-IN -i lo -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-HOST-IN -p igmp -j RETURN
-A PVEFW-HOST-IN -s 192.168.241.0/24 -j RETURN
-A PVEFW-HOST-IN -s 10.254.1.0/24 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-IN -s 192.168.241.0/24 -d 192.168.241.0/24 -p udp -m udp --dport 5404:5405 -j RETURN
-A PVEFW-HOST-IN -s 192.168.241.0/24 -p udp -m addrtype --dst-type MULTICAST -m udp --dport 5404:5405 -j RETURN
-A PVEFW-HOST-IN -j PVEFW-Drop
-A PVEFW-HOST-IN -j DROP
-A PVEFW-HOST-IN -m comment --comment "PVESIG:jUe7vq6DlOsIiWzUDZvyIcSkBWY"
-A PVEFW-HOST-OUT -o lo -j ACCEPT
-A PVEFW-HOST-OUT -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-OUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-OUT -p igmp -j RETURN
-A PVEFW-HOST-OUT -d 192.168.241.0/24 -p tcp -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-OUT -d 192.168.241.0/24 -p tcp -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-OUT -d 192.168.241.0/24 -p tcp -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-OUT -d 192.168.241.0/24 -p tcp -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-OUT -d 192.168.241.0/24 -p udp -m udp --dport 5404:5405 -j RETURN
-A PVEFW-HOST-OUT -p udp -m addrtype --dst-type MULTICAST -m udp --dport 5404:5405 -j RETURN
-A PVEFW-HOST-OUT -j RETURN
-A PVEFW-HOST-OUT -m comment --comment "PVESIG:jP0LGwnIAYXdHiudtSs6tGLMv8Y"
-A PVEFW-INPUT -j PVEFW-HOST-IN
-A PVEFW-INPUT -m comment --comment "PVESIG:+5iMmLaxKXynOB/+5xibfx7WhFk"
-A PVEFW-OUTPUT -j PVEFW-HOST-OUT
-A PVEFW-OUTPUT -m comment --comment "PVESIG:LjHoZeSSiWAG3+2ZAyL/xuEehd0"
-A PVEFW-Reject -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Reject -j PVEFW-DropBroadcast
-A PVEFW-Reject -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Reject -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
-A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
-A PVEFW-Reject -m comment --comment "PVESIG:CZJnIN6rAdpu+ej59QPr9+laMUo"
-A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
-A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:Hg/OIgIwJChBUcWU8Xnjhdd2jUY"
-A PVEFW-logflags -j DROP
-A PVEFW-logflags -m comment --comment "PVESIG:MN4PH1oPZeABMuWr64RrygPfW7A"
-A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-reject -s 224.0.0.0/4 -j DROP
-A PVEFW-reject -p icmp -j DROP
-A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
-A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
-A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
-A PVEFW-reject -j REJECT --reject-with icmp-host-prohibited
-A PVEFW-reject -m comment --comment "PVESIG:Jlkrtle1mDdtxDeI9QaDSL++Npc"
-A PVEFW-smurflog -j DROP
-A PVEFW-smurflog -m comment --comment "PVESIG:2gfT1VMkfr0JL6OccRXTGXo+1qk"
-A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
-A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
-A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
-A PVEFW-smurfs -m comment --comment "PVESIG:HssVe5QCBXd5mc9kC88749+7fag"
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
-A PVEFW-tcpflags -m comment --comment "PVESIG:CMFojwNPqllyqD67NeI5m+bP5mo"
-A tap105i0-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A tap105i0-IN -j GROUP-default-IN
-A tap105i0-IN -m mark --mark 0x80000000/0x80000000 -j ACCEPT
-A tap105i0-IN -j GROUP-sip_linux-IN
-A tap105i0-IN -m mark --mark 0x80000000/0x80000000 -j ACCEPT
-A tap105i0-IN -j PVEFW-Drop
-A tap105i0-IN -j DROP
-A tap105i0-IN -m comment --comment "PVESIG:6C/OawhpsGNSAWrk0bmMAIJaYnk"
-A tap105i0-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A tap105i0-OUT -m mac ! --mac-source 5E:D8:F3:62:C5:90 -j DROP
-A tap105i0-OUT -j MARK --set-xmark 0x0/0x80000000
-A tap105i0-OUT -j GROUP-default-OUT
-A tap105i0-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
-A tap105i0-OUT -j GROUP-sip_linux-OUT
-A tap105i0-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
-A tap105i0-OUT -g PVEFW-SET-ACCEPT-MARK
-A tap105i0-OUT -m comment --comment "PVESIG:Oxg9Gg5pHvMArH0hCR40MKSMXUg"
-A tap110i0-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A tap110i0-IN -j ACCEPT
-A tap110i0-IN -m comment --comment "PVESIG:hAo4J2yPT0j2EQimsTiZJ6YEufs"
-A tap110i0-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A tap110i0-OUT -m mac ! --mac-source 7A:9A:D5:49:B8:55 -j DROP
-A tap110i0-OUT -j MARK --set-xmark 0x0/0x80000000
-A tap110i0-OUT -g PVEFW-SET-ACCEPT-MARK
-A tap110i0-OUT -m comment --comment "PVESIG:/I4V8M08xuH3fYjhp3TpJSnJG0Q"
-A tap111i0-IN -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A tap111i0-IN -j ACCEPT
-A tap111i0-IN -m comment --comment "PVESIG:ELkaWVQzeYV73pYWlEVSQ16QF8U"
-A tap111i0-OUT -p udp -m udp --sport 68 --dport 67 -g PVEFW-SET-ACCEPT-MARK
-A tap111i0-OUT -m mac ! --mac-source 16:F3:25:F2:1E:1A -j DROP
-A tap111i0-OUT -j MARK --set-xmark 0x0/0x80000000
-A tap111i0-OUT -g PVEFW-SET-ACCEPT-MARK
-A tap111i0-OUT -m comment --comment "PVESIG:tK6SxVZBe11NMVDzhkvH9m/dKMA"
COMMIT
# Completed on Tue Sep 11 17:13:32 2018
 
Using the iptables setting you've posted we don't have the phenomenon (i.e. packet counter of sent packets to tap105i0-IN is identical to the number in the chain itself).

If you post the complete pvereport qbout your system we may see more.
 
Linux bridge implementation:
/etc/network/interfaces:
Code: 
auto lo
iface lo inet loopback

auto bond0
iface bond0 inet manual
    slaves eth0,eth1
    bond_miimon 100
    bond_mode active-backup
    mtu 9216

auto bond1
iface bond1 inet static
    address 10.254.1.2
    netmask  255.255.255.0
    slaves eth2,eth3
    bond_miimon 100
    bond_mode active-backup
    mtu 9216

auto eth0
iface eth0 inet manual
    bond-master bond0
    bond-primary eth0
    mtu 9216

auto eth1
iface eth1 inet manual
    bond-master bond0
    mtu 9216

auto eth2
iface eth2 inet manual
    bond-master bond1
    mtu 9216

auto eth3
iface eth3 inet manual
    bond-master bond1
    bond-primary eth3
    mtu 9216

auto vmbr0
iface vmbr0 inet static
    address 192.168.241.2
    netmask 255.255.255.0
    gateway 192.168.241.1
    bridge_ports bond0
    bridge_stp off
    bridge_fd 0
    mtu 9216


Open vSwitch implementation:
Install packages:
Code: 
apt-get install openvswitch-switch;
# Useful commands:
# ovs-appctl bond/show bond0;
# ovs-vsctl show;
Notes:
vlan interface MTUs don't need to be smaller than their bridges, as they are interchanged as untagged packets on the bond ports.​

/etc/rc.local:
Code: 
# Set active-backup bond slave interface priority:
ovs-appctl bond/set-active-slave bond0 eth0;
ovs-appctl bond/set-active-slave bond1 eth3;
/etc/network/interfaces:
Code: 
auto lo
iface lo inet loopback

allow-vmbr0 bond0
iface bond0 inet manual
    ovs_bridge vmbr0
    ovs_type OVSBond
    ovs_bonds eth0 eth1
    pre-up ( ifconfig eth0 mtu 9216 && ifconfig eth1 mtu 9216 )
    ovs_options bond_mode=active-backup tag=11 vlan_mode=native-untagged
    mtu 9216

auto vmbr0
allow-ovs vmbr0
iface vmbr0 inet manual
    ovs_type OVSBridge
    ovs_ports bond0 vlan11
    mtu 9216

allow-vmbr0 vlan11
iface vlan11 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_options tag=11
    ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
    address 192.168.241.2
    netmask 255.255.255.0
    gateway 192.168.241.1
    mtu 9216

allow-vmbr1 bond1
iface bond1 inet manual
    ovs_bridge vmbr1
    ovs_type OVSBond
    ovs_bonds eth2 eth3
    pre-up ( ifconfig eth2 mtu 9216 && ifconfig eth3 mtu 9216 )
    ovs_options bond_mode=active-backup tag=200 vlan_mode=native-untagged
    mtu 9216

auto vmbr1
allow-ovs vmbr1
iface vmbr1 inet manual
    ovs_type OVSBridge
    ovs_ports bond1 vlan200
    mtu 9216

allow-vmbr1 vlan200
iface vlan200 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr1
    ovs_options tag=200
    ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
    address 10.254.1.2
    netmask 255.255.255.0
    mtu 9216
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back