[SOLVED] Outbound traffic fully blocked with firewall enabled, but no DROP rules set

A

aljaxus

Active Member
Jul 6, 2019
43
5
28
Them Interwebz
aljaxus.eu
I already replied to another thread, but I think that starting a new one is a bit more appropriate because the details are probably quite different.
aljaxus said:
Sorry for sort of necro-posting, but I have the very same issue. Did you manage to somehow "fix" this or come to any conclusions?
Click to expand...

I also suspect that this issue is very closely related to this one.


All the data I can provide - request more if needed
Code: 
root@apollo:~# pveversion
pve-manager/6.0-9/508dcee0 (running kernel: 5.0.21-3-pve)
Code: 
root@apollo:~# cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface enp3s0f0 inet manual

iface enp3s0f1 inet manual

iface enp4s0f0 inet manual

iface enp4s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
    address  192.168.1.101
    netmask  255.255.255.0
    network     192.168.1.0
    broadcast 192.168.1.255
    gateway  192.168.1.1
    bridge-ports enp3s0f0
    bridge-stp off
    bridge-fd 0

auto vmbr1
iface vmbr1 inet static
    address  10.10.10.1
    netmask  255.255.255.0
    bridge-ports none
    bridge-stp off
    bridge-fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
    # 80 & 443 -> 1052 Centos7-proxy
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 80 -j DNAT --to 10.10.10.52:80
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 80 -j DNAT --to 10.10.10.52:80
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 443 -j DNAT --to 10.10.10.52:443
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 443 -j DNAT --to 10.10.10.52:443
    # 22 -> 152 Centos7-proxy
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 22 -j DNAT --to 10.10.10.52:22
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 22 -j DNAT --to 10.10.10.52:22

auto vmbr2
iface vmbr2 inet static
    address  10.10.20.1
    netmask  255.255.255.0
    bridge-ports none
    bridge-stp off
    bridge-fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.20.0/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.20.0/24' -o vmbr0 -j MASQUERADE

    # 25565 -> 2056 Centos7-TNsrv
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 25565 -j DNAT --to 10.10.20.56:25565
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 25565 -j DNAT --to 10.10.20.56:25565
Code: 
root@apollo:~# cat /etc/pve/firewall/cluster.fw
[OPTIONS]

policy_in: ACCEPT
enable: 1
ebtables: 1
log_ratelimit: burst=5,enable=1,rate=1/second

[ALIASES]

localNAT 10.0.0.0/8 # The local NAT-ted network

[group allowall] # Allow all internal traffic

OUT ACCEPT -log debug
IN ACCEPT -log debug
Code: 
root@apollo:~# cat /etc/pve/nodes/apollo/host.fw
[OPTIONS]

log_level_in: debug
smurf_log_level: debug
log_level_out: debug
enable: 1
tcp_flags_log_level: debug

[RULES]

GROUP allowall -i lo
GROUP allowall -i vmbr0
GROUP allowall -i vmbr1
GROUP allowall -i vmbr2
 
Last edited:
Code: 
root@apollo:~# pve-firewall status && iptables-save
Status: enabled/running
# Generated by iptables-save v1.8.2 on Sun Nov  3 19:29:09 2019
*filter
:INPUT ACCEPT [251:16272]
:FORWARD ACCEPT [5834:658974]
:OUTPUT ACCEPT [4:302]
:GROUP-allowall-IN - [0:0]
:GROUP-allowall-OUT - [0:0]
:PVEFW-Drop - [0:0]
:PVEFW-DropBroadcast - [0:0]
:PVEFW-FORWARD - [0:0]
:PVEFW-FWBR-IN - [0:0]
:PVEFW-FWBR-OUT - [0:0]
:PVEFW-HOST-IN - [0:0]
:PVEFW-HOST-OUT - [0:0]
:PVEFW-INPUT - [0:0]
:PVEFW-OUTPUT - [0:0]
:PVEFW-Reject - [0:0]
:PVEFW-SET-ACCEPT-MARK - [0:0]
:PVEFW-logflags - [0:0]
:PVEFW-reject - [0:0]
:PVEFW-smurflog - [0:0]
:PVEFW-smurfs - [0:0]
:PVEFW-tcpflags - [0:0]
-A INPUT -j PVEFW-INPUT
-A FORWARD -j PVEFW-FORWARD
-A OUTPUT -j PVEFW-OUTPUT
-A GROUP-allowall-IN -j MARK --set-xmark 0x0/0x80000000
-A GROUP-allowall-IN -m limit --limit 1/sec -j NFLOG --nflog-prefix  ":0:7:GROUP-allowall-IN: "
-A GROUP-allowall-IN -g PVEFW-SET-ACCEPT-MARK
-A GROUP-allowall-IN -m comment --comment "PVESIG:vl3CPIuUJuVbb4soq/AlFTzZldA"
-A GROUP-allowall-OUT -j MARK --set-xmark 0x0/0x80000000
-A GROUP-allowall-OUT -m limit --limit 1/sec -j NFLOG --nflog-prefix  ":0:7:GROUP-allowall-OUT: "
-A GROUP-allowall-OUT -g PVEFW-SET-ACCEPT-MARK
-A GROUP-allowall-OUT -m comment --comment "PVESIG:+di+Xg5Aye6T0ubL5IO+AUpyad0"
-A PVEFW-Drop -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Drop -j PVEFW-DropBroadcast
-A PVEFW-Drop -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Drop -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
-A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
-A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
-A PVEFW-Drop -m comment --comment "PVESIG:WDy2wbFe7jNYEyoO3QhUELZ4mIQ"
-A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
-A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
-A PVEFW-DropBroadcast -m comment --comment "PVESIG:NyjHNAtFbkH7WGLamPpdVnxHy4w"
-A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
-A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
-A PVEFW-FORWARD -m comment --comment "PVESIG:qnNexOcGa+y+jebd4dAUqFSp5nw"
-A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-FWBR-IN -m comment --comment "PVESIG:Ijl7/xz0DD7LF91MlLCz0ybZBE0"
-A PVEFW-FWBR-OUT -m comment --comment "PVESIG:2jmj7l5rSw0yVb/vlWAYkK/YBwk"
-A PVEFW-HOST-IN -i lo -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-HOST-IN -p igmp -j RETURN
-A PVEFW-HOST-IN -i vmbr0 -j GROUP-allowall-IN
-A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
-A PVEFW-HOST-IN -i vmbr1 -j GROUP-allowall-IN
-A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
-A PVEFW-HOST-IN -i vmbr2 -j GROUP-allowall-IN
-A PVEFW-HOST-IN -m mark --mark 0x80000000/0x80000000 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-IN -j RETURN
-A PVEFW-HOST-IN -m comment --comment "PVESIG:K6jvIZSaM9WDPqH1Zm8v3VaENPo"
-A PVEFW-HOST-OUT -o lo -j ACCEPT
-A PVEFW-HOST-OUT -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-OUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-OUT -p igmp -j RETURN
-A PVEFW-HOST-OUT -o vmbr0 -j GROUP-allowall-OUT
-A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
-A PVEFW-HOST-OUT -o vmbr1 -j GROUP-allowall-OUT
-A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
-A PVEFW-HOST-OUT -o vmbr2 -j GROUP-allowall-OUT
-A PVEFW-HOST-OUT -m mark --mark 0x80000000/0x80000000 -j RETURN
-A PVEFW-HOST-OUT -d 192.168.1.0/24 -p tcp -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-OUT -d 192.168.1.0/24 -p tcp -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-OUT -d 192.168.1.0/24 -p tcp -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-OUT -d 192.168.1.0/24 -p tcp -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-OUT -j RETURN
-A PVEFW-HOST-OUT -m comment --comment "PVESIG:LaOYsYRoTSQThfBS/k+Zuh/NElk"
-A PVEFW-INPUT -j PVEFW-HOST-IN
-A PVEFW-INPUT -m comment --comment "PVESIG:+5iMmLaxKXynOB/+5xibfx7WhFk"
-A PVEFW-OUTPUT -j PVEFW-HOST-OUT
-A PVEFW-OUTPUT -m comment --comment "PVESIG:LjHoZeSSiWAG3+2ZAyL/xuEehd0"
-A PVEFW-Reject -p tcp -m tcp --dport 43 -j PVEFW-reject
-A PVEFW-Reject -j PVEFW-DropBroadcast
-A PVEFW-Reject -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Reject -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
-A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
-A PVEFW-Reject -m comment --comment "PVESIG:CZJnIN6rAdpu+ej59QPr9+laMUo"
-A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
-A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:Hg/OIgIwJChBUcWU8Xnjhdd2jUY"
-A PVEFW-logflags -m limit --limit 1/sec -j NFLOG --nflog-prefix  ":0:7:PVEFW-logflags: DROP: "
-A PVEFW-logflags -j DROP
-A PVEFW-logflags -m comment --comment "PVESIG:no99k4QpAY/9Nh+01O7LyPKYCVE"
-A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-reject -s 224.0.0.0/4 -j DROP
-A PVEFW-reject -p icmp -j DROP
-A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
-A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
-A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
-A PVEFW-reject -j REJECT --reject-with icmp-host-prohibited
-A PVEFW-reject -m comment --comment "PVESIG:Jlkrtle1mDdtxDeI9QaDSL++Npc"
-A PVEFW-smurflog -m limit --limit 1/sec -j NFLOG --nflog-prefix  ":0:7:PVEFW-smurflog: DROP: "
-A PVEFW-smurflog -j DROP
-A PVEFW-smurflog -m comment --comment "PVESIG:oW+BTqwhnmvTgHOc33/APcSI7tg"
-A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
-A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
-A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
-A PVEFW-smurfs -m comment --comment "PVESIG:HssVe5QCBXd5mc9kC88749+7fag"
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
-A PVEFW-tcpflags -m comment --comment "PVESIG:CMFojwNPqllyqD67NeI5m+bP5mo"
COMMIT
# Completed on Sun Nov  3 19:29:09 2019
# Generated by iptables-save v1.8.2 on Sun Nov  3 19:29:09 2019
*nat
:PREROUTING ACCEPT [1140:81597]
:INPUT ACCEPT [3:270]
:OUTPUT ACCEPT [174:10525]
:POSTROUTING ACCEPT [1308:90867]
-A PREROUTING -i vmbr0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.10.10.52:80
-A PREROUTING -i vmbr0 -p tcp -m tcp --dport 443 -j DNAT --to-destination 10.10.10.52:443
-A PREROUTING -i vmbr0 -p tcp -m tcp --dport 22 -j DNAT --to-destination 10.10.10.52:22
-A PREROUTING -i vmbr0 -p tcp -m tcp --dport 25565 -j DNAT --to-destination 10.10.20.56:25565
-A POSTROUTING -s 10.10.10.0/24 -o vmbr0 -j MASQUERADE
-A POSTROUTING -s 10.10.20.0/24 -o vmbr0 -j MASQUERADE
COMMIT
# Completed on Sun Nov  3 19:29:09 2019
root@apollo:~#
 
Screenshot_2019-11-03 apollo - Proxmox Virtual Environment.png Screenshot_2019-11-03 apollo - Proxmox Virtual Environment(1).png Screenshot_2019-11-03 apollo - Proxmox Virtual Environment(5).png
Screenshot_2019-11-03 apollo - Proxmox Virtual Environment(4).png Screenshot_2019-11-03 apollo - Proxmox Virtual Environment(3).png
No firewall is enabled on VM level
This is the VM to which the data is proxied over the firewall
- the traceroute 1.1.1.1 command hangs
Code: 
[root@proxy ~]# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
^C
--- 1.1.1.1 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 4999ms

[root@proxy ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.10.52  netmask 255.255.255.0  broadcast 10.10.10.255
        inet6 fe80::ef9c:389f:bc05:8f7f  prefixlen 64  scopeid 0x20<link>
        ether 8e:a1:7a:43:ed:3b  txqueuelen 1000  (Ethernet)
        RX packets 2219  bytes 612414 (598.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2229  bytes 612952 (598.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 64  bytes 5376 (5.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 64  bytes 5376 (5.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@proxy ~]# traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets

All the VMs can reach each other
Code: 
[root@proxy ~]# ping 10.10.10.1
PING 10.10.10.1 (10.10.10.1) 56(84) bytes of data.
64 bytes from 10.10.10.1: icmp_seq=1 ttl=64 time=0.117 ms
64 bytes from 10.10.10.1: icmp_seq=2 ttl=64 time=0.226 ms
64 bytes from 10.10.10.1: icmp_seq=3 ttl=64 time=0.222 ms
64 bytes from 10.10.10.1: icmp_seq=4 ttl=64 time=0.254 ms
64 bytes from 10.10.10.1: icmp_seq=5 ttl=64 time=0.249 ms
^C
--- 10.10.10.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4000ms
rtt min/avg/max/mdev = 0.117/0.213/0.254/0.052 ms
Code: 
[root@proxy ~]# ping 10.10.10.53
PING 10.10.10.53 (10.10.10.53) 56(84) bytes of data.
64 bytes from 10.10.10.53: icmp_seq=1 ttl=64 time=0.806 ms
64 bytes from 10.10.10.53: icmp_seq=2 ttl=64 time=0.599 ms
64 bytes from 10.10.10.53: icmp_seq=3 ttl=64 time=0.618 ms
64 bytes from 10.10.10.53: icmp_seq=4 ttl=64 time=0.681 ms
64 bytes from 10.10.10.53: icmp_seq=5 ttl=64 time=0.579 ms
^C
--- 10.10.10.53 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4001ms
rtt min/avg/max/mdev = 0.579/0.656/0.806/0.086 ms
Code: 
[root@proxy ~]# ping 10.10.20.22
PING 10.10.20.22 (10.10.20.22) 56(84) bytes of data.
64 bytes from 10.10.20.22: icmp_seq=1 ttl=63 time=0.394 ms
64 bytes from 10.10.20.22: icmp_seq=2 ttl=63 time=0.601 ms
64 bytes from 10.10.20.22: icmp_seq=3 ttl=63 time=0.561 ms
64 bytes from 10.10.20.22: icmp_seq=4 ttl=63 time=0.596 ms
64 bytes from 10.10.20.22: icmp_seq=5 ttl=63 time=0.600 ms
64 bytes from 10.10.20.22: icmp_seq=6 ttl=63 time=0.593 ms
^C
--- 10.10.20.22 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5000ms
rtt min/avg/max/mdev = 0.394/0.557/0.601/0.078 ms


Conclusion;
  1. All VMs can reach each other and the host
  2. Host can reach all VMs and outside network (my local home network - 192.168.1.0/24)
  3. This issue is only active when the firewall is enabled
  4. Firewall doesn't have any DROP or REJECT rules, only ACCEPT rules

  5. I think that the only logical explanation for this that I can come up with my limited knowledge about PVE and pve-firewall is that the vmbr0 or enp3s0f0 interface(-s) have some firewall rules set that block the outbound traffic and I am not aware of them.
I would really appreciate any suggestions, questions, help because this issue is blocking progress for a few days now.
Kind regards, Aljaz S.
 
Last edited:
did you work out what this issue was I seem to have a similar issue?
 
Update; executing iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 on the host machine (root @ PVE host) "fixes" the issue.
Source: https://forum.proxmox.com/threads/f...also-have-to-reboot-to-fix.59811/#post-275871

I do not fully understand the issue and I do not know about any "correct" fix (in case that there's a better way of doing this)
Will keep this post updated in case anyone else has the same issue in the future and stumbles upon this post.
Regards, Aljaz S.

(edit1)
I also updated my /etc/network/interfaces file to the following
Code: 
root@apollo:~# cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface enp3s0f0 inet manual

iface enp3s0f1 inet manual

iface enp4s0f0 inet manual

iface enp4s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
    address  192.168.1.101
    netmask  255.255.255.0
    network     192.168.1.0
    broadcast 192.168.1.255
    gateway  192.168.1.1
    bridge-ports enp3s0f0
    bridge-stp off
    bridge-fd 0

auto vmbr1
iface vmbr1 inet static
    address  10.10.10.1
    netmask  255.255.255.0
    bridge-ports none
    bridge-stp off
    bridge-fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
        post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
    # 80 & 443 -> 1052 Centos7-proxy
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 80 -j DNAT --to 10.10.10.52:80
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 80 -j DNAT --to 10.10.10.52:80
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 443 -j DNAT --to 10.10.10.52:443
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 443 -j DNAT --to 10.10.10.52:443
    # 22 -> 152 Centos7-proxy
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 22 -j DNAT --to 10.10.10.52:22
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 22 -j DNAT --to 10.10.10.52:22

auto vmbr2
iface vmbr2 inet static
    address  10.10.20.1
    netmask  255.255.255.0
    bridge-ports none
    bridge-stp off
    bridge-fd 0

        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.20.0/24' -o vmbr0 -j MASQUERADE
        post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
        post-down iptables -t nat -D POSTROUTING -s '10.10.20.0/24' -o vmbr0 -j MASQUERADE

    # 25565 -> 2056 Centos7-TNsrv
    post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 25565 -j DNAT --to 10.10.20.56:25565
    post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 25565 -j DNAT --to 10.10.20.56:25565

(edit2) I just rebooted the host server after enabling the datacenter-level firewall, tested for open ports using nmap and it all works.
Code: 
[aljaxus@aljaxus ~]$ nmap 192.168.1.101 -Pn
Starting Nmap 7.80 ( https://nmap.org ) at 2019-11-06 18:59 CET
Nmap scan report for 192.168.1.101
Host is up (0.0042s latency).
Not shown: 997 closed ports
PORT    STATE SERVICE
22/tcp  open  ssh
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 2.60 seconds
[aljaxus@aljaxus ~]$

Code: 
[root@proxy ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.10.52  netmask 255.255.255.0  broadcast 10.10.10.255
        inet6 fe80::ef9c:389f:bc05:8f7f  prefixlen 64  scopeid 0x20<link>
        ether 8e:a1:7a:43:ed:3b  txqueuelen 1000  (Ethernet)
        RX packets 3413  bytes 1030032 (1005.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3679  bytes 1050053 (1.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 64  bytes 5376 (5.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 64  bytes 5376 (5.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@proxy ~]# ping google.com
PING google.com (172.217.16.142) 56(84) bytes of data.
64 bytes from zrh04s06-in-f142.1e100.net (172.217.16.142): icmp_seq=1 ttl=54 time=25.1 ms
64 bytes from zrh04s06-in-f142.1e100.net (172.217.16.142): icmp_seq=2 ttl=54 time=24.3 ms
64 bytes from zrh04s06-in-f142.1e100.net (172.217.16.142): icmp_seq=3 ttl=54 time=24.1 ms
64 bytes from zrh04s06-in-f142.1e100.net (172.217.16.142): icmp_seq=4 ttl=54 time=25.7 ms
64 bytes from zrh04s06-in-f142.1e100.net (172.217.16.142): icmp_seq=5 ttl=54 time=23.5 ms
64 bytes from zrh04s06-in-f142.1e100.net (172.217.16.142): icmp_seq=6 ttl=54 time=26.3 ms
^C
--- google.com ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5005ms
rtt min/avg/max/mdev = 23.584/24.891/26.399/0.968 ms
[root@proxy ~]# curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
[root@proxy ~]#
 
Last edited:
Sorry to rehash this thread, but had a similar issue with a Cisco VIC 1227 network card. I had to setup the VLAN Aware setting on the bridge devices on the host networking section to make everything work correctly. I am not using VLANs, but I believe the VIC 1227 is using them internally which is why this must be enabled. May be a similar case for you as well.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back