1 public IP, two bridges, port forwarding

leonidas_o

Member
Apr 17, 2022
68
5
8
Hello Guys,
currently running proxmox 7.2-3 on a dedicated server. I got only one public ip (eno1) and two bridges (vmbr0, vmbr1). The idea is to have some VM's over vmbr0 (more management like tooling) and some others over vmbr1. Does that even make sense to set it up that way?

I tried to setup the port forwarding, but I can't reach the services on the VM. The VM is a rocky linux (selinux enabled). Datecenter firewall is On and has the entries IN ACCEPT IMAP, IN ACCEPT IMAPS, ... etc. . Firewall on the Node is on as well, no further rules specified.
The Mailserver VM is using vmbr1. At the beginning I had the SMTP, IMAP port forwardings only on vmbr1, then added them to vmbr0 as well.

My /etc/network/interfaces file looks like:

Bash:
auto eno1
iface eno1 inet dhcp

auto eno2
iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.2.1.1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.2.1.0/24' -o eno1 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.2.1.0/24' -o eno1 -j MASQUERADE
        post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
        post-down iptables -t raw -D PREROUTING  -i fwbr+ -j CT --zone 1
        #post-up   iptables -t raw -I POSTROUTING -i fwbr+ -j CT --zone 1
        #post-down iptables -t raw -D POSTROUTING -i fwbr+ -j CT --zone 1
        # port forwarding
        post-up   iptables -t nat -A PREROUTING -i eno1 -p udp --dport 51820 -j DNAT --to 10.2.1.2:51820
        post-down iptables -t nat -D PREROUTING -i eno1 -p udp --dport 51820 -j DNAT --to 10.2.1.2:51820
        post-up   iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 2222 -j DNAT --to 10.2.1.2:22
        post-down iptables -t nat -D PREROUTING -i eno1 -p tcp --dport 2222 -j DNAT --to 10.2.1.2:22
        # SMTP
        post-up   iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 25 -j DNAT --to 10.2.2.2:25
        post-down iptables -t nat -D PREROUTING -i eno1 -p tcp --dport 25 -j DNAT --to 10.2.2.2:25
        # SMTPS
        ...
        # Submission
        ...
        # IMAP
        ...
        # IMAPS
        ...

auto vmbr1
iface vmbr1 inet static
        address 10.2.2.1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.2.2.0/24' -o eno1 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.2.2.0/24' -o eno1 -j MASQUERADE
        # port forwarding
        # SMTP
        post-up   iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 25 -j DNAT --to 10.2.2.2:25
        post-down iptables -t nat -D PREROUTING -i eno1 -p tcp --dport 25 -j DNAT --to 10.2.2.2:25
        # SMTPS
        ...
        # Submission
        ...
        # IMAP
        ...
        # IMAPS
        ...

The mailserver setup, DNS settings etc. seems to be okay. I can receive emails, I can send emails via webmail. I just can't setup my local email client (Apple Mail), it shows: "Unable to verify account name or password".

Screenshot 2022-06-13 at 12.11.59.png

I'm pretty sure this setup here between eno1, eno2, vmbr0, vmbr1 and the fact that I got only one public IP is simply wrong. So I would need a little help here.

Thanks a lot.
 
Last edited:
Hi,

for clarification: eno1 is the interface that gets a public IP? Did you check if you can reach the ports for IMAP(S) for example via something like telnet <hostname> <port>?
 
Hi,

for clarification: eno1 is the interface that gets a public IP? Did you check if you can reach the ports for IMAP(S) for example via something like telnet <hostname> <port>?
yes, eno1 has the public IP.
I checked with netcat:

nc -vz HOSTNAME 143
nv -vz IP_OF_HOST 143
nv -vz IP_OF_VM 143
I get: nc: connectx to HOSTNAME port 143 (tcp) failed: Connection refused. The same error message for IP_OF_HOST. When using IP_OF_VM (so the internal IP address when inside VPN) that works: Connection to 10.2.2.2 port 143 [tcp/imap] succeeded!
 
Hi,

let's first check if you can connect to it if you disable the firewall completely. If this fixe it, it should just playing with the rules a bit.
 
The Datacenter rules are so simple there is not much room to "play" with them.

Screenshot 2022-06-13 at 13.29.17.png

Nevertheless, I deactivated the firewall on Datacenter and on the Node to check your point (haven't restarted the Node, shouldn't be required, I guess). Got the same result as above.
I don't think it's the firewall, rather the network device-, linux bridge setup. My setup has no bridge ports for any of the vmbr's, no gateway. I don't know if that could be the culprit, or the /etc/network/interfaces file in combination with that.
 
Last edited:
ah,

i think I'm getting blind ... :). You have two rules:

Code:
#vmbr0
post-up   iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 25 -j DNAT --to 10.2.2.2:25

#vmbr1
post-up   iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 25 -j DNAT --to 10.2.2.2:25

They both tell the system to listen on its public interface on port 25. I'm not sure if this is a typo here, but could it be that you have multiple rules for the same port?
 
No it's not a typo, rather a lack of network knowledge on my side :confused:
That's what I tried to express in the first post of this thread:

"The Mailserver VM is using vmbr1. At the beginning I had the SMTP, IMAP port forwardings only on vmbr1, then added them to vmbr0 as well."

But the behaviour was the same when I had that rule only on vmbr1.
 
Did you try to restart the system as well at some point?

I would try to clean up the duplicated entries, else this gets confusing if you have to edit two entries ;).

If restarting doesn't help, you could post the output of iptables-save -f <filename> as a file here.
 
Of course, on any change of the /etc/network/interfaces file, I restart the whole server. So if I now remove the duplicate entry:
#vmbr0
post-up iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 25 -j DNAT --to 10.2.2.2:25

and restart the system, then I know for sure, this won't help, because this is then the "initial point" where I started. As said in the opening post, no duplicate entry, only the entries on vmbr1 and then of course I restarted the server. It wasn't working, so I started to test a few things, added the same port forwarding rules to vmbr0 etc. So basically where I'm now stuck. Nothing was working, so I opened the post here.
I will remove the duplicate entries, before the next restart, but I don't want to restart the server just to remove the duplicates right now, because there are other VM's running which I'm using.

Any other ideas?


The iptables output (still with the duplicates):

Bash:
# Generated by iptables-save v1.8.7 on Mon Jun 13 14:39:47 2022
*filter
:INPUT ACCEPT [158:9920]
:FORWARD ACCEPT [70345:4193162]
:OUTPUT ACCEPT [124:9376]
:PVEFW-Drop - [0:0]
:PVEFW-DropBroadcast - [0:0]
:PVEFW-FORWARD - [0:0]
:PVEFW-FWBR-IN - [0:0]
:PVEFW-FWBR-OUT - [0:0]
:PVEFW-HOST-IN - [0:0]
:PVEFW-HOST-OUT - [0:0]
:PVEFW-INPUT - [0:0]
:PVEFW-OUTPUT - [0:0]
:PVEFW-Reject - [0:0]
:PVEFW-SET-ACCEPT-MARK - [0:0]
:PVEFW-logflags - [0:0]
:PVEFW-reject - [0:0]
:PVEFW-smurflog - [0:0]
:PVEFW-smurfs - [0:0]
:PVEFW-tcpflags - [0:0]
:f2b-sshd - [0:0]
-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
-A INPUT -j PVEFW-INPUT
-A FORWARD -j PVEFW-FORWARD
-A OUTPUT -j PVEFW-OUTPUT
-A PVEFW-Drop -j PVEFW-DropBroadcast
-A PVEFW-Drop -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Drop -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Drop -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Drop -p udp -m multiport --dports 135,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 137:139 -j DROP
-A PVEFW-Drop -p udp -m udp --sport 137 --dport 1024:65535 -j DROP
-A PVEFW-Drop -p tcp -m multiport --dports 135,139,445 -j DROP
-A PVEFW-Drop -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Drop -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Drop -p udp -m udp --sport 53 -j DROP
-A PVEFW-Drop -m comment --comment "PVESIG:83W..."
-A PVEFW-DropBroadcast -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type MULTICAST -j DROP
-A PVEFW-DropBroadcast -m addrtype --dst-type ANYCAST -j DROP
-A PVEFW-DropBroadcast -d 224.0.0.0/4 -j DROP
-A PVEFW-DropBroadcast -m comment --comment "PVESIG:Ny..."
-A PVEFW-FORWARD -m conntrack --ctstate INVALID -j DROP
-A PVEFW-FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-FORWARD -m physdev --physdev-in fwln+ --physdev-is-bridged -j PVEFW-FWBR-IN
-A PVEFW-FORWARD -m physdev --physdev-out fwln+ --physdev-is-bridged -j PVEFW-FWBR-OUT
-A PVEFW-FORWARD -m comment --comment "PVESIG:qnN..."
-A PVEFW-FWBR-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-FWBR-IN -m comment --comment "PVESIG:Ij..."
-A PVEFW-FWBR-OUT -m comment --comment "PVESIG:2j..."
-A PVEFW-HOST-IN -i lo -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-IN -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-IN -m conntrack --ctstate INVALID,NEW -j PVEFW-smurfs
-A PVEFW-HOST-IN -p igmp -j RETURN
-A PVEFW-HOST-IN -p tcp -m tcp --dport 993 -j RETURN
-A PVEFW-HOST-IN -p tcp -m tcp --dport 143 -j RETURN
-A PVEFW-HOST-IN -p tcp -m tcp --dport 587 -j RETURN
-A PVEFW-HOST-IN -p tcp -m tcp --dport 465 -j RETURN
-A PVEFW-HOST-IN -p tcp -m tcp --dport 25 -j RETURN
-A PVEFW-HOST-IN -i vmbr0 -p tcp -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-IN -p udp -m udp --sport 51820 --dport 51820 -j RETURN
-A PVEFW-HOST-IN -p tcp -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-IN -p tcp -m set --match-set PVEFW-0-management-v4 src -m tcp --dport 60000:60050 -j RETURN
-A PVEFW-HOST-IN -j PVEFW-Drop
-A PVEFW-HOST-IN -j DROP
-A PVEFW-HOST-IN -m comment --comment "PVESIG:z7B..."
-A PVEFW-HOST-OUT -o lo -j ACCEPT
-A PVEFW-HOST-OUT -m conntrack --ctstate INVALID -j DROP
-A PVEFW-HOST-OUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A PVEFW-HOST-OUT -p igmp -j RETURN
-A PVEFW-HOST-OUT -d SECRETIPADDRESS/24 -p tcp -m tcp --dport 8006 -j RETURN
-A PVEFW-HOST-OUT -d SECRETIPADDRESS/24 -p tcp -m tcp --dport 22 -j RETURN
-A PVEFW-HOST-OUT -d SECRETIPADDRESS/24 -p tcp -m tcp --dport 5900:5999 -j RETURN
-A PVEFW-HOST-OUT -d SECRETIPADDRESS/24 -p tcp -m tcp --dport 3128 -j RETURN
-A PVEFW-HOST-OUT -j RETURN
-A PVEFW-HOST-OUT -m comment --comment "PVESIG:sgkB..."
-A PVEFW-INPUT -j PVEFW-HOST-IN
-A PVEFW-INPUT -m comment --comment "PVESIG:+5iM..."
-A PVEFW-OUTPUT -j PVEFW-HOST-OUT
-A PVEFW-OUTPUT -m comment --comment "PVESIG:LjHo..."
-A PVEFW-Reject -j PVEFW-DropBroadcast
-A PVEFW-Reject -p icmp -m icmp --icmp-type 3/4 -j ACCEPT
-A PVEFW-Reject -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A PVEFW-Reject -m conntrack --ctstate INVALID -j DROP
-A PVEFW-Reject -p udp -m multiport --dports 135,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 137:139 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --sport 137 --dport 1024:65535 -j PVEFW-reject
-A PVEFW-Reject -p tcp -m multiport --dports 135,139,445 -j PVEFW-reject
-A PVEFW-Reject -p udp -m udp --dport 1900 -j DROP
-A PVEFW-Reject -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
-A PVEFW-Reject -p udp -m udp --sport 53 -j DROP
-A PVEFW-Reject -m comment --comment "PVESIG:h3Dy..."
-A PVEFW-SET-ACCEPT-MARK -j MARK --set-xmark 0x80000000/0x80000000
-A PVEFW-SET-ACCEPT-MARK -m comment --comment "PVESIG:Hg..."
-A PVEFW-logflags -j DROP
-A PVEFW-logflags -m comment --comment "PVESIG:MN4..."
-A PVEFW-reject -m addrtype --dst-type BROADCAST -j DROP
-A PVEFW-reject -s 224.0.0.0/4 -j DROP
-A PVEFW-reject -p icmp -j DROP
-A PVEFW-reject -p tcp -j REJECT --reject-with tcp-reset
-A PVEFW-reject -p udp -j REJECT --reject-with icmp-port-unreachable
-A PVEFW-reject -p icmp -j REJECT --reject-with icmp-host-unreachable
-A PVEFW-reject -j REJECT --reject-with icmp-host-prohibited
-A PVEFW-reject -m comment --comment "PVESIG:Jl..."
-A PVEFW-smurflog -j DROP
-A PVEFW-smurflog -m comment --comment "PVESIG:2g..."
-A PVEFW-smurfs -s 0.0.0.0/32 -j RETURN
-A PVEFW-smurfs -m addrtype --src-type BROADCAST -g PVEFW-smurflog
-A PVEFW-smurfs -s 224.0.0.0/4 -g PVEFW-smurflog
-A PVEFW-smurfs -m comment --comment "PVESIG:Hs..."
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -g PVEFW-logflags
-A PVEFW-tcpflags -p tcp -m tcp --sport 0 --tcp-flags FIN,SYN,RST,ACK SYN -g PVEFW-logflags
-A PVEFW-tcpflags -m comment --comment "PVESIG:CM..."
-A f2b-sshd -j RETURN
COMMIT
# Completed on Mon Jun 13 14:39:47 2022
# Generated by iptables-save v1.8.7 on Mon Jun 13 14:39:47 2022
*raw
:PREROUTING ACCEPT [6229788:3818047653]
:OUTPUT ACCEPT [96755:70068570]
-A PREROUTING -i fwbr+ -j CT --zone 1
COMMIT
# Completed on Mon Jun 13 14:39:47 2022
# Generated by iptables-save v1.8.7 on Mon Jun 13 14:39:47 2022
*nat
:PREROUTING ACCEPT [1045599:68076067]
:INPUT ACCEPT [1981:138716]
:OUTPUT ACCEPT [1640:116623]
:POSTROUTING ACCEPT [849823:52764120]
-A PREROUTING -i eno1 -p udp -m udp --dport 51820 -j DNAT --to-destination 10.2.1.2:51820
-A PREROUTING -i eno1 -p tcp -m tcp --dport 2222 -j DNAT --to-destination 10.2.1.2:22
-A PREROUTING -i eno1 -p tcp -m tcp --dport 25 -j DNAT --to-destination 10.2.2.2:25
-A PREROUTING -i eno1 -p tcp -m tcp --dport 465 -j DNAT --to-destination 10.2.2.2:465
-A PREROUTING -i eno1 -p tcp -m tcp --dport 587 -j DNAT --to-destination 10.2.2.2:587
-A PREROUTING -i eno1 -p tcp -m tcp --dport 143 -j DNAT --to-destination 10.2.2.2:143
-A PREROUTING -i eno1 -p tcp -m tcp --dport 993 -j DNAT --to-destination 10.2.2.2:993
-A PREROUTING -i eno1 -p tcp -m tcp --dport 25 -j DNAT --to-destination 10.2.2.2:25
-A PREROUTING -i eno1 -p tcp -m tcp --dport 465 -j DNAT --to-destination 10.2.2.2:465
-A PREROUTING -i eno1 -p tcp -m tcp --dport 587 -j DNAT --to-destination 10.2.2.2:587
-A PREROUTING -i eno1 -p tcp -m tcp --dport 143 -j DNAT --to-destination 10.2.2.2:143
-A PREROUTING -i eno1 -p tcp -m tcp --dport 993 -j DNAT --to-destination 10.2.2.2:993
-A POSTROUTING -s 10.2.1.0/24 -o eno1 -j MASQUERADE
-A POSTROUTING -s 10.2.2.0/24 -o eno1 -j MASQUERADE
COMMIT
# Completed on Mon Jun 13 14:39:47 2022
 
Just wondering if you might be better off installing a dedicated VM to handle all your routing and NAT requirements. Something like pfsense is a common option and you may find it easier to manage than getting your hands dirty with iptables commands.

If you're working on a hosted setup, which I suspect is the case, then you will still need to have some basic iptables rules to route all external traffic via eno1, but once that's setup and working, running things like a mailserver behind the firewall should be pretty straightforward.
 
I thought about installing opnsense a while ago, but I'm not sure if it's worth it. I know there is a ton of features with pfsense/ opnsense but still, I'm not sure If I should go that route. That would mean redirect everything through the interface to the VM (pfsense/opnsense) and then redirecting to other VMs or the back to the host. I think it would make more sense to put a dedicated machine straight in front of the promox server, but that's not planed for now.
Messing around with iptables commands is just when setting it up, once it's done, you actually don't touch that anymore. So in my case it's actually just port-forwarding. That's actually not a big deal.
I think just the initial vmbr0 and vmbr1 setup has to be done correctly and then the few port forwardings. hmm don't know if it's worth it to setup pfsense/opnsense.

Nevertheless, is it even possible to setup two or more linux bridges (vmbr0, vmbr1, maybe even more) with just one network device? Somehow It doesn't look like that.
 
I got some new findings:

1. I adapted and also removed vmbr1 completely from the interfaces file, also changed the IP and set vmbr0 for the mailserver VM. Then I rebooted and with that I killed my whole proxmox server. The culprit here was the change of the entry bridge-ports none to bridge-ports eno1. After the reboot, It simply didn't come up. I had to activate the remote access terminal to fix/ revert the interfaces file back to bridge-ports none. Can someone explain why is that completely killing the network connection on the server?
I remember a lot of "entered blocking state" messages on the console.

Bash:
auto lo
iface lo inet loopback

auto eno1
iface eno1 inet dhcp

auto eno2
iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.2.1.1/24
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        ...


2. Now with the mailserver on vmbr0, I have the same issue as before on vmbr1. I don't know why I had the idea, but I disconnected from the VPN and suddenly nc -vz HOSTNAME 143 was working. With disconnected VPN I could setup my local Email App as well. As soon as I connect to the vpn again, the connection dies. Which is pretty strange, because I'm redirecting everything through the VPN and when using netcat, I'm using the hostname which is setup in my domain providers DNS settings. I can do a "dig" on that hostname and it shows the correct IP address. Something must be blocking it. But I had the firewall already completely disabled (see above when I had the setup with vmbr1). This is pretty strange, any ideas on what could cause that behaviour? Also, right now, it is not working anymore, both netcat and the email app is not able to connect, with or without vpn, but haven't changed or restarted the server or done anything except going to bed yesterday night.


3. I beginn to believe that my vmbr1 setup was correct and that if I had disconnected from vpn it would work like it is working now with vmbr0 and disconnected vpn. I think I will test that again. But that means I can setup e.g. vmbr0 - vmbr100 or even more if I want to and separate the VM's via these bridges. Is that correct? Is there any kind of security benefit when doing that with these virtual bridges? Or is it simply a structural benefit?
 
Last edited:
1.)
If you assign an interface to a bridge, it doesn't get its own IP address. Only the bridge interface gets an IP in that case. In your case, the only IP was 10.2.1.1 which can't be reached from the internet.

2.)
This depends on your VPN config why you can't reach your server. I would guess then. Do you have the right routes to reach the server on your client?

3.)
I also would guess, that it would have worked :). The security aspect depends on what you are doing. It would be a lot of configuration if you set up a brdige for every VM.
 
1.a: okay that means, I can assign it like that, but I would have to "not overwrite" the IP address which was assigned via dhcp for eno1? So something like:
Code:
auto vmbr0
iface vmbr0 inet dhcp
        bridge_ports eno1
        bridge_stp off
        bridge_fd 0
        
        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        ...

correct?

1.b: I got another network device (eno2), what can I actually do with that, what are the use cases for such a setup with only one public IP addresses but two network devices? (I'm lacking the network part almost entirely, so sorry if I come up with stupid questions, coming from the dev, devOps side)


2. No as clarified in the last sentence in (2), I can't connect anymore, don't know why, with or without vpn. How can I troubleshoot that? Btw. I got another port forward (post-up iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 2222 -j DNAT --to 10.2.1.2:22) for ssh, which is working only when not in vpn. I should solve that maybe first, as it is for sure related to mailserver port forward issue. So speaking for the ssh connection issue, which always works when vpn=off, it looks to me, that the port forward rules are somehow just working when coming from outside. But when vpn=on, I'm actually inside the network, so if now trying to execute the ssh command, I get a timeout, maybe the it needs a separate post forward for such internal requests?


3. I wouldn't create a bridge for each VM, that sounds a bit crazy, but just a few bridges
 
1.)
Did you take a look at this?
https://pve.proxmox.com/wiki/Network_Configuration

For a NAT configuration, you can't just have vmbr0 get an IP address via DHCP. The host needs to be in the same subnet to be able to NAT translate it.

1b.)
For example: you could have a local network with the second IP so two servers can talk to each other on some internal network, for example in some cluster setup, or accessing storage that isn't available on the public internet.

2.)
Is the VPN running on the proxmox host, or are you running it in a VM/container? It might need its own NAT&port forward rule if its running on the host. You are currently only translating traffic comming from eno1
 
Sure, I did, one of the first doc pages I read when setting up proxmox.

No, the vpn is on a VM. I'm currently messing around with Iptable rules.

First I tried removing the interface completely (iptables -t nat -A PREROUTING -p tcp --dport 143 -j DNAT --to 10.2.1.10:143), I thought maybe that means, do the port forward without looking at the interface. But this results in (vpn=on) not being able to connect to either my own mailserver, nor an external mailserver. My Email app, just can't reach any of the email servers anymore.

Then reverted back to iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 143 -j DNAT --to 10.2.1.10:143 and added iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 143 -j DNAT --to 10.2.1.10:143. So now I got eno1 and vmbr0. The netcat command is working with and without vpn. My local Email app can connect to this mailsver BUT it can't connect to an external email server anymore.

So the solution was to add the destination flag when matching (for the internal bridge when inside VPN). Therefore two rules, one for eno1 when coming from outside, when vpn is off (which is still strange, that it just worked once yesterday night and then stopped working) and one rule for vmbr0 where the vpn VM is operating at.

Code:
# IMAP
post-up   iptables -t nat -A PREROUTING -i eno1 -p tcp --dport 143 -j DNAT --to 10.2.2.2:143
post-down iptables -t nat -D PREROUTING -i eno1 -p tcp --dport 143 -j DNAT --to 10.2.2.2:143
post-up   iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 143 -d MYHOSTNAME -j DNAT --to 10.2.1.10:143
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 143 -d MYHOSTNAME -j DNAT --to 10.2.2.2:143
...
 
  • Like
Reactions: shrdlicka

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!