Best Way to Port Forward on Single IP Dev Node

[PMoxSudo]

Hello,

I currently have two proxmox nodes(physical, on different ip ranges). Also have 3 proxmox VM-s on the same /24 NAT(10.0.60.0) on proxmox node 1.

On 10.0.60.0/24 there are only those 3 proxmox vm-s to avoid confusion.

Both of my proxmox nodes have only a single ipv4. I've got 3 proxmox VM-s for testing as I've broken my hypervisors too many times already.

My question:

What is the best way to do port forwarding from the host to the guests?

Here's what my /etc/network/interfaces file looks like(the rest of my subnets are just copies of vmbr1):

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

allow-hotplug eth0

iface eth0 inet manual

iface eth1 inet manual

auto vmbr0
iface vmbr0 inet static
  address  123.123.123.123
  netmask  255.255.255.0
     gateway 123.123.123.1
  bridge_ports eth0
  bridge_stp off
  bridge_fd 0

auto vmbr1
iface vmbr1 inet static
  address 10.0.10.1
  netmask 255.255.255.0
  bridge_ports none
  bridge_stp off
  bridge_fd 0

  post-up echo 1 > /proc/sys/net/ipv4/ip_forward

   post-up iptables -t nat -A POSTROUTING -s 10.0.10.0/24 -o vmbr0 -j SNAT --to-source 123.123.123.123
   post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 10000 -j DNAT --to-destination 10.0.10.2:3389
   post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 10001 -j DNAT --to-destination 10.0.10.2:22
  post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 10002 -j DNAT --to-destination 10.0.10.3:22
  post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 10003 -j DNAT --to-destination 10.0.10.4:22

This exactly achieves what I need, but any changes don't seem take effect until a reboot has taken place. My dedicated server(pmox node 1) takes minutes to reboot at online.net.

I get network in and out of my VM-s, so it works.

I've tried a few days ago to add manual iptables rules to the host, but it seems those are automatically dropped/disregarded by proxmox software running in the background.

Is there a better way to run VMs on a single IP?

Would something like using Tinc VPN be better?

Thank you for help.

pve version:
4.1.5

Installed over debian 8 via the jessie wiki.

 

[PMoxSudo]

Edit:

This is not a solution. It worked, but it seems restarting networking nulled network connectivity on all other vms on other subnets.

So I don't recommend doing this.

End edit.

I'm documenting this for my own reference and perhaps it'll help others as I couldn't find answers to adding new public port forwards to VM IP-s on private subnets without rebooting my server.

I've got over 25 vmbrX interfaces(10.0.10.0/24, 10.0.20.0/24 etc.I thought I could separate LAN networks via tasks(desktops, webserver with mysql, test vm-s,proxmox test vm lan, etc) to separate "labs". It now seems separation works. My "new test vm" on 10.0.250.2 can't ping VM-s other subnets. One of my earlier VM-s could ping VM-s on other subnets, I don't know what changed.

I figured out roughly how I can add new port forwards without rebooting the physical server.

I could manage to add new port forwarding rules to the vm via:

sudo nano /etc/network/interfaces

added new public port to forward to my "test new vm" on 10.0.250.2:

What it looks like:

auto vmbr25
iface vmbr25 inet static
  address 10.0.250.1
  netmask 255.255.255.0
  bridge_ports none
  bridge_stp off
  bridge_fd 0

#NAT
post-up iptables -t nat -A POSTROUTING -s 10.0.250.0/24 -o vmbr0 -j SNAT --to-source 123.123.123.123
post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 10000 -j DNAT --to-destination 10.0.250.2:22
post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 10001 -j DNAT --to-destination 10.0.250.2:22
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 10001 -j DNAT --to-destination 10.0.250.2:22
post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 10002 -j DNAT --to-destination 10.0.250.2:22
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 10002 -j DNAT --to-destination 10.0.250.2:22

Each of those ports represents a phase of my testing/vm reboots and such until I got what I wanted.

Added these:

post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 10002 -j DNAT --to-destination 10.0.250.2:22
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 10002 -j DNAT --to-destination 10.0.250.2:22

That's add the 10002 port to forward to VM IP at port 22. Also delete the rule when the interface goes down.

Restarted twice(one should be enough) the vmbr25 interface:

sudo ifdown vmbr25
sudo ifup vmbr25

on the host.

Restarted networking:

sudo /etc/init.d/networking restart.

Also restarted eth0 in a shell script in /root/eth0 containing:
sudo ifdown eth0
sudo ifup eth0

and another shell script containing/regarding vmbr0(masques eth0) in /root/vmbr0:
sudo ifdown vmbr0
sudo ifup vmbr0

Stopped the vm(sudo poweroff would have been enough probably) via webgui and rebooted it with start.

Now the VM is avaiable on the new port over ssh from my workplace.

There are probably redundant/irrelevant steps there, but this worked.

Edit:

Looks like this also pushed the other vm-s offline.