Help with routed config and VMs

[pfscott]

At my data center I have 1 NIC and multiple public IP addresses on the same subnet. I've installed proxmox and can access it fine on one of my IPs at xx.xx.133.50.

I would like to make my other IPs available for VMs. It sounds like I need to use a routed setup. I tried creating a bridge with a single IP, for example xx.xx.133.51/26 but when I try to ping from within the vm all I get is "host unreachable". I feel like I'm missing something. I've read the guides. Any suggestions?

 

[pfscott]

Anyone? I've switched to using a second subnet just to match the proxmox docs but I still can't get a connection in or out of the vm. Here's my interfaces:

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet static
        address xxx.yy.57.37/27
        gateway xxx.yy.57.33
        dns-nameservers 8.8.8.8 8.8.4.4
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/eno1/proxy_arp
#dns-** options are implemented by the resolvconf package, if installed

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
        address aa.bb.133.1/26
        bridge-ports none
        bridge-stp off
        bridge-fd 0

 

[bobmc]

If your proxmox host and your vm's are going to be on the same subnet, the default setup should work fine - e.g

auto lo
iface lo inet loopback

auto ens18
iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
        address xx.yy.zz.100/24
        gateway xx.yy.zz.1
        bridge-ports eth0
        bridge-stp off
        bridge-fd 0

where xx.yy.zz.100 is the IP you have assiggned to your proxmox host and xx.yy.zz.1 is the gateway address of your LAN. Then any VM or container attached to vmbr0 will be able to reach anything else on the same network and will be able to use DHCP or static IP's depending on the configuration of your network.

 

[fredlong]

At my data center I have 1 NIC and multiple public IP addresses on the same subnet. I've installed proxmox and can access it fine on one of my IPs at xx.xx.133.50.

I would like to make my other IPs available for VMs. It sounds like I need to use a routed setup. I tried creating a bridge with a single IP, for example xx.xx.133.51/26 but when I try to ping from within the vm all I get is "host unreachable". I feel like I'm missing something. I've read the guides. Any suggestions?

I just went through this and info on it seems to be few and far between for something that seems like a pretty typical setup. So to save the next person from going through what I did, I'll post it here. It's actually pretty simple.

I wanted to dedicate 1 public IP to Proxmox itself and route the others public IPs to individual VMs. First we create a new Linux Bridge interface in the Network section of the main server called vmbr1 with a new vlan IP CDIR, I used 10.0.10.0/24. Leave the Gateway and Ports/Slaves empty. Then we need to set all of the incoming public IPs that we want to use in /etc/network/interfaces on the vmbr0 interface in the main server shell and modify vmbr1 as follows:

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

iface enp3s0f0 inet manual

iface enp3s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 123.33.6.222/29
        gateway 123.33.6.221
        bridge-ports eno3
        bridge-stp off
        bridge-fd 0

iface vmbr0 inet static
        address 123.33.6.223/29
        gateway 123.33.6.221
        bridge-ports eno3
        bridge-stp off
        bridge-fd 0

# etc... etc... etc...

auto vmbr1
iface vmbr1 inet static
        address 10.0.10.0/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -s '10.0.10.0/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.0.10.0/24' -o vmbr0 -j MASQUERADE

Assign all of the VMs that you want to give a public IP the vmbr1 adapter for their network devices. And then for each of those VMs you will need to configure a static vlan IP within the CDIR of the vmbr1 bridge. In Debian, you can edit the /etc/network/interfaces file of each VM (be sure to assign the gateway to the address of vmbr1):

allow-hotplug ens18
iface ens18 inet static
        address 10.0.10.1/24 # <<<< change this for each VM
        gateway 10.0.10.0
        dns-nameservers 8.8.8.8 8.8.4.4

That will get us OUT on the VMs. To get IN on them, we need to set some IP routing rules with iptables on the main server:

iptables -t nat -A PREROUTING -p tcp -d 123.33.6.222 --dport 22 -i vmbr0 -j DNAT --to-destination 10.0.10.1:22
iptables -t nat -A PREROUTING -p tcp -d 123.33.6.223 --dport 22 -i vmbr0 -j DNAT --to-destination 10.0.10.2:22
# etc... etc... etc...

Hope that gets people at least started.

 

[mr.x]

auto vmbr0
iface vmbr0 inet static
address 123.33.6.222/29
gateway 123.33.6.221
bridge-ports eno3
bridge-stp off
bridge-fd 0

iface vmbr0 inet static
address 123.33.6.223/29
gateway 123.33.6.221
bridge-ports eno3
bridge-stp off
bridge-fd 0

Hi,
does this work? Two times vmbr0?

BR
Jan