[SOLVED] PVE4.2 Root Server Main IP & additional /32 IPs for VMs

[houbidoo]

Hey Forum,

I need help with my basic network setup.

Situation:
I have a Root Server @ Netcup DataCenter with working Proxmox 4.2 Setup on the host.
The Server has a Main IP 111.111.111.111 Netmask 255.255.252.0 Gateway 111.111.108.1
For my VMs i ordered 3 additional IP-Addresses. These addresses are /32 addresses routed to my Main IP by the provider. I like to use LXC containers with the additional IPs.

Problem:
How to setup the network configuration that I can get the setup running?
Proxmox Host - Main IP /22
vm1 - 1st /32 additional IP
vm2 - 2nd /32 additional IP
......

Info:
with Proxmox 3.x it worked fine with openvz vms, I just used bridged setup and put in the /32 IP-Address without any gateway.

 

[dietmar]

You can simply use the host vmbr0 address (111.111.111.111) as gateway. But you need to enable ip forwarding on the host to make
that work:

echo 1 > /proc/sys/net/ipv4/ip_forward

There should also be a route for the additional network on the host.

 

[houbidoo]

Good morning,
Thx for quick help.

It doesn't work right now....

Config (host):
root@host:/home/# cat /proc/sys/net/ipv4/ip_forward
1
root@host:/home/# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto vmbr0
iface vmbr0 inet static
address 37.xxx.xxx.172
netmask 255.255.252.0
gateway 37.xxx.xxx.1
bridge_ports eth0
bridge_stp off
bridge_fd 0

Config (VM):
root@cloud:~# cat /etc/network/interfaces
auto eth0
iface eth0 inet static
address 188.xx.xx.69
netmask 255.255.255.255
post-up ip route add 37.xxx.xxx.172 dev eth0
post-up ip route add default via 37.xxx.xxx.172 dev eth0
pre-down ip route del default via 37.xxx.xxx.172 dev eth0
pre-down ip route del 37.xxx.xxx.172 dev eth0

Debugging Info via tcpdump @Host Maschine:
-->> ICMP is getting to the VM, but not back

root@host:/home/# tcpdump | grep aftr | grep ICMP
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:51:33.972744 IP aftr-37-xxx-xxx-153.unity-media.net > cloud.xxx.de: ICMP echo request, id 62819, seq 513, length 64
09:51:33.972770 IP aftr-37-xxx-xxx-153.unity-media.net > cloud.xxx.de: ICMP echo request, id 62819, seq 513, length 64
09:51:34.970985 IP aftr-37-xxx-xxx-153.unity-media.net > cloud.xxx.de: ICMP echo request, id 62819, seq 514, length 64
09:51:34.971039 IP aftr-37-xxx-xxx-153.unity-media.net > cloud.xxx.de: ICMP echo request, id 62819, seq 514, length 64
09:51:35.971047 IP aftr-37-xxx-xxx-153.unity-media.net > cloud.xxx.de: ICMP echo request, id 62819, seq 515, length 64
09:51:35.971096 IP aftr-37-xxx-xxx-153.unity-media.net > cloud.xxx.de: ICMP echo request, id 62819, seq 515, length 64
^C129 packets captured
167 packets received by filter
0 packets dropped by kernel

 

[dietmar]

post-up ip route add 37.xxx.xxx.172 dev eth0
post-up ip route add default via 37.xxx.xxx.172 dev eth0
pre-down ip route del default via 37.xxx.xxx.172 dev eth0
pre-down ip route del 37.xxx.xxx.172 dev eth0

Why eth0? Please use vmbr0 instead.

 

[houbidoo]

Why eth0? Please use vmbr0 instead.

It's one of the VMs....
I tried to use vmbr0... (even if I change the interface-name from eth0 to vmbr0 at the VM it is not working)

I see packets going to the interface when I ping from outside,
when I try to ping from the vm to outside I can see packets on the interface, too.....BUT ALL ERRORS

root@cloud:~# cat /etc/network/interfaces
auto vmbr0
iface vmbr0 inet static
address 188.xx.xx.68
netmask 255.255.255.255
post-up ip route add 37.xxx.xxx.172 dev vmbr0
post-up ip route add default via 37.xxx.xxx.172 dev vmbr0
pre-down ip route del default via 37.xxx.xxx.172 dev vmbr0
pre-down ip route del 37.xxx.xxx.172 dev vmbr0

vmbr0
Link encap:Ethernet HWaddr 36:61:36:63:38:32
inet addr:188.xx.xx.68 Bcast:0.0.0.0 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7984 errors:0 dropped:0 overruns:0 frame:0
TX packets:33 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1160369 (1.1 MiB) TX bytes:2718 (2.6 KiB)

 

[houbidoo]

do we need some proxy arp or something, too?

 

[houbidoo]

soooooo,
good news:

It works with the following config:

Host ##
#######
root@host:/# cat /etc/network/interfaces
auto lo
iface lo inet loopback auto vmbr0
iface vmbr0 inet static
address 37.xxx.xxx.172
netmask 255.255.252.0
gateway 37.xxx.xxx.1
bridge_ports eth0
bridge_stp off
bridge_fd 0
up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
up echo 1 > /proc/sys/net/ipv4/ip_forward

up ip route add 188.xx.xx.68/32 dev vmbr0
up ip route add 188.xx.xx.69/32 dev vmbr0
up ip route add 188.xx.xx.70/32 dev vmbr0

#######
VMs ###
#######
root@mail:~# cat /etc/network/interfaces
auto eth0
iface eth0 inet static
address 188.xx.xx.68
netmask 255.255.255.255
post-up ip route add 37.xxx.xxx..172 dev eth0
post-up ip route add default via 37.xxx.xxx.172 dev eth0
pre-down ip route del default via 37.xxx.xxx.172 dev eth0
pre-down ip route del 37.xxx.xxx.172 dev eth0