[SOLVED] KVM bridged on Hetzner, can't add second IP

[petrogazz]

Hello,

we have a proxmox 4.4 node hosted on Hetzner and configured with bridged networking. The machine works fine with this setup and hosts 4 KVMs with additional IPs (no subnet) with separate MACs.

Each VM has a single IP and is working as expected (Centos cPanel VMs).

We are trying to add a second IP to one of the VMs. We added a second network interface from Proxmox interface with the appropriate separate MAC from Hetzner and added the /etc/sysconfig/network-scripts/ifcfg-eth1 in Centos VM.

When we enable the interface it comes up and it is reacheable. The other IP however (eth0) cannot be reached after we enable eth1 (is up in Centos but not reachable from the outside world). In simple words only one IP can be functional at a time.

The main IP of the server and the first IP of the VPS are in the same subnet 136.243.1XX.XXX but the new additional IP is from a different one 138.201.4X.XXX.

Does anyone has an idea what is going on. I am attaching the network config files of the node and KVM.

ps Sorry for the long post and thank you in advance for your time.

Proxmox node /etc/network/interfaces

auto lo
iface lo inet loopback
iface lo inet6 loopback

auto eth0
iface eth0 inet manual

# device: eth0
auto  vmbr0
iface vmbr0 inet static
  #pre-up iptables-restore < /etc/iptables.rules
  address   136.243.XXX.XXX
  netmask   255.255.255.YYY
  pointopoint 136.243.XXX.ZZZ
  gateway   136.243.XXX.ZZZ
  bridge_ports eth0
  bridge_stp off
  bridge_fd 1
  bridge_hello 2
  bridge_maxage 12
  # default route to access subnet
  up route add -net 136.243.XXX.YYY netmask 255.255.255.YYY gw 136.243.XXX.ZZZ vmbr0

iface vmbr0 inet6 static
  address 2a01:XXXX:XXXX:XXXX::2
  netmask 64
  gateway fe80::1
  up sysctl -p

auto vmbr1
iface vmbr1 inet static
  address  10.0.0.1
  netmask  255.255.255.0
  bridge_ports none
  bridge_stp off
  bridge_fd 0

  post-up echo 1 > /proc/sys/net/ipv4/ip_forward
  post-up   iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
  post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE

KVM /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
HWADDR=00:50:WW:XX:ZZ:YY
TYPE=Ethernet
UUID=xxxxxxxxxxxxxxxxxxxxxxxx
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPV6INIT=no
PEERDNS=yes
IPADDR=136.243.XXX.WWW
NETMASK=255.255.255.YYY
GATEWAY=136.243.XXX.YYY
POINTOPOINT=136.243.XXX.YYY
DNS1=8.8.8.8
DNS2=8.8.4.4
DNS3=213.133.100.100

KVM /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1
TYPE=Ethernet
HWADDR=00:50:WW:XX:ZZ:Y2
UUID=yyyyyyyyyyyyyyyyyyyyyyyyyyy
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPV6INIT=no
PEERDNS=yes
IPADDR=138.201.XX.YYY
NETMASK=255.255.255.ZZZ
GATEWAY=138.201.XX.WWW
POINTOPOINT=138.201.XX.WWW
DNS1=8.8.8.8
DNS2=8.8.4.4
DNS3=213.133.100.100

 

[manu]

I am not a specialist of CentOS networking but I think the problem you have here is that your two IPs belong to different networks, and you're defining here:

GATEWAY=136.243.XXX.YYY
GATEWAY=138.201.XX.WWW

two different default gateways.

This will not work.

My advice here would be that the IP you use for eth1 use the same subnet as eth0 so you can use the same default gateway, and use the "new" IP range on a different VM.

 

[petrogazz]

Thanks Manu for your answer!

I was wondering, can I use another IP from the second subnet as a gateway?

 

[petrogazz]

I found a solution for assigning the second IP to the Centos KVM, I'm sharing it with all of you here:

1. Do not assign a separate MAC for the second IP in hetzner robot.

2. Add the following in Proxmox node /etc/network/interfaces at the end of vmbr0 section

up ip route add 138.201.XX.WWW/32 dev vmbr0

3. Add the second network interface for the KVM in proxmox GUI (MAC auto).

4. Create network script in Centos KVM /etc/sysconfig/network-scripts/ifcfg-eth1 using the main IP of proxmox node as a gateway

DEVICE=eth1

TYPE=Ethernet

UUID=yyyyyyyyyyyyyyyyyyyyyyyyyyy

ONBOOT=yes

NM_CONTROLLED=no

BOOTPROTO=static

IPV6INIT=no

PEERDNS=yes

IPADDR=138.201.XX.YYY

NETMASK=255.255.255.ZZZ

GATEWAY=136.243.XXX.XXX

POINTOPOINT=136.243.XXX.XXX

DNS1=8.8.8.8

DNS2=8.8.4.4

DNS3=213.133.100.100

5. Reboot proxmox node (I did it because after network reload all VM IPs where unreachable, maybe there is a way to avoid it)

Both IPs are now functional on the Centos KVM and properly recognized in cPanel.