Multiple Public IP help plz!

102020 · May 17, 2013

Hey all,

I'm new to proxmox, trying to figure this out - been on Citrix for a while, and seems a lot simpler.

Anyways, I'll give my info and tell you want I'm trying to accomplish:

IP ADDRESSES
Primary IP address : xxx.xxx.164.207
Primary IP mask : 255.255.255.224
Gateway : xxx.xxx.164.193

SECONDARY IP ADDRESS
Usable secondary addresses: xxx.xxx.7.144-151
Mask : 255.255.255.248

The above was provided by my data center, the server has 1 NIC, the primary IP I have working on proxmox itself (so I can connect via gui/ssh, access the internet)
The secondary block of public IP is what I want my virtual machines to use. How do I go about that? Data center says secondary block can route through primary gateway, but I'm not having any luck getting internet on the virtual machines - I can ping from proxmox to vm and vice versa, but no external (apt-get, ping, etc).

Please tell me something helpful, I'm guessing some sort of routing is needed from second IP block to gateway?

spirit · May 17, 2013

Hi , proxmox don't support routed network config (because for this we need to manage vms ip from proxmox), but I think it should be possible to do it manually.

on your guest
--------------
setup a secondary ip with proxmox host as gateway

on you proxmox host
--------------------
#sysctl -w net.ipv4.ip_forward=1 # allow forwarding of IPv4

#route add -host <ip-of-your-guest> dev <tap-device> # add route to the client

with tap-device = tap<vmid>i<interfacenumber>

by example, for vmid 115 - eth0 = tap115i0

More informations here :
http://www.linux-kvm.org/page/Networking

mo_ · May 17, 2013

102020 said:
Data center says secondary block can route through primary gateway

This means that the gateway will accept these IPs coming from your server too, Id think. Typically the datacenter requires you to use certain MAC addresses for the additional IPs, which they provide you (upon request). So you probably just have to make sure the VMs/CTs use the correct MAC address for the additional IPs.

102020 · May 17, 2013

So my question then - see here: http://pve.proxmox.com/wiki/Network_Model

It does mention routed configuration, is it incorrect? I did try it, seemed I could only '1 way' ping though, wasn't getting response from primary to secondary ip.

If anyone can shed some light on that, it'd be great. if i have to not use the primary IP, so be it (not that I will ever accept that though!)

102020 · May 17, 2013

mo_ said:
This means that the gateway will accept these IPs coming from your server too, Id think. Typically the datacenter requires you to use certain MAC addresses for the additional IPs, which they provide you (upon request). So you probably just have to make sure the VMs/CTs use the correct MAC address for the additional IPs.

So this is from my DC, based on VMWare, does it make sense based on the information recommended above?

http://kb.iweb.com/entries/21140298-VMware-Configuring-additional-IPs-Linux-

102020 · May 17, 2013

http://kb.iweb.com/entries/21140298-VMware-Configuring-additional-IPs-Linux-

This is from my data center, does it make sense based on info above?

mo_ · May 18, 2013

102020 said:
So my question then - see here: http://pve.proxmox.com/wiki/Network_Model

It does mention routed configuration, is it incorrect?

Theres always more than 1 way to do things. Its just that personally, I find the configuration posted as "routed" setup there weird because Ive never seen it like that.

Your setup should require a rather simple setup. You only need what is listed on the Network_Model page as the default setup. Simply use that and directly assign the additional IPs to your VMs from inside the respective VMs (or via webinterface if you want to use openvz with venet).

spirit · May 19, 2013

102020 said:
So my question then - see here: http://pve.proxmox.com/wiki/Network_Model

It does mention routed configuration, is it incorrect? I did try it, seemed I could only '1 way' ping though, wasn't getting response from primary to secondary ip.

I don't known why proxmox wiki say it's routed, because it's a NAT.
so, this setup, your guest vm ip are natted (translated) to your host ip, so the outgoing ip is your host ip.
But with this setup, you can't access from outside to your vm. (And you need only 1 public ip)

With a routed setup, the traffic need to go in/out from your vms. So your hosted need to send packets coming for outside to your guest tap interface.
I think you can choose the NAT method in the proxmox GUI, so your tap will not be a a bridge, and they will not be a problem with the vms MAC address.
They add the route to tap from my previous example, it should work.

aderumier · May 21, 2013

102020 said:
http://kb.iweb.com/entries/21140298-VMware-Configuring-additional-IPs-Linux-

This is from my data center, does it make sense based on info above?

Hi, the above is wrong, can you repost it ?

I have re-read proxmox doc, I think I understand how it should work.

This config should work:

Code:

auto lo
iface lo inet loopback


auto eth0
iface eth0 inet static
        address  xxx.xxx.164.207
        netmask  255.255.255.224
        gateway  xx.xxx.164.193
        post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
	post-up echo 1 > /proc/sys/net/ipv4/ip_forward




auto vmbr0
iface vmbr0 inet static
        address  xxx.xxx.7.144   (or any ip of your secondary range)
        netmask  255.255.255.24
        bridge_ports none
        bridge_stp off
        bridge_fd 0

you should be able to ping both ips from outside.

Now, in your vm config, create nic, bridge on vmbr0
with this ip configuration inside guest

Code:

auto eth0
iface eth0 inet static
        address  xxx.xxx.7.145   
        netmask  255.255.255.24
        gateway xxx.xxx.7.144

Test, and tell me if it's ok

102020 · May 25, 2013

aderumier said:
Hi, the above is wrong, can you repost it ?

I have re-read proxmox doc, I think I understand how it should work.

This config should work:

Code:

auto lo iface lo inet loopback auto eth0 iface eth0 inet static address xxx.xxx.164.207 netmask 255.255.255.224 gateway xx.xxx.164.193 post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp post-up echo 1 > /proc/sys/net/ipv4/ip_forward auto vmbr0 iface vmbr0 inet static address xxx.xxx.7.144 (or any ip of your secondary range) netmask 255.255.255.24 bridge_ports none bridge_stp off bridge_fd 0

you should be able to ping both ips from outside.

Now, in your vm config, create nic, bridge on vmbr0
with this ip configuration inside guest

Code:

auto eth0 iface eth0 inet static address xxx.xxx.7.145 netmask 255.255.255.24 gateway xxx.xxx.7.144

Test, and tell me if it's ok

hmmm, that seems to have made both main and secondary IP unreachable.

Do I need a route from secondary to primary?
This is my info they gave me:

IP ADDRESSES
Primary IP address : x.x.164.207
Primary IP mask : 255.255.255.224
Gateway : x.x.164.193

SECONDARY IP ADDRESS
Usable secondary addresses: x.x.7.144-151
Mask : 255.255.255.248

I really appreciate your help on this.

spirit · May 26, 2013

102020 said:
hmmm, that seems to have made both main and secondary IP unreachable.

Do I need a route from secondary to primary?
This is my info they gave me:

IP ADDRESSES
Primary IP address : x.x.164.207
Primary IP mask : 255.255.255.224
Gateway : x.x.164.193

SECONDARY IP ADDRESS
Usable secondary addresses: x.x.7.144-151
Mask : 255.255.255.248

I really appreciate your help on this.

The main idea is to route traffic between your primary ip and secondary ip.

little schema :

x.x.164.193 Hertner router------------->x.x.164.207 (eth0) - x.x.7.144 (vmbr0) <--------------vmguest (x.x.7.145)

I just notice that if my previous post, I write wrong mask for secondary ip (255.255.255.24 , instead 255.255.255.248)

Can you post your original proxmox server /etc/network/interfaces ?

102020 · May 28, 2013

spirit said:
The main idea is to route traffic between your primary ip and secondary ip.

little schema :

x.x.164.193 Hertner router------------->x.x.164.207 (eth0) - x.x.7.144 (vmbr0) <--------------vmguest (x.x.7.145)

I just notice that if my previous post, I write wrong mask for secondary ip (255.255.255.24 , instead 255.255.255.248)

Can you post your original proxmox server /etc/network/interfaces ?

This is the default (what I set it back to):

auto lo
iface lo inet loopback

auto vmbr0
iface vmbr0 inet static
address x.x.164.207
netmask 255.255.255.224
gateway x.x.164.193
bridge_ports eth0
bridge_stp off
bridge_fd 0

102020 · May 28, 2013

Ok, this is what I have thus far since the original, eth0 is working, but vmbr0 is not.

# network interface settings
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address x.x.164.207
netmask 255.255.255.224
gateway x.x.164.193
post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

auto vmbr0
iface vmbr0 inet static
address x.x.7.144
netmask 255.255.255.248
bridge_ports none
bridge_stp off
bridge_fd 0

102020 · Jun 3, 2013

spirit · Jun 3, 2013

Hi, are you sure that x.x.7.144 is correctly routed (in the internet routeurs) through x.x.164.207 ?

Can you do a

#traceroute x.x.7.144

?

And see if it's hanging on x.x.164.207 ?

102020 · Jun 3, 2013

in proxmox, i can hit x.x.7.144
root@proxmox:~# traceroute x.x.7.144
traceroute to x.x.7.144 (x.x.7.144), 30 hops max, 60 byte packets
1 x.x.7.144 (x.x.7.144) 0.032 ms 0.012 ms 0.012 ms

From within proxmox, it can ping. my debian vm has x.x.7.145 set.

from within the vm i can ping x.x.7.144:
root@proxmox:~# traceroute x.x.7.144
traceroute to x.x.7.144 (x.x.7.144), 30 hops max, 60 byte packets
1 x.x.7.144 (x.x.7.144) 0.029 ms 0.009 ms 0.009 ms

im testing with ping to: 8.8.8.8 (google dns) which does get a response on my other servers/desktops.
from within the vm i can ping x.x.164.207:
test:/# ping 72.55.164.207
PING 72.55.164.207 (72.55.164.207) 56(84) bytes of data.
64 bytes from 72.55.164.207: icmp_seq=1 ttl=64 time=0.034 ms
64 bytes from 72.55.164.207: icmp_seq=2 ttl=64 time=0.014 ms

so this would seem that internally i can ping, but the seems to not be able to hit anything outside (aka the internet).

this is what's in my ifconfig in the vm:

test:/# ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:336 (336.0 B) TX bytes:336 (336.0 B)

venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:127.0.0.2 P-t-P:127.0.0.2 Bcast:0.0.0.0 Mask:255.255.255.255
UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:75 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:504 (504.0 B) TX bytes:6271 (6.1 KiB)

venet0:0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:x.x.7.145 P-t-P:x.x.7.145 Bcast:0.0.0.0 Mask:255.255.255.255
UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1

spirit · Jun 3, 2013

Ok, I think we are close to find the solution

Do you have setup your vm network card in bridge mode, on vmbr0 ?

Do you have setup x.x.7.144 as gateway in your vm ?

102020 · Jun 3, 2013

in proxmox gui under network i have:
name: eth0
active: yes
autostart: yes
ports/slaves:
ip address: x.x.164.207
subnet mask: 255.255.255.224
gateway: x.x.164.193

name: vmbr0
active: yes
autostart: yes
ports/slaves:
ip address: x.x.7.144
subnet mask: 255.255.255.248
gateway:

this is reflected as (in /etc/network/interfaces):
# network interface settings
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 72.55.164.207
netmask 255.255.255.224
gateway 72.55.164.193
post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

auto vmbr0
iface vmbr0 inet static
address 174.142.7.144
netmask 255.255.255.248
bridge_ports none
bridge_stp off
bridge_fd 0

in proxmox gui, under vm, under network:
type: ip address
ip address/name: x.x.7.145

in proxmox ssh, i can ping x.x.164.207, x.x.164.193, x.x.7.144, x.x.7.145
in proxmox ssh, vzctl enter 101, i can ping x.x.164.207, x.x.7.144, x.x.7.145 but not the gateway @ x.x.164.193, nor any external IP (and im guessing thats the problem).

spirit · Jun 4, 2013

102020 said:
in proxmox gui under network i have:
name: eth0
active: yes
autostart: yes
ports/slaves:
ip address: x.x.164.207
subnet mask: 255.255.255.224
gateway: x.x.164.193

name: vmbr0
active: yes
autostart: yes
ports/slaves:
ip address: x.x.7.144
subnet mask: 255.255.255.248
gateway:

this is reflected as (in /etc/network/interfaces):
# network interface settings
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 72.55.164.207
netmask 255.255.255.224
gateway 72.55.164.193
post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

auto vmbr0
iface vmbr0 inet static
address 174.142.7.144
netmask 255.255.255.248
bridge_ports none
bridge_stp off
bridge_fd 0

in proxmox gui, under vm, under network:
type: ip address
ip address/name: x.x.7.145

in proxmox ssh, i can ping x.x.164.207, x.x.164.193, x.x.7.144, x.x.7.145
in proxmox ssh, vzctl enter 101, i can ping x.x.164.207, x.x.7.144, x.x.7.145 but not the gateway @ x.x.164.193, nor any external IP (and im guessing thats the problem).

I never test it with openvz container, can you try with a kvm vm to compare ?

Also in your vm, do you have setup x.x.7.144 as default gateway ? (I think yes, because you can ping x.x.164.207)

can you post your vm config ? (/etc/pve/qemu-server/vmid.conf).

Also, maybe can you try to remove "post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp" , not sure it's usefull in a routed config.

102020 · Jun 4, 2013

spirit said:
I never test it with openvz container, can you try with a kvm vm to compare ?

Also in your vm, do you have setup x.x.7.144 as default gateway ? (I think yes, because you can ping x.x.164.207)

can you post your vm config ? (/etc/pve/qemu-server/vmid.conf).

Also, maybe can you try to remove "post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp" , not sure it's usefull in a routed config.

I just setup a kvm vm, /etc/network/interfaces is now:
auto lo
iface lo inet loopback

allow-hotplug eth0
iface eth0 inet static
address x.x.7.145
netmask 255.255.255.248
network x.x.7.144
broadcast x.x.7.151
gateway x.x.7.144
dns-nameservers 8.8.8.8 8.8.4.4

when i try to ping x.x.7.144, it now says to add -b to ping broadcast, i do so, and get no response. can't ping x.x.164.207 or x.x.164.193 either

this is the vmid.conf:
root@proxmox:/etc/pve/qemu-server# cat 100.conf
bootdisk: ide0
cores: 1
ide0: local:100/vm-100-disk-1.qcow2,size=32G
ide2: local:iso/debian-5010-i386-netinst.iso,media=cdrom,size=156262K
memory: 512
name: test
net0: e1000=C2:1D:B7:80:1C:AA,bridge=vmbr0
ostype: l26
sockets: 1

Multiple Public IP help plz!

Active Member

Distinguished Member

Active Member

Active Member

Active Member

Active Member

Active Member

Distinguished Member

Well-Known Member

Active Member

Distinguished Member

Active Member

Active Member

Active Member

Distinguished Member

Active Member

Distinguished Member

Active Member

Distinguished Member

Active Member