Dear,
in first, it´s nice, that IPs and Gateways can add direct in the GUI of proxmox 4.x, but I thing so, it must been warned first, that the CIDR will not been used for the IP-block the hoster has get versus the IPs, the virtual machine will get only, otherwise a spoofing with active IP-Filter is possible! So in normal situations must have the lxc-container every time only a IP/32 !!! This works fine too with the gateway expl. in an IP-Block!
The next problem is the routing with many IPs, expl. 4 IPs! Here I thing so, they had bugs!
In first: The hoster / ISP has an IP-Block 23.257.12.0/27 and expl. gateway 23.257.12.31 and from them the CT100 had get 2 IPs /32 !
23.257.12.10/32 -> eth0
45.12.257.10/32 -> eth1
23.257.12.14/32 -> eth2
98.257.12.15/32 -> eth3
The LXC-Container will not let start, when you have 2 times the same gateway !!!
A bad workarround is, to change to /27 - block !!!
23.257.12.10/27 -> eth0
45.12.257.10/32 -> eth1
23.257.12.14/27 -> eth2
98.257.12.15/32 -> eth3
Just in this scenario the IPs can been spoofed in the same IP-block! Expl. the user in the virtual machine can change the IP 23.257.12.10 to 23.257.12.11 and so on!
Next is, that the routing is wrong! With many IPs the communication from inside the virtual machine to outside is going only via one NIC! You can test out with:
ping -I 23.257.12.10 8.8.8.8
or ping -I 45.12.257.10 8.8.8.8
or ping -I 23.257.12.14 8.8.8.8
and so on!
A workarround is goes back to the root, as we used before in proxmox 3.x - We are sorry and hope, that they will fixed for automatic:
In the GUI we configure the NICs only with the MAC and let free the IP field and the Gateway! Then we start the LCX machine and enter with pct enter 100 in the virtualmachine! When we have set before the IPs via GUI, then we must check in /etc/network/interfaces that only the lo interface is defined! So, then must look so:
# Auto generated lo interface
auto lo
iface lo inet loopback
Then we must exit the virtualmachine and restart one time!
We go again in the virtual machine and write and start our network-start script - ok, yes you must edit manual one time and the absolute correct the IPs and gateway, but then is running fine! We have written a small script for proxmox 3.x and this help us for this too!
ip link set eth0 up
ip addr add 221.12.43.221/32 dev eth0
route add 65.12.15.254 eth0
ip route add 221.12.43.221 dev eth0 table 1000
ip route add 0.0.0.0/0 via 65.12.15.254 dev eth0 table 1000
ip rule add from 221.12.43.221 table 1000
ip rule add to 221.12.43.221 table 1000
route add default gw 221.12.43.221 dev eth0
ip link set eth1 up
ip addr add 218.90.12.136/32 dev eth1
route add 65.12.15.254 eth1
ip route add 218.90.12.136 dev eth1 table 1001
ip route add 0.0.0.0/0 via 65.12.15.254 dev eth1 table 1001
ip rule add from 218.90.12.136 table 1001
ip rule add to 218.90.12.136 table 1001
route add default gw 218.90.12.136 dev eth1
ip link set eth2 up
ip addr add 218.90.224.89/32 dev eth2
route add 65.12.15.254 eth2
ip route add 218.90.224.89 dev eth2 table 1002
ip route add 0.0.0.0/0 via 65.12.15.254 dev eth2 table 1002
ip rule add from 218.90.224.89 table 1002
ip rule add to 218.90.224.89 table 1002
route add default gw 218.90.224.89 dev eth2
ip link set eth3 up
ip addr add 98.90.214.206/27 dev eth3
route add 98.90.214.222 eth3
ip route add 98.90.214.206 dev eth3 table 1003
ip route add 0.0.0.0/0 via 98.90.214.222 dev eth3 table 1003
ip rule add from 98.90.214.206 table 1003
ip rule add to 98.90.214.206 table 1003
route add default gw 98.90.214.206 dev eth3
in first, it´s nice, that IPs and Gateways can add direct in the GUI of proxmox 4.x, but I thing so, it must been warned first, that the CIDR will not been used for the IP-block the hoster has get versus the IPs, the virtual machine will get only, otherwise a spoofing with active IP-Filter is possible! So in normal situations must have the lxc-container every time only a IP/32 !!! This works fine too with the gateway expl. in an IP-Block!
The next problem is the routing with many IPs, expl. 4 IPs! Here I thing so, they had bugs!
In first: The hoster / ISP has an IP-Block 23.257.12.0/27 and expl. gateway 23.257.12.31 and from them the CT100 had get 2 IPs /32 !
23.257.12.10/32 -> eth0
45.12.257.10/32 -> eth1
23.257.12.14/32 -> eth2
98.257.12.15/32 -> eth3
The LXC-Container will not let start, when you have 2 times the same gateway !!!
A bad workarround is, to change to /27 - block !!!
23.257.12.10/27 -> eth0
45.12.257.10/32 -> eth1
23.257.12.14/27 -> eth2
98.257.12.15/32 -> eth3
Just in this scenario the IPs can been spoofed in the same IP-block! Expl. the user in the virtual machine can change the IP 23.257.12.10 to 23.257.12.11 and so on!
Next is, that the routing is wrong! With many IPs the communication from inside the virtual machine to outside is going only via one NIC! You can test out with:
ping -I 23.257.12.10 8.8.8.8
or ping -I 45.12.257.10 8.8.8.8
or ping -I 23.257.12.14 8.8.8.8
and so on!
A workarround is goes back to the root, as we used before in proxmox 3.x - We are sorry and hope, that they will fixed for automatic:
In the GUI we configure the NICs only with the MAC and let free the IP field and the Gateway! Then we start the LCX machine and enter with pct enter 100 in the virtualmachine! When we have set before the IPs via GUI, then we must check in /etc/network/interfaces that only the lo interface is defined! So, then must look so:
# Auto generated lo interface
auto lo
iface lo inet loopback
Then we must exit the virtualmachine and restart one time!
We go again in the virtual machine and write and start our network-start script - ok, yes you must edit manual one time and the absolute correct the IPs and gateway, but then is running fine! We have written a small script for proxmox 3.x and this help us for this too!
ip link set eth0 up
ip addr add 221.12.43.221/32 dev eth0
route add 65.12.15.254 eth0
ip route add 221.12.43.221 dev eth0 table 1000
ip route add 0.0.0.0/0 via 65.12.15.254 dev eth0 table 1000
ip rule add from 221.12.43.221 table 1000
ip rule add to 221.12.43.221 table 1000
route add default gw 221.12.43.221 dev eth0
ip link set eth1 up
ip addr add 218.90.12.136/32 dev eth1
route add 65.12.15.254 eth1
ip route add 218.90.12.136 dev eth1 table 1001
ip route add 0.0.0.0/0 via 65.12.15.254 dev eth1 table 1001
ip rule add from 218.90.12.136 table 1001
ip rule add to 218.90.12.136 table 1001
route add default gw 218.90.12.136 dev eth1
ip link set eth2 up
ip addr add 218.90.224.89/32 dev eth2
route add 65.12.15.254 eth2
ip route add 218.90.224.89 dev eth2 table 1002
ip route add 0.0.0.0/0 via 65.12.15.254 dev eth2 table 1002
ip rule add from 218.90.224.89 table 1002
ip rule add to 218.90.224.89 table 1002
route add default gw 218.90.224.89 dev eth2
ip link set eth3 up
ip addr add 98.90.214.206/27 dev eth3
route add 98.90.214.222 eth3
ip route add 98.90.214.206 dev eth3 table 1003
ip route add 0.0.0.0/0 via 98.90.214.222 dev eth3 table 1003
ip rule add from 98.90.214.206 table 1003
ip rule add to 98.90.214.206 table 1003
route add default gw 98.90.214.206 dev eth3
