Proxmox 2 as a router for CT KVM and LAN

C

c0mputerking

Renowned Member
Oct 5, 2011
174
5
83
Hello all i have a daul nic setup on my proxmox server and i am tring to get it working as a router for both CT KVM and real machines right now it is working for openvz ip/venet and my real machines. However it does not work for KVM or if try to get a veth working in a openvz container

eth0 is connected to the internet vi DHCP (unfortunetly) i can change this if nessary
auto eth0
iface eth0 inet dhcp

eth1 is connected to a router and my internal LAN
# network interface settings
auto eth1
iface eth1 inet static
address 192.168.1.200
netmask 255.255.255.0
broadcast 192.168.1.255
network 192.168.1.0

vmbr0 I have added after reading several tutorials this configuration works for my LAN ie i have internet on the LAN.
It also works for openvz containers that use a ip/venet
however it does not work for KVM or if try to get a veth working in a openvz container
auto vmbr0
iface vmbr0 inet static
address 192.168.1.254
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0


Some back story when i create the default vmbr0 using the gui it seems to break most everthing on my network ie the proxmox host can no longer ping google nor can any of the virtual machines or LAN machines. I thought i would try and keep it simple having all the the virtual machines and real LAN machines on the same subnet and use proxmox server as a gateway (shorewall). However i am not opposed to having 2 seperate networks and routing them togother if nessary/better anyhow how do i get veth and KVM's working on this router style of setup?? I have also tried adding vmbr1 using the GUI which also seems to break my network.
#auto vmbr1
#iface vmbr1 inet static
# address 192.168.1.253
# netmask 255.255.255.0
# gateway 192.168.1.200
# bridge_ports eth1
# bridge_stp off
# bridge_fd 0
 
From what I can tell, nothing connected to your vmbr0 would be able to get to 192.168.1.200 as it is the interface address on eth1 which is not bridged with the VMs.

Correct me if I'm wrong but you want all computers on your LAN network to connect to your PVE machine as a gateway (which would have appropriate NAT rules etc)?

A simpler solution would be to create a small (1CPU, 256mb ram, 8GB HDD) KVM with 2 NICs running PfSense/IPCop/other and route everything through that VM. You would have VETH1 connected to VMBR0 where all of your local network exists (192.168.1.xxx) and then create a VETH2 connected to a new bridge (VMBR1->eth1) which you plug into your internet connection. When you setup VMBR1 don't fill anything in other than the slave port (eth1), it will do nothing other than act as a bridge to the physical eth1 for your firewall distro. Assign IPs as appropriate in the VM firewall distro and you're good to go.

This way you do minimal changes to the host machine which is favourable when running VMs. On top of that, WAN and LAN traffic are correctly separated and you don't have to much around with iptables changes to the host. At the end of the day, it's much easier to backup a whole (yet small) VM with all the routing/firewall settings than files all over the host machine.

Here's a layout as best/simple as I can put it:

VMBR0:
IP = 192.168.1.100/24
GW = 192.168.1.254
Port = eth0 -> goes to your LAN switch
VETH1 of your firewall KVM connects here and is assigned gateway address (192.168.1.254) as "LAN" facing interface.

VMBR1:
IP = no IP
GW = N/A
Port = eth1 -> your internet connection comes in here (modem etc.)
VETH2 of your firewall KVM connects here and will have your External internet IP address assigned by PPPOE or whatever your internet connection uses.

End result, all LAN machines go to your KVM firewall and NAT out (I'm assuming you'll setup NAT as the ips you quote are all non-routable) to your internet connection, including the Proxmox Host to do NTP/updates/VZ template downloads etc. This can also work as an inline firewall for that Proxmox Node if configured a little differently.

Hope that helps :)
 
Thank you for your very detailed post, and i am going to take your advice and setup a pfsense KVM firewall machine. Besides i checked out the pfsense website who could resist all that eye candy compared to shorewall, and i have been looking for an excuse to run my first OS love FreeBSD :D

However i have a couple of questions putting what you describle into practice here is what i managed to make believe for a interfaces file. However do i need ips for eth0 or eth1 or how do I access the host? I have put ips in there but are they correct maybe you could show me your interface file? I plug ISP into eth1 so it still should be dhcp right? as right now i am on dhcp extrenal IP (can get this changed to static if nessary) ok i should stop now i am confusing myself here and have not even gotten to the KVM or veth parts yet.

root@sun:/etc/network# vi interfaces
#iface eth0 inet manual

#iface eth1 inet manual

auto eth0
iface eth0 inet static
address 192.168.1.200
netmask 255.255.255.0
broadcast 192.168.1.255
network 192.168.1.0

auto eth1
iface eth1 inet dhcp

auto vmbr0
iface vmbr0 inet static
address 192.168.1.253
netmask 255.255.255.0
gateway 192.168.1.254
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet manual
bridge_ports eth1
bridge_stp off
bridge_fd 0

PS i'm a bit chicken to pull the trigger on this interfaces file, as my networking is working so well last couple of days haha!!

PPS thanks again for you help
 
Ok i have tryed the interfaces file above but it is not working, on the host i cannot ping google but i can ping my laptop which is on the LAN. On my laptop i can ping the proxmox host however i cannot get to my proxmox web GUI to setup the pfsense machine. I can howerver ssh into the proxmox host not sure what i have done wrong please help.
 
I have been using pfsense in a KVM as a production router for more than a year and it works very well. No need to assign IP addresses to physical interfaces eth0 or eth1. Given your setup here's a valid interfaces file below, that will work. Notice I have the proxmox host's IP assigned to vmbr0 and setup the gateway to the IP of 192.168.1.254, which will be the LAN IP that is setup in the pfsense KVM.

You will now need to create a KVM using pfsense 2.0.1 CD. A small 4Gb disk with 512Mb memory works great. Make sure to setup two NICs one on vmbr0 and one on vmbr1. It helps to write down the MAC address that proxmox generates for each NIC to make sure you assign the correct interface in pfsense to WAN and LAN.

In pfsense assign the NIC on vmbr1 to WAN with DHCP.
And assign the NIC on vmbr0 to LAN with IP address: 192.168.1.254.

Keep in mind the OVZ containers will use the default router assigned to vmbr0. KVMs and Real machines need to be assigned via DHCP or manually. Why not setup DHCP and DNS forwarding inside pfsense... that way you can use all that eyecandy. ;)


auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.1.200
netmask 255.255.255.0
gateway 192.168.1.254
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet manual
bridge_ports eth1
bridge_stp off
bridge_fd 0
 
Thanks I actully just stumbled threw getting it working before i read your latest post :D and was just enjoying the pfsense icandy setting everything up dhcp and the like. here is the interfaces file i came up with (same as yours) still a littled shocked that this is working haha

# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

#auto eth1
#iface eth1 inet dhcp

#auto eth0
#iface eth0 inet static
# address 192.168.1.200
# netmask 255.255.255.0
# broadcast 192.168.1.255
# network 192.168.1.0

auto vmbr0
iface vmbr0 inet static
address 192.168.1.200
netmask 255.255.255.0
gateway 192.168.1.254
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet manual
bridge_ports eth1
bridge_stp off
bridge_fd 0

Also used the realtek virtual nics in proxmox wasnt sure if bsd would like the paravitulized stuff what are you using/suggesting? best preformace ect just about to use the 99 install to hardrive feature.

couple of questions as most of this setup still blows my mind haha and i might just be putting this here for my own sanity haha

gateway is now vmbr0 which is 192.168.1.254

proxmox is now accessed using vmbr0 that i have set to 192.168.1.200 and also gateway of 192.168.1.254??

I am going to setup pfsense on static LAN ip of 192.168.1.201 which i should use for dns dhcp ntp server etc as well as the pfsense web GUI

example I set my router for wireless client to gateway to ip of 192.168.1.1 (routers like this ip) gateway 192.168.1.254 dns ntp of 192.168.1.201
dhcp for most other devices since pfsense can do that now.

What about the proxmox host itself does it still need shorewall running do i still need to have ip forwarding enables probably not right ? Ok might fingers are rambling on here

BIG PS i cannot get pfsense to find a harddrive to install to tried virtual drive also tried ide drive installer cannot find a drive any advice here?
 
For the hard drive I have setup IDE raw and it works on my 2.0 and 1.9 installs. I basically accept the defaults for both disk and network.

I did a quick test install and virtio disks/network don't work for pfsense.

You don't need anything extra on your proxmox host (shorewall, forwarding) since the pfsense is the gateway. The only thing to remember is that when you reboot the proxmox server your network will be gone for all other boxes, until it comes back up obviously.

You could get a second box and have two pfsense KVM's running with virtual IP's and carp for an HA router setup. I have this setup for a small business install and it is surprisingly fast. Now I can reboot one node for kernel updates and don't loose any network access for inbound or outbound traffic. I even have a site to site VPN setup for an remote office and the failover of the OpenVPN server works great as well.
 
You must log in or register to reply here.
Top Bottom