Proxmox in a Data Center

[kobaltz]

I'd like to think that I'm fairly tech savvy. I'm able to be brought up to speed on new topics fairly quickly. However, I am always in search for more knowledge and proper procedures!

I have been using Proxmox in by basement cluster for a few years now. I've never ran into any major issues. I love Proxmox. However, I do have some confusions on the best practices when using Proxmox in a Data Center. Currently in my basement, I have a Cable Modem > Netgear Router > Managed Switch > 2 Proxmox Servers. I've never really worried about security since the Proxmox Servers have always been behind the Netgear Router's firewall. The only port forwarding I enabled was 80 and 443 to a Container on one of the Proxmox servers. It acted as a reverse proxy gateway to handle the internet traffic based on the domain name. On the servers, I had multiple Ruby on Rails containers, Web server containers and MySQL Server containers. Everything works flawlessly.

My confusion is if I were to buy a couple of rack servers and want to put these in a local data center, how would I maintain the security? What is the best practice?

Should I purchase a hardware firewall that will be in between the network at the data center and the servers I'm racking?

Would this firewall contain all of the Public IP Addresses and I would port forward them as needed?

Outside of using a DHCP server, what is the best practice for having local IP addresses for the containers?

I think that I'm just making it more complicated than it needs to be.

 

[anthonysomerset]

Its generally simpler to manage to get a Hardware firewall to put your servers behind. Proxmox runs no iptables firewalls on the hosts and it would get pretty complex and messy to do this yourself and you would then also have to run software firewalls on all your VM's as well

Most providers will setup Cisco hardware firewalls in a Routed fashion - that means your public IP's are managed on the firewall and you either use Port Forwarding (known as PAT on cisco devices) or Static NAT rules if you have enough IP addresses to do 1:1 NAT to internal addresses

The benefit of using most DC grade (ie rack mount) firewalls is that they can do VLANs so you can totally isolate say your Proxmox cluster network from your VM network and not allow communication between them without having to run 2 physically separate networks (useful if your budget is low)

Internal IP's depends on the network setup and how you are isolated in the DC (this does vary from DC to DC) but if everything is behind a firewall and on your own switch just do some thinking, how many VM's are you going to run? that will let you decide what private subnet size to use and the like - IP addressing, you have 2 options right now, run your own DHCP server, either from the firewall if it can, or a VM/server or just have an excel spreadsheet and statically assign IP's at setup time

Proxmox Feature Request:
What i'd love to see on that front in future though is for proxmox to run internal DHCP on the host servers so that VM's can be DHCP configured and proxmox can control the IP on the VM/Container (this isn't such a problem with openVZ just KVM) the only "competing product" that i know of that does this is solusVM and I just started migrating from them because of the lack of live migrations and shared storage support

 

[goeldi]

iptables ... it would get pretty complex and messy to do this yourself and you would then also have to run software firewalls on all your VM's as well

I don't think that it is messier than managing them via a hardware firewall.
The iptables on the node will be simple (everything closed except e.g. port 8006 and 22).

 

[kobaltz]

It sounds like a hardware firewall will be what I need and the best route to take. Thanks guys for your suggestions!

 

[rugby]

We've got a PFSense firewall there handling NAT, DHCP, and a few other services and our 4 Proxmox servers behind it. I nat everything and do 1-to-1 NAT for servers that need it. this way I can run more servers than I have public IP's for.

 

[RobFantini]

Rugby, could you help me understand the use and benefit of " 1-to-1 NAT for servers " ?

We also use pfsense and I'm looking for way to improve security.

 

[rugby]

1-to-1 NAT makes the nat'ed server use it's external Nat'ed IP as it's "from" address. It uses this instead of the default gateway of the firewall in the DC.

For instance, we have a 64.X.X.X IP for our gateway externally, but out IP range is 216.X.X.X for our allocated addresses. I have several email servers Nat'd with 1-to-1 and the emails from those servers will come from the 216.X.X.X address instead of the 64.X.X.X address. Pretty important for email servers to send and receive on the same IP.

 

[RobFantini]

Thank you, that makes perfect sense.

 

[c0mputerking]

I may be off base, or missing the point, but i have setup sophos UTM 9 within my proxmox server using a KVM it is free if just using basic functionality which is all i/u would need. You could use PFsense like this, but being BSD it does not support virtio devices, yet so i went with UTM 9 as it has support for virto network and hardrives.

You could use UTM 9 to do all your DNS, DHCP. ect, i do not and have a separate OVZ container running DNS, and DHCP the latter of which i rarely want or need as all my machines are set to static internal IPs.

It was a bit of a complicated setup and i had lots of help here on these forums getting it going, but it seems to works flawlessly and being a KVM makes it nice and tidy not having to add anything to your hypervisor but still being housed within your proxmox system/hardware.

I sure you could find my posts about it on here, or if you like i could go into more detail or shoot you off some config examples.

 

[Kaya]

I choose to use proxmox VM with SHOREWALL on it as firewall, I think it's the more flexible solution.

 

[rugby]

I choose to use proxmox VM with SHOREWALL on it as firewall, I think it's the more flexible solution.

Just out of curiosity what makes it more flexible then having a dedicated firewall?

 

[Kaya]

Just out of curiosity what makes it more flexible then having a dedicated firewall?

One , for example, is that on hardware failure u can immediatly migrate the firewall on another server.
And u can use virtual interface even if u have not enought physical ethernet port on.

 

[anthonysomerset]

I don't think that it is messier than managing them via a hardware firewall.
The iptables on the node will be simple (everything closed except e.g. port 8006 and 22).

well theres no central management as you have to put software firewalls on every VM as well, also using openvz containers further complexifies the firewall setup (you either have to explicitly pass off VZ container traffic by IP to the container or manage the VZ container network access from the hostnode firewall - neither of which allow easy migration support)

I may be off base, or missing the point, but i have setup sophos UTM 9 within my proxmox server using a KVM it is free if just using basic functionality which is all i/u would need. You could use PFsense like this, but being BSD it does not support virtio devices, yet so i went with UTM 9 as it has support for virto network and hardrives.

You could use UTM 9 to do all your DNS, DHCP. ect, i do not and have a separate OVZ container running DNS, and DHCP the latter of which i rarely want or need as all my machines are set to static internal IPs.

It was a bit of a complicated setup and i had lots of help here on these forums getting it going, but it seems to works flawlessly and being a KVM makes it nice and tidy not having to add anything to your hypervisor but still being housed within your proxmox system/hardware.

I sure you could find my posts about it on here, or if you like i could go into more detail or shoot you off some config examples.

Pfsense/FreeBSD does support VirtIO but its not an out of the box install - see http://doc.pfsense.org/index.php/VirtIO_Driver_Support

i wouldn't recommend running the firewall in a VM on the host server you want to protect - there is a chance to get a race condition on network which could break things and leave you stuck - always better to have the firewall as a separate device when you can (either a purpose built firewall hardware or server running something like pfsense or untangle etc)

 

[typo3usa]

We use proxmox in our datacenter - and absolutely love it.
I dropped use of all our VMWare stuff and have migrated to proxmox - now have well over 200 nodes.

I have chatted with Chris from PFSense a number of times - and will tell you - He is one of the best guys for support - he monitors the community very well.
PFSense and shorewall are very close in their roots - but I think pfSense is a much better solution

We have ran into some issues as of late running BGP and grabbing full tables - Chris came on site to help us - and even recommended another solution for BGP when PFSense had a bug that we needed an immediate resolution for.

For places where we need routing that requires support including certifications we use Mikrotik or Cicso - as clients sometimes from the datacenter require more than just an opensource solution.

Mikrotik rocks - blows cisco away in regards to price - but for a firewall solution IMHO you cannot beat pfsense.
Very easy to implement.

hit me off list if stuck - be happy to help - I owe that much to the pf community after Chris came to help.
One other thing to note - he came out on July 4th - yes the 4th on a Holiday to take care of our Issues - can't find folks in the OpenSource community that are willing to do that - that often