Proxmox in an Academic lab environment- Experience and best practices?

[TomasB]

Hi all,

Proxmox is truly great and I have used it at home for about a year for many purposes. Now I'm considering upgrading some of my servers at my lab and would need some input on what would be the best practice and experience from others. I lead a neuroscience lab focusing on gene therapy, bioengineering and bioinformatics. All employees are now expected to learn and utilize bioinformatics in their work, some full time and others intermittently. Our datasets are huge and ever expanding and thus have significant computing and memory needs. Thus, it is not feasible for each student to have their own computation workstation but we have a group of servers which they run their analysis on. We also need to make sure that we can make the analysis reproducible and sharable to others so we utilize container technologies a lot. Many of our students coming in are also meeting linux/bioinformatics for the first time so they have a lot to take in and need to break things to learn.

Here is what I would like to do: Setup Proxmox and develop template LXC containers and VMs (depending on the task) which I then will start up for each student/project. This will give them full autonomy in this but ensures that they cannot destroy each others work, I can restore containers/VMs to functional states and I can load balance/move the containers between the servers. All in all, I think that this will work great for us.

Now to the question: Our university network is very restrictive and all computers are given a public IP. I also need to ensure that all the tools the students will install (probably with lousy passwords as well) will not be hacked. My wish would be to setup a NAT/firewall which the students would connect into using IPsec VPN with a shared secret (so that it is easy, working with Win/Mac without additional software et.c.). Then the NAT/firewall would also distribute stable internal IPs so that the students can know which IPs are pointing to their containers. All containers would through this have internet access.

What would be the recommendations for me to accomplish this? Are there any suitable tools / tutorials on how to accomplish it entirely in software on one of the Proxmox servers? Does anyone have experience in setting up such a system? Alternatively, is it a better option to handle this with a hardware router such as a Ubiquity EdgeRouter? I run those at home and in the summer cottage and have been very satisfied with them.

 

[guletz]

Hi @TomasB

Can you give more details?

the students would connect into using IPsec VPN with a shared secret

From where ? From outside university network only, or maybe both(from inside)?

I also need to ensure that all the tools the students will install (probably with lousy passwords as well) will not be hacked

Very hard to do ....!!!!

All containers would through this have internet access.

- for what kind of services? http/https only is not OK ?

 

[TomasB]

Hi @TomasB

Can you give more details?


From where ? From outside university network only, or maybe both(from inside)?



Very hard to do ....!!!!



- for what kind of services? http/https only is not OK ?

–Initially (and most likely permanently) this would only be reached from inside the university.

– It is indeed a challenge but I hope a firewall/nat will reduce the risks somewhat.

–I would prefer if they could reach everything from the inside out (not vice versa) but at least http/https and ssh would be needed.

 

[guletz]

–Initially (and most likely permanently) this would only be reached from inside the university.

– It is indeed a challenge but I hope a firewall/nat will reduce the risks somewhat.

–I would prefer if they could reach everything from the inside out (not vice versa) but at least http/https and ssh would be needed.

Some though:

- you can use a VM with a http/https proxy for any traffic that will go from students CT to Internet (where you can make many ACLs that fits your needs) - this will help you to also reduce your bandwith (cache) and improve the attack surface
- you can also make a setup for transparent proxy, so the CT do not need to be modified for proxy use
- you will need to use nat only for proxy VM for http/https and for ssh only

The most interseting problem to solve (from my point of view) is ipsec that will need to push for any clients the proper routes to their work CT. Another probem will be that you will need to do many tasks for this:
- credentials for any students (user/password add and remove and setup the proper routes add/remove for their CT), and if you need this very often and/or for many students then this could be a problem(or a very big problem)
- take also in account that ipsec is not so friendly for firewalls (including university network)

One solution could be to use a dhcp server in the same proxy vm for any CT, and to push then this ip pool to any ipsec client (I guessing that your students will acces their CT using ssh) You say that any student must know the ip of their CTs.... for this you can use the same dhcp server who can also create a dns record (so you need a internal dns server - could be on the same VM as for proxy) using a let say a smart naming convention like:

- a student can have a uniq ID, and base on that ID, the hostname of his own CT could be like ID-containerID, where containerID={001,002,..n}

I guess only, that It will be more simple to manage all of this using pppoe insted of ipsec (lower resurce , ...)

Anyway this project of you is not so simple, and you need many knowledge to finish. In other simple words you need to make a wall using many bricks and all of the briks need to fit in this wall... and if only a single brick is not fit in this wall, then bad things could be happen; )

Good luck, you will need!

 

[RobFantini]

Hello
First let me say I am far from a vpn/network expert.

We have used pfsense openvpn servers for around 8 years. We paid pfsense tech support to do the initial setup . We use peer to peer vpn for remote offices and Remote Access ( SSL/TLS + User Auth).

It has been easy to manage especially using the OpenVPN Client Export add on package.

here is a link if you care to check it out: https://www.netgate.com/docs/pfsense/vpn/openvpn/index.html

 

[TomasB]

Thank you both for very good insights and recommendations. There is clearly a lot of things to get right here and clearly there are security implications if things work less than optimally. I'm confident that everything can be virtualised in software but after reading these comments and other discussions around I have decided for a hardware router/firewall solution as a starting point. This will be the Ubiquiti EdgeRouter ER-6P. For the initial setup all the required services will run in this, DHCP, firewall, NAT, and VPN. I have had great experience with their hardware/software solutions privately and I think that it will be good enough to get started. All the terabytes of sequencing data will already be stored on the servers behind the firewall so the single gigabit external interface will not cause a significant bottleneck. Then when everything is up and running I may virtualise the services on by one until the point when I can potentially remove the EdgeRouter.

 

[guletz]

This will be the Ubiquiti EdgeRouter ER-6P

Yes, it could be a good start point! Even better will be in my own oppinion a Mikrotik who can do even more Also you could use a CHR virtualised image(can be used in Proxmox) from Mikrotik. If you have already some experience with EdgeRouter, will be very easy ....

 

[RJ Dowell]

TomasB said:
Proxmox is truly great and I've been using it for my personal purposes for a year already. I'm a head of a neuroscience lab that deals with bioengineering, gene therapy and bioinformatics. And I need something like a computation workstation for students to be able to do my homework for me tasks as many students need to break things to learn
...

>>>>>
Quote collapsed
>>>>>

Hi,
Can I run pfsense as a virtual machine in Proxmox? And then add the ER-PoE. So in case the pfsense fails, the ER-PoE could be a good backup, right? When I add Ubiquiti EdgeRouter Pro, will I be able to connect to my Proxmox server without issues?