Proxmox for Data Science?

[phillipshield]

Happy New Year!

I'm studying the possibility of implementing a Proxmox cluster to be used on data science computing servers. I need help making a good decision.

We have 2 branches located 30km apart. In both branches we have:

• A computing server* with 2x CPU sockets (Intel Xeon Gold 6248R) and 768Gb RAM and 10x 2Tb SSDs
• A storage/log-in/scheduler/web servers with 2x CPU sockets (Intel Xeon Silver 4216) and 64Gb RAM.
• A DAS with a lot of SSDs.

Plus a single AMD computing server (with similar specs) located at the first branch. *Also in this same branch, the Intel computing server has a nvidia tesla GPU.

What I would like to do is to have one or more cluster with:

• The computing nodes, to run VMs and/or Containers, whatever performs better, to do data science computing.
• The storage/log-in/scheduler nodes running VMs with all the supporting services, including the migration of the storage and web servers from a third branch soon to be retired.
• Snapshots and cloning capabilities.

I'm currently evaluating different file systems, but there seems to be just way too many options and the information is not usually easy to digest. What do you recommend?

 

[oguz]

hi,

you have quite a lot of RAM and space, so a good option could be ZFS with software RAID for extra redundancy? [0] [1]

The computing nodes, to run VMs and/or Containers, whatever performs better, to do data science computing.

not sure if VM or CT is better, that depends quite a bit on the stuff you're running. if you need a GPU for the calculations then it might make sense to do a VM with passthrough [2]

hope this helps!

[0]: https://pve.proxmox.com/wiki/ZFS_on_Linux
[1]: https://pve.proxmox.com/wiki/Storage:_ZFS
[2]: https://pve.proxmox.com/wiki/ZFS_on_Linux

 

[phillipshield]

hi,

you have quite a lot of RAM and space, so a good option could be ZFS with software RAID for extra redundancy? [0] [1]


not sure if VM or CT is better, that depends quite a bit on the stuff you're running. if you need a GPU for the calculations then it might make sense to do a VM with passthrough [2]

hope this helps!

[0]: https://pve.proxmox.com/wiki/ZFS_on_Linux
[1]: https://pve.proxmox.com/wiki/Storage:_ZFS
[2]: https://pve.proxmox.com/wiki/ZFS_on_Linux

Thank you Oguz.

 

[phillipshield]

What if one of our requirements is the use of RAID 5?

 

[Dunuin]

ZFS can do something like raid5. Its called raidz1. But if you only got HW raid controllers and no HBA you shouldn't use ZFS ontop of that.
But raid5 in general isn't great as a VM storage because IOPS won't scale with the number of disks.

 

[fv1]

What about GlusterFS, is it an option and why?