Can Proxmox use HDFS as a secondary datastore alongside LINSTOR? Would NFS Gateway work?

[averageITguy]

Hi Everyone, We are currently testing Proxmox to see if its a solution we can move to. Right now in our testing we have a 3-node Proxmox Cluster with LINSTOR/DRBD as our primary storage backend. We also have a separate High Performance Cluster in our environment which runs HDFS as a single large shared filesystem across all nodes. Our goal is to minimize storage buildout on the Proxmox side by leverage the large amount of storage in our HPC/HDFS stack. We are looking at a two datastore layout.

Proxmox Cluster - 3 nodes.
Datastore 1: LINSTOR/DRBD - fast, HA, critical VMs
Datastore 2: HDFS - large capacity, non-critical VMs

This would be similar to having a fast SSD datastore and a slower HDD datastore in the same hypervisor cluster. I know Proxmox does not natively speak HDFS. Linstor manages our primary storage and that works well. HDFS has a NFS gateway feature that can expose the filesystem as an NFS mount.

Our question is exposing HDFS via NFS Gateway and adding it as an NFS datastore in Proxmox work for running VMs? Has anyone done something similar or have experience running VMs off an NFS-backed HDFS datastore in Proxmox?

We understand that HDFS was designed for large storage but are curious whether there is any practical path to leveraging that existing storage for Proxmox non-critical VMs. The goal is to purely leverage existing storage capacity rather than building out more storage on the Proxmox side. Just want to know if its worth exploring.

Thanks for any insight.

 

[alexskysilk]

Our question is exposing HDFS via NFS Gateway and adding it as an NFS datastore in Proxmox work for running VMs?

I cant speak for the idiosyncracies of hdfs, but the approach is supportable in general and raw/qcow2 over nfs is a first class citizen with PVE. Depending on the IOPs capabilities of the back end it may or may not be suitable for virtualization but thats a different discussion.

We understand that HDFS was designed for large storage but are curious whether there is any practical path to leveraging that existing storage for Proxmox non-critical VMs.

Your specific deployment is good for the same purposes you originally deployed it. if the workload you intend to deploy on pve is similar it will work as well.