Using a Node as a SAN?

[modem]

Quick question. I have a HPE DL350p server that has 3 drive cages with a total of 22 drive bays all connecting to 3 HBA cards. I have ProxMox 8.1 installed on a 256GB SD flash card mounted to the motherboard. Currently I have all 1TB SATA standard consumer drives in those bays, but I eventually would go to 2-4TB drives.

Is it possible to use that server as both a node AND a SAN type setup with it's storage? If so, how is that possible in ProxMox?

Thanks!

 

[waltar]

SAN is block storage and yes you can export your storage as iscsi storage for raw files to other nodes.
But I would use it as NAS nfs file storage to other nodes using with qcow2 files as it's much easier to handle.
But anyway in both cases it's a singlepoint of failure and your vm's/lxc's cannot run further if this node die so monitor it and have spare disk defined.
In case of NAS pve updates with just a reboot are not a problem to virtual machines on other nodes as nfs is intended in protocol.
Don't think that would work that nice with iscsi also.

 

[modem]

Thanks for the reply. I think I need to clarify my question a bit. I do realize SAN's use block storage and iscsi backend for connectivity. My goal is not to make this server an actual SAN system, but rather, what options would I have for using the storage space on this single node when other nodes do not have the same amount of storage space? Especially if I cluster this one server with other servers that have less storage capacity?

Is there any benefit of / possibility of creating a storage pool on the HPE (that has 22 drives) that is shared among 3-4 nodes so that this HPE server acts like a drive shelf where all VM's are stored on a shared storage pool managed by all other nodes? Or is it better to just have different nodes storing their own VM's on them and using the storage pools on the HPE as purely for backup?

I'm just looking for the best setup with one server having this much more storage than all the rest of the servers/nodes/

 

[waltar]

When you use a kind of shared storage you can migrate vm's fast between nodes which takes quiet a while longer with each node has own storage and put the backup on the HPE also.
Just be prepared for any kind of hw crash and have a copy of your vm's+lxc's on another node also to bring your environment up fast again !!
I prefere shared storage as it's easy for daily usage and a switch to other datastore easy which hopefully should not happen unplanned while any node hw crash is just on node less available.
If you do all local and a node hw crash you have ever to do on v-machines restore to other nodes which is more time wasteful.
But however this is very personal on thinking.

 

[bmunier]

Another question for me : are there any technical limitations when we use a SAN in place of intrated storage (HCI) or CEPH ? I had heard about limiting snapshots ? or bakcup problem ?

 

[alexskysilk]

installed on a 256GB SD flash card

bad idea. pve is a full debian, not an embedded OS and will murder your flash card.

 

[modem]

bad idea. pve is a full debian, not an embedded OS and will murder your flash card.

Even though it's only the OS that's located there while the VM stores are on spinning SATA drives?

 

[alexskysilk]

Another question for me : are there any technical limitations when we use a SAN in place of intrated storage (HCI) or CEPH ? I had heard about limiting snapshots ? or bakcup problem ?

It depends how you mean "SAN." In the strictest sense, SAN refers to shared block storage, which in this case means iscsi. Within a PVE environment, iscsi is limited to full fat LVM which eliminates the potential for snapshots. HOWEVER, if you use a supported back end provider such as TrueNAS, you can use zfs over iscsi which will provide the necessary functionality.

Alternatively, you may just share NFS from your central storage, which has no such limitations.

 

[alexskysilk]

Even though it's only the OS that's located there while the VM stores are on spinning SATA drives?

yes.