TrueNAS to provide storage to Proxmox Cluster. Does it make sense?

[ihr]

Hello,

I’m struggling with a dilema: Does it makes sense to install TrueNAS to provide shared storage to a Proxmox cluster?

I already have 3 bare metal computers configured in a cluster. But at this point, the virtual machines running in the cluster use the storage of the host. I want to migrate the storage to a centralized storage (in another bare metal computer) and I’ve two options:

1) Install TrueNAS in that computer and, somehow, provide storage to the datacenter. I Don’t know if this is possible at all or if the storage has to be added to each one of the servers in the cluster.
2) Install proxmox in the computer and share the storage from it.

What do you think?

 

[Dunuin]

You could use a 4th TrueNAS server so provide a NFS share as a shared storage for your 3 PVE nodes. iSCSI would also be an option.
But this is also not ideal. Great thing about a cluster is, that it is redundant and your services can still run in case a server fails. With all data centralized on a single server you again get that single point of failure. If your TrueNAS server then got a problem, all 3 PVE nodes will stop working.
Would be better to skip that TrueNAS server and use Ceph as a shared storage distributed across all 3 PVE nodes. Then any of those 3 PVE nodes could fail and all data would still be available to the remaining two nodes.

 

[ihr]

Thank you @Dunuin for your response. It is quite the right answer based on the information I provided in my first email. Let me develop further to constrain my use case.

1) The 4 servers run at different in hardware. They all have decent CPUs for the purpose (relatively old, i7 and i5), different memory size and different hard drives attached. Only one of them (the storage server) contains 6 HDD 6Tb each which is enough for the purpose.

2) It is just a "development and testing environment" for internal use in the company. We are not providing external services from that infrastructure and we don't need maximum performance.

CEPH will not work for two reasons: Hardware differences and Network is just 1Gb and contains another big bunch of PCs on the same physical lan.

So, the alternatives are (as suggested)
1) Install TrueNAS on the storage server and provide iSCSI (or NFS) shares from it
2) Install Proxmox on it and try to do the same. But, I think Proxmox cannot share local media unless CEPH is installed. Am I right?

 

[Dunuin]

2) Install Proxmox on it and try to do the same. But, I think Proxmox cannot share local media unless CEPH is installed. Am I right?

At least not through the webUI/API. But nothing prevents you from manually setting up NFS exports on the CLI. PVE is more or less a Debian, so everything that is possible with a normal Debian server should be possible with PVE as well.

 

[ihr]

That's right! Thank you!

 

[jtremblay]

Thank you @Dunuin for your response. It is quite the right answer based on the information I provided in my first email. Let me develop further to constrain my use case.

1) The 4 servers run at different in hardware. They all have decent CPUs for the purpose (relatively old, i7 and i5), different memory size and different hard drives attached. Only one of them (the storage server) contains 6 HDD 6Tb each which is enough for the purpose.

2) It is just a "development and testing environment" for internal use in the company. We are not providing external services from that infrastructure and we don't need maximum performance.

CEPH will not work for two reasons: Hardware differences and Network is just 1Gb and contains another big bunch of PCs on the same physical lan.

So, the alternatives are (as suggested)
1) Install TrueNAS on the storage server and provide iSCSI (or NFS) shares from it
2) Install Proxmox on it and try to do the same. But, I think Proxmox cannot share local media unless CEPH is installed. Am I right?

I'm going through this same process. In my first deployment, I used the (4th machine) Truenas machine to export NFS shares to the 3 PVE machines. I have two networks, users LAN and storage LAN. This all works well. For my second deployment, I'm using 5 machines, two Truenas, three PVE.
I was going to use glusterfs to create a mirrored pair, but....
Now I'm just going to try "replication" or "Syncthing" to create a mirror.

 

[cave]

1) Install TrueNAS on the storage server and provide iSCSI (or NFS) shares from it

Install Prox on the fourth, add it to the cluster and install TrueNas as a VM with NFS Share to the cluster.

Addon: a LXC with Debian to Run as Proxmox Backup Server to the Cluster and use the NFS Share as the PBS DataStorage.

Will work, but will have a non-redundant Bottleneck to the Cluster as Dunuin mentioned.

If there are no VM's hosted on the TrueNas Share as Datastorage, just random media, your cluster will still 'work' if node 4 breaks.