3 node shared or replicated?

p0lar

Member
Mar 1, 2019
7
0
21
60
I have 3 node cluster with NFS shared disk and local ZFS disk on each node.

Question is, would shared NFS or replication be better for failover?
 
Shared NFS means yo u have the latest VM/CT state instant, with ZFS replication the VM disk state/content may be 15 minutes old (or whatever your smallest replication interval is).

Also, the whole disk is replicated on all nodes where you have setup a job, if you have a single NFS the VM/CT disk is stored only once (depending on your NFS, this could also not be ideal if it does not provides some sort of reliability/failure correction measurements).

So to give a better answer we'd need to know what your NFS setup would look like.
 
NFS is served from a Gig connected Synology NAS. better than 15 minutes of lag on the local disk.

THanks
 
better than 15 minutes of lag on the local disk.

That depends: If you NAS fails, you will have nothing, it is your single point of failure (SPOF). If you have replicated the data via ZFS, you will have almost up-to-date data on another node.
 
The way that I use proxmox with nfs is this: I have 6 nodes in my cluster and 2 qnap TS451 units. The nas units are setup in raid 6 so their is some fault tolerance. They are also connected to the nodes using 2 bonded gigabit nics.

The vm disks are split between the two nas units and I also have each vm backup every night to both nas units. So I have the vm drive and a backup on one nas unit and another backup on the other.
 
The way that I use proxmox with nfs is this: I have 6 nodes in my cluster and 2 qnap TS451 units. The nas units are setup in raid 6 so their is some fault tolerance. They are also connected to the nodes using 2 bonded gigabit nics.

The vm disks are split between the two nas units and I also have each vm backup every night to both nas units. So I have the vm drive and a backup on one nas unit and another backup on the other.

Better than nothing but still SPOF. If one NAS fails, all VMs that have VM disks on both storages will fail too.

To have a real high available PVE system, you need to have everything at least twice and running without manual intervention. With respect to storage, you need one dual-headed NAS box (or a real SAN) in order to have dedicated shared storage. The other way is to have distributed shared storage via CEPH.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!