ZFS over iSCSI for HA

[Blaise Saunders]

Hi there!

I have 5*ProxMox servers running 5.2-9, one of which has a number of hard drives in them, I would like to share those disks to the other 4 servers in the cluster so that I can run HA. (to clarify, the other 4 servers contain minimal storage that I do not wish to store VMs on)

My understanding is that I need to create a ZFS pool and share it over iSCSI, but I'm a little unsure about how to proceed, is it a pool or volume I need to share? And I am confused as to the use of the SSH commands on the Wiki page... Could somebody please point me in the direction or clarify?

Thanks in advance!!

- Blaise

 

[Klaus Steinberger]

you can do that of course, but it is not HA then!

Think of it: The server with the disks is the single point of failure

 

[Blaise Saunders]

Think of it: The server with the disks is the single point of failure

Fair point! We are poor for hardware at the moment, the SAN is quite reliable but the compute nodes are not, as the business takes off we plan to add disk redundancy...
Would our design choice limit our abilities to duplicate the SAN?

 

[Klaus Steinberger]

hmm , you could pack drdb underneath for replication, but it has its own pitfalls.

can you spread out the disks equally to the nodes? Then CEPH would be an option.

 

[Blaise Saunders]

can you spread out the disks equally to the nodes? Then CEPH would be an option.

The nodes vary greatly in speed and in networking, and we have invested in some Mellanox ConnectX cards to connect the main server to the SAN in hopes of using RDMA with iSCSI, so that is the preferred technology for now... Also the nodes are connected with teamed 4 port ethernet cards.
For some background, we are hoping to run some data backups for small businesses, we have some redundant NASs to store the data on, but we need some servers for ingress, verification, and hopefully virtual booting at a later date (probably to a separate ESXi server).

 

[Klaus Steinberger]

ok, why not use the redundant NAS also for images?

 

[Blaise Saunders]

ok, why not use the redundant NAS also for images?

Well the NASs only have 4GbE (teamed) and are lacking in specs (dual core atom, 4GB SO-DIMM etc.) we figure a beefier pair of SANs would work better

 

[LnxBil]

we figure a beefier pair of SANs would work better

I'd first go with a dual-headed single box 2 HE SAN, e.g. Fujitsu Eternus DX100 which is a dumb block storage that is therefore very cheap, but in itself totally redundant if you have redundant power lines. To have a ZFS-based redundant system, you have to have external software, because ZFS itself is not clusterable.

My understanding is that I need to create a ZFS pool and share it over iSCSI, but I'm a little unsure about how to proceed,

ZFS-over-iSCSI that is currently implemented is for special backend storage devices that will be controlled by PVE. There is ongoing development for Linux based backends and a FreeNAS backend. For more information, please refer to the article on the wiki:

https://pve.proxmox.com/wiki/Storage:_ZFS_over_iSCSI