iSCSI on Proxmox itself for SQL Server Failover Cluster Instance (FCI) VMs

[Ronan311]

Hi, we're currently working on a project that requires us to implement a SQL Server Failover Cluster Instance (FCI) using Windows Server Failover Clustering (WSFC) with iSCSI-based shared storage.

The challenge is that our SAN only supports Fibre Channel (FC). As a result, the Proxmox cluster can only present the SAN storage to VMs as virtual disks backed by FC storage.

I would like to ask if there is a supported method to create an iSCSI gateway or target on the Proxmox cluster itself so that the Windows VMs can connect to the shared storage using the Microsoft iSCSI Initiator. Ideally, the iSCSI service should also be highly available across the Proxmox cluster rather than being tied to a single Proxmox node so we can avoid a situation when an entire Promox host go down, storage and service still available.

 

[mfederanko]

It should be possible to pass through the FC disks to the guests by manually editing the VM configs.
Depending on your exact situation you will also have to set the following fields (or more) in the VM config:
* serial=...
* wwn=...
* shared=1

If you use multipath you will also have to set [0].
Please test the setup thoroughly, it isn't something we currently officially support in Proxmox VE.

Alternatively you can also set up a iSCSI target using standard tools available in Debian, that is not integrated into PVE though.
The arch wiki has a short but to the point wiki entry on how to setup a iSCSI target with LIO [1].

[0] https://www.qemu.org/docs/master/sy...multipath-devices-and-persistent-reservations
[1] https://wiki.archlinux.org/title/ISCSI/LIO

 

[bbgeek17]

Manuel already noted this is outside what PVE supports. I'd go a step further: I can't recommend doing either of these in production.

It's worth being clear about what WSFC actually needs from that disk. It isn't so much the block device as SCSI-3 persistent reservations, and that's where both approaches get complicated.

One thing to get out of the way up front: with a passthrough FC LUN, you should plan on never migrating the SQL VMs. Reservations are bound to the initiator, so a VM that moves to another host shows up at the array as a stranger. HBA passthrough can't migrate at all. And pausing a WSFC node long enough to move it can trip the heartbeat and cause a failover on its own. The FCI is your HA. Pin the two VMs to separate hosts and let WSFC handle host loss.

Even with migration off the table, the FC LUN passthrough has problems. The initiator is the PVE host's HBA, not the Windows VM, so the array registers keys against the host's WWPNs. If both SQL VMs ever end up on the same host for some reason, they share one I_T nexus and overwrite each other's keys, so anti-affinity becomes a correctness requirement rather than a preference. With multipath, every path has to carry the registration, which means qemu-pr-helper built against libmpathpersist and multipathd configured with reservation_key so it re-registers paths that come back. PVE has no built-in option for any of this. You'd add the pr-helper object to the VM by hand via the args: line, run qemu-pr-helper as a separate service on every host, and PVE would be unaware it exists.

LIO is a different story. Its PR implementation is functional and Windows cluster validation passes against it; that's not the concern. The concern is simple: it's not HA. It runs on one host, and when that host reboots or dies, both SQL nodes lose the disk at the same time. If you do it anyway, the backing storage should be directly on the FC LUN... not a qcow2, not a file on a filesystem, and no write caching anywhere in the path. SQL Server assumes an acknowledged write is on stable media, and it counts on that for data integrity.

If I were in your shoes, I'd confirm with the array vendor whether iSCSI can be licensed on it. That would make most of this go away. You can bridge the storage network to your VMs, and then the guest IQNs are the initiators, and everything is vendor-tested.

If it really has to be an FCI on this SAN, then pass real HBA ports (or NPIV/SR-IOV functions) through to the two VMs and pin them. You've already given up migration, so you lose nothing.

Whichever way you go, run the SCSI-3 PR validation test in the final topology and again after any host/path change. The failure conditions are where the problems hide.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox