I’m currently working for a company that runs a vCenter cluster using 1 Gb/s iSCSI storage on Synology SANs. Everything works perfectly snapshots, thin provisioning, multipath, etc. We’re now looking to migrate to Proxmox, and the big question is: how can we achieve the same storage capabilities under Proxmox?
Current setup
5-node Proxmox cluster
Around 650 GB RAM total
24 cores per node
1 Gb/s network (yes, I know…)
Mechanical SAS/SATA drives (7.2K / 15K RPM)
No SSDs available
Based on the Proxmox storage documentation, only two storage options seem to meet our needs:
- Shared storage
- Native snapshot support
- Thin provisioning
Those two options are:
- Ceph
- ZFS over iSCSI
Ceph :
I tried setting up a small Ceph cluster with 12 OSDs (2 TB SATA 7200 RPM each). The result: very poor performance, especially on writes and IOPS. The 1 Gb/s network wasn’t the main bottleneck it’s clearly the mechanical disks. While Ceph does technically meet our requirements (shared, snapshots, thin provisioning), it’s simply not viable without SSDs. It feels like overkill for our hardware.
ZFS over iSCSI :
I also tested ZFS over iSCSI, using a Debian server with targetcli (LIO) as the iSCSI provider. The performance is much better than Ceph, and close to what we had on vCenter. However, there are too many persistent issues:
- Cloning VMs fails with errors like GET_LBA_STATUS
- Some operations are very slow
These issues are well-known and unresolved for years:
https://bugzilla.proxmox.com/show_bug.cgi?id=4046
https://forum.proxmox.com/threads/e...est-5-ascq-invalid_field_in_cdb-0x2400.95416/
https://forum.proxmox.com/threads/lsi-sas2308-scsi-controller-unsupported-sa-0x12.78785/
It seems the root cause is with LIO, which doesn't properly handle certain SCSI commands expected by Proxmox.
My question :
I’m trying to build a shared, snapshot-compatible, thin-provisioned storage solution that works with limited hardware (mechanical drives, no SSDs, 1 Gb network).
Has anyone managed to achieve this kind of setup reliably?
Are there any viable alternatives to Ceph or ZFS over iSCSI?
Is there any way to reuse my Synology SANs with Proxmox in a proper way ? (NFS is too slow for our usage)
I’d really appreciate any feedback !
Thanks in advance!
Current setup
5-node Proxmox cluster
Around 650 GB RAM total
24 cores per node
1 Gb/s network (yes, I know…)
Mechanical SAS/SATA drives (7.2K / 15K RPM)
No SSDs available
Based on the Proxmox storage documentation, only two storage options seem to meet our needs:
- Shared storage
- Native snapshot support
- Thin provisioning
Those two options are:
- Ceph
- ZFS over iSCSI
Ceph :
I tried setting up a small Ceph cluster with 12 OSDs (2 TB SATA 7200 RPM each). The result: very poor performance, especially on writes and IOPS. The 1 Gb/s network wasn’t the main bottleneck it’s clearly the mechanical disks. While Ceph does technically meet our requirements (shared, snapshots, thin provisioning), it’s simply not viable without SSDs. It feels like overkill for our hardware.
ZFS over iSCSI :
I also tested ZFS over iSCSI, using a Debian server with targetcli (LIO) as the iSCSI provider. The performance is much better than Ceph, and close to what we had on vCenter. However, there are too many persistent issues:
- Cloning VMs fails with errors like GET_LBA_STATUS
- Some operations are very slow
These issues are well-known and unresolved for years:
https://bugzilla.proxmox.com/show_bug.cgi?id=4046
https://forum.proxmox.com/threads/e...est-5-ascq-invalid_field_in_cdb-0x2400.95416/
https://forum.proxmox.com/threads/lsi-sas2308-scsi-controller-unsupported-sa-0x12.78785/
It seems the root cause is with LIO, which doesn't properly handle certain SCSI commands expected by Proxmox.
My question :
I’m trying to build a shared, snapshot-compatible, thin-provisioned storage solution that works with limited hardware (mechanical drives, no SSDs, 1 Gb network).
Has anyone managed to achieve this kind of setup reliably?
Are there any viable alternatives to Ceph or ZFS over iSCSI?
Is there any way to reuse my Synology SANs with Proxmox in a proper way ? (NFS is too slow for our usage)
I’d really appreciate any feedback !
Thanks in advance!
