Enterprise Environment: Proxmox with Subscription, TrueNAS Enterprise and zfs-over-iscsi

[ulysse31]

Hello all,

On the company I work for, we use Proxmox as a production environment and they are supported with subscription. We are quite happy of its stability and its feature rich GUI.
On the other hand, we have a production storage, which is a TrueNAS Enterprise, with production support form iXSystems.
We were very interested in the zfs-over-iscsi feature : the possibility to have easily per VM zvol with its snapshots directly on the production storage ...
We were a bit disappointed when we discovered that TrueNAS/FreeNAS type of storage, are not directly supported by Proxmox (no direct template available).

We know that there is third party repo that can bring "freenas-proxmox" plugin, and with it support for freenas/truenas storage type for zfs-over-iscsi. It seems this project exists since a while and seems working following the various comments.

Unfortunately, as a company, we cannot rely on a third party repo in a production environment, because, what if a Proxmox upgrade breaks the plugin ? We cannot imagine having all our VMs broken/unavailable because of update mismatch ...
That is why I am asking : Does Proxmox intend one day to integrate TrueNAS/FreeNAS template directly ? or will it be kept as a third party repo forever ? in that last case, it would mean it will never be production compliant ...

Thanks a lot for your answers,


Regards,

--
Ulysse31

 

[LnxBil]

AFAIK it depends on the used iscsi target implementation. IIRC, on the FreeNAS plugin side, they map each api-call from the PVE side to the FreeNAS rest api in order to get everything working. If you would have the same iscsi target implementation, it should work out of the box, because PVE will just ssh into your box and create the zfs datasets as it sees fit. I cannot say what's the case on a TrueNAS enterprise, you have to check that.

Also, since the beginning of the FreeNAS-ZFS-over-iSCSI-patch, there was the introduction of the pve storage plugin, which could be used very easily to add new storage systems and may be exactly what you want (from a dev perspective), I don't know if the plugin already complies with that, but it is a good starting point.

 

[bbgeek17]

Unfortunately, as a company, we cannot rely on a third party repo in a production environment, because, what if a Proxmox upgrade breaks the plugin ? We cannot imagine having all our VMs broken/unavailable because of update mismatch ...

When using two independent products you want to be aware of the support matrix between them. If you had a Cisco Fiber switch and Dell SAN - you'd never want to upgrade one of them without checking with the other vendor.
It is absolutely the same for PVE&FreeNAS. Either you need to be very aware of what the changes are for each of your upgrades (including OS level updates: userland packages (i.e. iscsiadm) and Kernel), or you need to reach out to each vendor before you push the upgrade button.

The above requirement, however, has a serious flaw in your situation. PVE does not directly support FreeNAS. The iscsi/ZFS plugin is built and tested against well known OpenSource packages/OS. My guess is that FreeNAS is not part of PVE test cycle in any way. They dont track FreeNAS Kernel changes or packaging. There was, indeed, recently an issue with a similar storage product (https://forum.proxmox.com/threads/issue-after-upgrade-to-7-2-3.109003/page-2).

On the other hand, Proxmox is not part of the FreeNAS test cycle either. As you said, the two are glued together via a 3rd party maintained repo. Looking at the commit activity, the repo is active and the main contributor is engaged. You may want to reach out directly and inquire about support if that's important to you.

That is why I am asking : Does Proxmox intend one day to integrate TrueNAS/FreeNAS template directly ? or will it be kept as a third party repo forever ? in that last case, it would mean it will never be production compliant ...

Its impossible for PVE to take in a 3rd party repo without author's explicit permission and/or support. Generally, if someone wants to submit code to PVE they have to follow an appropriate procedure https://pve.proxmox.com/wiki/Developer_Documentation. Accepting this code would mean that PVE, as a company, becomes responsible for maintaining it. I'd say the chances of this happening are very low.

Proxmox provides good integration for most commonly used storage protocols (iSCSI,NFS,CIFS) that are served by most free and commercial storage products. For a more intricate integration Proxmox/PVE provides an API for 3rd party plugins that interested vendors can utilize to participate in PVE infrastructure. As an example, Blockbridge has invested in creating one. We will always maintain and support it. Proxmox6 and Proxmox7 are part of our CI/CD - every single code change to our product is always tested through entirety of PVE related storage operations. This provides our common customers with a fully supported environment.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox