TrueNAS Storage Plugin

[warlocksyno]

@eugenevdm Are you able to reproduce this issue?

I installed Ubuntu 25.10 on the cluster with the NVMe transport type, added 5GB and was still able to write to filesystem. Moved it over to iSCSI and added another 5GB and was still able to write to the filesystem.

 

[jt_telrite]

Uhh, yes! This should actually be fixed with the "weight" volume that should be created automatically. This keeps the iSCSI target alive on the TrueNAS side. TrueNAS will turn off it's portal advertising when there's no volume being shared. So the plug is supposed to create a "weight" volume that is always alive to prevent that from happening. Let me know if you see that volume, if not, I will look into why it's not being created.



Yes! That's actually in the works at the moment. I'm updating the storage configuration tool to go step-by-step to pick the pool, create a dataset that's optimal for your setup (or put in manual settings), and then enable iSCSI/NVMe blah blah blah. That way it's basically as easy as configuring pools and then creating an API key in TrueNAS, then from that point on there isn't much of a reason to actually log into TrueNAS any more unless you're needing to do some really manual work.



Hmm, are you saying there's orphaned iSCSI targets that are trying to be logged into?

This should be the only thing you need to do in TrueNAS after setting up the dataset you want:
View attachment 92756

Once the target is created with the Portal Group ID and Initiator ID configured, the plugin should take care of the rest. If not, let me know.

And Alpha branch has the latest and greatest, the beta one will have a more stable updates to it. For now I would use the alpha branch while troubleshooting.


That's interesting you are getting some somewhat okay performance on the writes. Do you have sync=off on your dataset?

yes I see and have that "volume"

When I make a mistake in configuration and think I have corrected it, including associating all of the truenas objects as pictured in you iscsi group picture.
this reports no sessions
sudo iscsiadm -m session
this reports none to find
iscsiadm -m discovery -t sendtargets -p 172.16.8.1:3260
some variant of the this is "required"
sudo iscsiadm -m session -r 3
or this
sudo iscsiadm -m node -T "iqn.2005-10.org.freenas.ctl:vm" --op delete
then if it reports a session
sudo iscsiadm -m session
this
sudo iscsiadm -m node -T "iqn.2005-10.org.freenas.ctl:vm" -p "172.16.8.1:3260" --login

this is usually only after I have screwed up , ALOT

ALSO, sync=standard (the default)

 

[Phoenix85]

I‘m trying to understand where the value might be for me. I have a Homelab with Proxmox Server , Proxmox Backup Server and TrueNAS Server and one Server who monitors the others ( Grafana , Node etc. ) Proxmox and PBS store everything in a share on the TrueNAS.

What can I achieve with installing this plugin to Proxmox ?

 

[warlocksyno]

What can I achieve with installing this plugin to Proxmox ?

The short of it is you'd be using TrueNAS like a block storage device. There's less overhead than NFS/SMB, so depending on your network you could see a significant performance increase. Especially if you're using NVMe TCP and have some faster networking equipment like 10GbE+.

@bbgeek17 Actually would be able to give you a good run down.

Their product is built on this entire premise and works fantastic.

 

[Phoenix85]

Hi,

Network is 10GB via DAC Cable.

Are there plans to get it natively integrated into Proxmox ? Or will it always be a plugin via API?

 

[warlocksyno]

I mean, once you install the plugin and get your storage config setup correctly, it makes TrueNAS show up and act almost identical to how you use NFS at the moment. But on the back end the plugin is taking care of everything by sending API calls to Proxmox and to TrueNAS to get them aligned to what you're needing it to do.

AFAIK, eventually, Proxmox Solutions has said they want to have storage plugins configurable via the GUI instead of messing with the storage.cfg file.

 

[bbgeek17]

Thank you for the kind words @warlocksyno

@Phoenix85 , If you’re asking whether Proxmox GmbH would absorb a third-party–developed plugin for a third-party commercially marketed storage product, the answer is very likely “no.”

Pulling such a plugin into the official distribution would imply that Proxmox becomes fully responsible for its future development, testing, and support.

This situation is precisely why a unified, well-defined storage API exists: it allows storage vendors, who have full understanding and control of their products, to integrate with the virtualization infrastructure.

The support for 3rd party storage plugins will always come from the community or storage vendors directly.

PS I do believe there is some work in progress on the framework that would allow storage plugins to also integrate with UI, by extending the API capabilities. However, we have never seen one-time GUI integration as a show-stopper for implementation.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Johannes S]

Since @bbgeek17 mentions the role of storage providers: In theory ixsystems (the company behind TrueNAS) could maintain such a plugin. In practice I wouldn't hold my breath though since they also have virtualization and lxc support in their most recent versions of TrueNAS, so I think it's unlikely that they will help their competitor. I would be happy to be wrong though

The short of it is you'd be using TrueNAS like a block storage device. There's less overhead than NFS/SMB, so depending on your network you could see a significant performance increase. Especially if you're using NVMe TCP and have some faster networking equipment like 10GbE+.

And in theory you don't even need the plugin you can also do this manually since in the end it uses the ISCSI protocol which is already supported by ProxmoxVE. But the plugin is way easier to setup and gives additional comfort features. For this reason I highly appreciate warlocksyns efforts on the plugin, I'm considering switching my manual setup to it. I'm just ab bit anxious how stable it will be given the fact that ixsystems had a tendency to overhaul major features in the last years on a regular base. I don't want to risc to loose the functionality through a breaking change update of TrueNAS

 

[PwrBank]

And in theory you don't even need the plugin you can also do this manually since in the end it uses the ISCSI protocol which is already supported by ProxmoxVE. But the plugin is way easier to setup and gives additional comfort features. For this reason I highly appreciate warlocksyns efforts on the plugin, I'm considering switching my manual setup to it. I'm just ab bit anxious how stable it will be given the fact that ixsystems had a tendency to overhaul major features in the last years on a regular base. I don't want to risc to loose the functionality through a breaking change update of TrueNAS

Thanks

I am keeping in contact with iXSystems about the plugin and it's features. They reached out to talk about what they could do to help with APIs and fixing bugs or features needed for the plugin to function. Hopefully over time I can communicate more with them to head off any issues that may arise.

Personally, the only thing I'm worried about with TrueNAS is their path for containers. Past that, I don't see any reason for them to start limiting API access for datasets and transportation function (NVMe, iSCSI, etc)

And really, if they did start doing stupid stuff - It'll get forked.

 

[warlocksyno]

Yeah, not sure if TrueNAS would really offer their own plugin for Proxmox since that is an iffy place to be. If they plan on expanding into true Hypervisor territory, it'd be a conflict of interest for sure.

But I'd hope they'd stick to just doing what they're good at and prioritizing the storage aspect. If it was storage first and maybe Hypervisor third, sure. It's nice to have if that's all you need.