We have been running the TrueNAS storage plugin for Proxmox VE (github.com/truenas/truenas-proxmox-plugin) in production for months: three nodes on PVE 9.2.4, NVMe/TCP against TrueNAS SCALE 25.10, plus iSCSI for testing. Two of the things we hit are not specific to that plugin, so they are worth posting here whatever you use.
1) qemu-img convert to a block storage runs with the default -t unsafe. Moving or migrating a disk onto any block storage other than zfspool runs qemu-img convert with the default cache mode. If a write error lands after the data is already in the page cache, it is dropped: the copy completes, qemu-img exits 0 and the task shows OK, while the destination is missing data. Reproduced on PVE 9.2.4 (pve-qemu-kvm 11.0.2-1) with a loop device behind dm-flakey (1 s up, 60 s down) and a 2 GiB source with no zero regions: with the default cache mode the copy returns rc=0 and 1.9 of 2 GiB differ from the source, with only "lost async page write" in dmesg; the same run with -t none fails loudly with rc=1. Filed with the full recipe as bug 8054 on bugzilla.proxmox.com.
2) A snapshot taken on the array blocks rollback in Proxmox. If something on the TrueNAS side snapshots the zvol (a periodic snapshot task is enough) and that snapshot is newer than your last PVE snapshot, qm rollback refuses with "not most recent snapshot", and there is no way to see or remove the offending snapshot from Proxmox because it is not in the guest config.
On the plugin itself: upstream is moving fast in beta and a few things bit us, so we keep a fork with the fixes applied and prebuilt packages, and send everything back as pull requests. Open upstream right now: a portal that was down at connect time is never retried (#94), iSCSI CHAP against 25.10 (#95), unbounded API retries stalling pvestatd while the array is unreachable (#97), and importing snapshots taken on TrueNAS into the guest config so they appear in the Snapshots tab with rollback and delete, for VMs and containers (#110). Also reported: the install.sh linked from the README is from an older lineage and fails with "broker: scfg missing api_host/api_key" (#107).
If you want to try the fork:
curl -sSL https://github.com/alfonsokuen/truenas-proxmox-plugin/releases/latest/download/install-idk.sh | bash
It downloads the .deb from the release and verifies its sha256 before installing; --dry-run shows what it would do without touching anything. Packages and per-release notes: github.com/alfonsokuen/truenas-proxmox-plugin/releases
One NVMe/TCP note if you use that transport: SCALE 25.10 advertises no MDTS on the target, so a Linux initiator can build very large commands that the target then rejects. Until that is fixed on the target side (patch under review on linux-nvme) the plugin option tn_nvme_max_io_kb 1024 keeps it bounded.
Happy to answer questions here.
1) qemu-img convert to a block storage runs with the default -t unsafe. Moving or migrating a disk onto any block storage other than zfspool runs qemu-img convert with the default cache mode. If a write error lands after the data is already in the page cache, it is dropped: the copy completes, qemu-img exits 0 and the task shows OK, while the destination is missing data. Reproduced on PVE 9.2.4 (pve-qemu-kvm 11.0.2-1) with a loop device behind dm-flakey (1 s up, 60 s down) and a 2 GiB source with no zero regions: with the default cache mode the copy returns rc=0 and 1.9 of 2 GiB differ from the source, with only "lost async page write" in dmesg; the same run with -t none fails loudly with rc=1. Filed with the full recipe as bug 8054 on bugzilla.proxmox.com.
2) A snapshot taken on the array blocks rollback in Proxmox. If something on the TrueNAS side snapshots the zvol (a periodic snapshot task is enough) and that snapshot is newer than your last PVE snapshot, qm rollback refuses with "not most recent snapshot", and there is no way to see or remove the offending snapshot from Proxmox because it is not in the guest config.
On the plugin itself: upstream is moving fast in beta and a few things bit us, so we keep a fork with the fixes applied and prebuilt packages, and send everything back as pull requests. Open upstream right now: a portal that was down at connect time is never retried (#94), iSCSI CHAP against 25.10 (#95), unbounded API retries stalling pvestatd while the array is unreachable (#97), and importing snapshots taken on TrueNAS into the guest config so they appear in the Snapshots tab with rollback and delete, for VMs and containers (#110). Also reported: the install.sh linked from the README is from an older lineage and fails with "broker: scfg missing api_host/api_key" (#107).
If you want to try the fork:
curl -sSL https://github.com/alfonsokuen/truenas-proxmox-plugin/releases/latest/download/install-idk.sh | bash
It downloads the .deb from the release and verifies its sha256 before installing; --dry-run shows what it would do without touching anything. Packages and per-release notes: github.com/alfonsokuen/truenas-proxmox-plugin/releases
One NVMe/TCP note if you use that transport: SCALE 25.10 advertises no MDTS on the target, so a Linux initiator can build very large commands that the target then rejects. Until that is fixed on the target side (patch under review on linux-nvme) the plugin option tn_nvme_max_io_kb 1024 keeps it bounded.
Happy to answer questions here.
