TrueNAS Storage Plugin

warlocksyno said:
@SInisterPisces @boomshankerx

Hmm, weird. Nah, it works fine afaik. I've been using TPM+EFI+Q35 based VMs on it just fine.
Click to expand...
Having looked at your code a bit I think your plugin manages iSCSI at a lower level than the native ZFS over iSCSI implementation. Proxmox relies on iscsidirect to make the connections.

iscsidirect in QEMU refers to using the built-in iSCSI initiator (via libiscsi) to allow a virtual machine to connect directly to an iSCSI Logical Unit Number (LUN). This bypasses the host operating system's storage stack, meaning the host does not mount the LUN; instead, QEMU manages the connection directly when the VM starts.
Click to expand...
 
Last edited:
boomshankerx said:
Having looked at your code a bit I think your plugin manages iSCSI at a lower level than the native ZFS over iSCSI implementation. Proxmox relies on iscsidirect to make the connections.
Click to expand...
Proxmox has added support for TPM storage for ZFS over iSCSI in qemu-server: 9.1.10
https://bugzilla.proxmox.com/show_bug.cgi?id=3662

Confirmed this fix is now in the no-subscription repo. Confirmed working on my test systems.
 
  • Like
Reactions: SInisterPisces
I am trying to configure nvme/tcp for my truenas scale installation, and the plugin, as well as the manual storage configuration seems to be failing with a 101 error. I think this may be because I am using non-standard ports for my truenas installation (83/http and 447/https). From what I can tell there is not a way to enter a port using the script (validation of the IP fails), nor manually in storage.cfg (entering IP:Port breaks the proxmox UI).

Is there a way to make the plugin work with non-standard ports, or perhaps make an enhancement request?
 
So I finally moved the TPM disk from local to zfs-over-iscsi storage on TrueNAS (vm-300-disk-2) for my Win VM. Happy this is working now.
Unfortunately I can not back up the VM anymore. Output is the following:
Code: 
INFO: starting new backup job: vzdump 300 --notification-mode notification-system --mode stop --node PVE --remove 0 --storage PBS --notes-template '{{guestname}}'
INFO: Starting Backup of VM 300 (qemu)
INFO: Backup started at 2026-06-11 10:52:24
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: MyVM
INFO: include disk 'scsi0' 'NVME-VM:vm-300-disk-1' 128G
INFO: include disk 'efidisk0' 'NVME-VM:vm-300-disk-0' 1M
INFO: include disk 'tpmstate0' 'NVME-VM:vm-300-disk-2' 4M
TrueNAS [INFO] : Connected
TrueNAS [INFO] : Authenticated
TrueNAS [INFO] : Version: TrueNAS-25.10.3.1
INFO: creating Proxmox Backup Server archive 'vm/300/2026-06-11T08:52:24Z'
INFO: starting kvm to execute backup task
swtpm_setup: Not overwriting existing state file.
kvm: iSCSI GET_LBA_STATUS failed at lba 0: SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_FIELD_IN_CDB(0x2400)
INFO: attaching TPM drive to QEMU for backup
TrueNAS [WARN] : Remote closed connection
TrueNAS [INFO] : Disconnected
TrueNAS [WARN] : No response received
TrueNAS [WARN] : Extent not found for path: zvol/NVME/VM/vm-300-disk-2
Use of uninitialized value $lun in int at /usr/share/perl5/PVE/Storage/Custom/TrueNASPlugin.pm line 300.
INFO: starting backup via QMP command
INFO: started backup task 'b1672e24-3b63-410c-81cc-458c4d5270bf'
INFO: efidisk0: dirty-bitmap status: created new
INFO: scsi0: dirty-bitmap status: created new
INFO: tpmstate0-backup: dirty-bitmap status: created new
INFO:   0% (408.0 MiB of 10.1 TiB) in 3s, read: 136.0 MiB/s, write: 130.7 MiB/s
INFO:   1% (103.8 GiB of 10.1 TiB) in 14m 17s, read: 123.9 MiB/s, write: 123.2 MiB/s
INFO:   2% (207.4 GiB of 10.1 TiB) in 29m 27s, read: 116.6 MiB/s, write: 116.0 MiB/s
The extent is defenitely available in TrueNAS. The VM is provisioned with 128G but is trying to back up the complete storage of 10T. I stopped after 30min. This did not happen before I did not move the tpm disk to the TrueNAS storage.

removing the tpm disk will make the back up work properly:
Code: 
INFO: starting new backup job: vzdump 300 --mode stop --notification-mode notification-system --node PVE --notes-template '{{guestname}}' --storage PBS --remove 0
INFO: Starting Backup of VM 300 (qemu)
INFO: Backup started at 2026-06-11 11:32:10
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: MyVM
INFO: include disk 'scsi0' 'NVME-VM:vm-300-disk-1' 128G
INFO: include disk 'efidisk0' 'NVME-VM:vm-300-disk-0' 1M
TrueNAS [INFO] : Disconnected
TrueNAS [WARN] : Connection timed out. Reconnecting...
TrueNAS [INFO] : Connected
TrueNAS [INFO] : Authenticated
INFO: creating Proxmox Backup Server archive 'vm/300/2026-06-11T09:32:10Z'
INFO: starting kvm to execute backup task
kvm: iSCSI GET_LBA_STATUS failed at lba 0: SENSE KEY:ILLEGAL_REQUEST(5) ASCQ:INVALID_FIELD_IN_CDB(0x2400)
INFO: starting backup via QMP command
INFO: started backup task 'a86636b3-ba75-4fdf-b394-cf2418e704e5'
INFO: efidisk0: dirty-bitmap status: created new
INFO: scsi0: dirty-bitmap status: created new
INFO:   1% (1.6 GiB of 128.0 GiB) in 3s, read: 561.3 MiB/s, write: 410.7 MiB/s
INFO:   2% (3.0 GiB of 128.0 GiB) in 6s, read: 474.7 MiB/s, write: 406.7 MiB/s
INFO:   3% (4.0 GiB of 128.0 GiB) in 9s, read: 317.3 MiB/s, write: 266.7 MiB/s
INFO:   4% (5.3 GiB of 128.0 GiB) in 12s, read: 457.3 MiB/s, write: 421.3 MiB/s
INFO:   5% (6.8 GiB of 128.0 GiB) in 16s, read: 374.0 MiB/s, write: 356.0 MiB/s
INFO:   6% (7.9 GiB of 128.0 GiB) in 19s, read: 385.3 MiB/s, write: 274.7 MiB/s
INFO:   7% (9.2 GiB of 128.0 GiB) in 22s, read: 446.7 MiB/s, write: 326.7 MiB/s
INFO:   8% (10.4 GiB of 128.0 GiB) in 25s, read: 393.3 MiB/s, write: 366.7 MiB/s
INFO:   9% (12.0 GiB of 128.0 GiB) in 29s, read: 411.0 MiB/s, write: 395.0 MiB/s
INFO:  10% (13.4 GiB of 128.0 GiB) in 32s, read: 504.0 MiB/s, write: 477.3 MiB/s
INFO:  11% (14.6 GiB of 128.0 GiB) in 35s, read: 382.7 MiB/s, write: 366.7 MiB/s
INFO:  12% (15.5 GiB of 128.0 GiB) in 39s, read: 246.0 MiB/s, write: 231.0 MiB/s
INFO:  13% (17.0 GiB of 128.0 GiB) in 42s, read: 510.7 MiB/s, write: 456.0 MiB/s
INFO:  14% (18.3 GiB of 128.0 GiB) in 45s, read: 454.7 MiB/s, write: 273.3 MiB/s
INFO:  17% (22.7 GiB of 128.0 GiB) in 48s, read: 1.5 GiB/s, write: 13.3 MiB/s
INFO:  21% (27.2 GiB of 128.0 GiB) in 51s, read: 1.5 GiB/s, write: 22.7 MiB/s
INFO:  22% (28.7 GiB of 128.0 GiB) in 54s, read: 509.3 MiB/s, write: 406.7 MiB/s
INFO:  23% (29.5 GiB of 128.0 GiB) in 57s, read: 268.0 MiB/s, write: 253.3 MiB/s
INFO:  24% (30.9 GiB of 128.0 GiB) in 1m, read: 485.3 MiB/s, write: 438.7 MiB/s
INFO:  25% (32.4 GiB of 128.0 GiB) in 1m 3s, read: 516.0 MiB/s, write: 414.7 MiB/s
INFO:  26% (33.8 GiB of 128.0 GiB) in 1m 6s, read: 486.7 MiB/s, write: 401.3 MiB/s
INFO:  27% (35.2 GiB of 128.0 GiB) in 1m 9s, read: 469.3 MiB/s, write: 426.7 MiB/s
INFO:  28% (36.4 GiB of 128.0 GiB) in 1m 12s, read: 393.3 MiB/s, write: 329.3 MiB/s
INFO:  29% (37.5 GiB of 128.0 GiB) in 1m 15s, read: 392.0 MiB/s, write: 372.0 MiB/s
INFO:  30% (38.8 GiB of 128.0 GiB) in 1m 18s, read: 442.7 MiB/s, write: 381.3 MiB/s
INFO:  32% (41.7 GiB of 128.0 GiB) in 1m 21s, read: 988.0 MiB/s, write: 304.0 MiB/s
INFO:  35% (45.3 GiB of 128.0 GiB) in 1m 24s, read: 1.2 GiB/s, write: 202.7 MiB/s
INFO:  37% (47.6 GiB of 128.0 GiB) in 1m 27s, read: 760.0 MiB/s, write: 321.3 MiB/s
INFO:  38% (49.4 GiB of 128.0 GiB) in 1m 30s, read: 644.0 MiB/s, write: 370.7 MiB/s
INFO:  39% (50.8 GiB of 128.0 GiB) in 1m 33s, read: 472.0 MiB/s, write: 389.3 MiB/s
INFO:  40% (52.2 GiB of 128.0 GiB) in 1m 36s, read: 478.7 MiB/s, write: 377.3 MiB/s
INFO:  41% (53.6 GiB of 128.0 GiB) in 1m 39s, read: 453.3 MiB/s, write: 402.7 MiB/s
INFO:  43% (55.5 GiB of 128.0 GiB) in 1m 42s, read: 654.7 MiB/s, write: 261.3 MiB/s
INFO:  45% (57.8 GiB of 128.0 GiB) in 1m 45s, read: 801.3 MiB/s, write: 326.7 MiB/s
INFO:  46% (59.2 GiB of 128.0 GiB) in 1m 48s, read: 464.0 MiB/s, write: 314.7 MiB/s
INFO:  48% (62.7 GiB of 128.0 GiB) in 1m 51s, read: 1.2 GiB/s, write: 238.7 MiB/s
INFO:  50% (65.1 GiB of 128.0 GiB) in 1m 54s, read: 812.0 MiB/s, write: 313.3 MiB/s
INFO:  51% (66.4 GiB of 128.0 GiB) in 1m 57s, read: 433.3 MiB/s, write: 409.3 MiB/s
INFO:  52% (67.7 GiB of 128.0 GiB) in 2m, read: 464.0 MiB/s, write: 464.0 MiB/s
INFO:  53% (69.1 GiB of 128.0 GiB) in 2m 3s, read: 465.3 MiB/s, write: 465.3 MiB/s
INFO:  55% (70.4 GiB of 128.0 GiB) in 2m 6s, read: 452.0 MiB/s, write: 452.0 MiB/s
INFO:  56% (71.7 GiB of 128.0 GiB) in 2m 9s, read: 454.7 MiB/s, write: 454.7 MiB/s
INFO:  59% (76.3 GiB of 128.0 GiB) in 2m 12s, read: 1.5 GiB/s, write: 140.0 MiB/s
INFO:  64% (82.5 GiB of 128.0 GiB) in 2m 15s, read: 2.1 GiB/s, write: 0 B/s
INFO:  69% (88.6 GiB of 128.0 GiB) in 2m 18s, read: 2.0 GiB/s, write: 0 B/s
INFO:  74% (94.8 GiB of 128.0 GiB) in 2m 21s, read: 2.0 GiB/s, write: 0 B/s
INFO:  78% (100.8 GiB of 128.0 GiB) in 2m 24s, read: 2.0 GiB/s, write: 0 B/s
INFO:  83% (106.9 GiB of 128.0 GiB) in 2m 27s, read: 2.0 GiB/s, write: 0 B/s
INFO:  88% (113.0 GiB of 128.0 GiB) in 2m 30s, read: 2.0 GiB/s, write: 0 B/s
INFO:  92% (119.0 GiB of 128.0 GiB) in 2m 33s, read: 2.0 GiB/s, write: 0 B/s
INFO:  97% (125.0 GiB of 128.0 GiB) in 2m 36s, read: 2.0 GiB/s, write: 0 B/s
INFO: 100% (128.0 GiB of 128.0 GiB) in 2m 38s, read: 1.5 GiB/s, write: 2.3 MiB/s
INFO: backup is sparse: 72.26 GiB (56%) total zero data
INFO: backup was done incrementally, reused 83.98 GiB (65%)
INFO: transferred 128.00 GiB in 158 seconds (829.6 MiB/s)
INFO: stopping kvm after backup task
INFO: adding notes to backup
INFO: Finished Backup of VM 300 (00:02:42)
INFO: Backup finished at 2026-06-11 11:34:52
INFO: Backup job finished successfully
TASK OK
 
Last edited:
You must log in or register to reply here.
Top Bottom
Back