What open source solutions are available to use "ZFS over iSCSI with Proxmox"?

Discussion in 'Proxmox VE: Installation and configuration' started by al.semenenko88, Mar 26, 2018.

  1. raku

    raku Member

    Joined:
    Apr 16, 2016
    Messages:
    35
    Likes Received:
    3
    You need to create dataset not zvol. Proxmox is responsible for creating virtual disks as zvols inside the base dataset.
     
  2. mihanson

    mihanson New Member

    Joined:
    Nov 1, 2018
    Messages:
    11
    Likes Received:
    0
    Thank you @raku for clarifing that for me. So starting fresh with this, here's what I've learned for anyone who may need to know. I hope it can be helpful:

    1.) Get the FreeNAS patches and apply them per the instructions in the README.md.
    2.) On FreeNAS 11.1, insure that your GUI is listening on ALL IPv4 addresses if you have more than one configured (System > General > Web GUI IPv4 Address (0.0.0.0)).
    3.) Create a *dataset* (not a zvol) in your FreeNAS storage pool to store your VM images.
    4.) Create an iSCSI target on FreeNAS for Proxmox to interface with. I called mine "proxmox": Sharing > Block (iSCSI) > Targets
    5.) Optional: In a seprate terminal window watch /var/log/syslog while performing operations on Proxmox: tail -f /var/log/syslog
    6.) In the Proxmox GUI, go to Datacenter > Storage > Add > ZFS-over-iSCSI
    iSCSI Provider: FreeNAS-API
    ID: <What you want your storage called in Proxmox>
    Portal: <IP Address your FreeNAS listens to for iSCSI connections>
    Pool: <Path to the dataset you created in step 3. i.e. DataDump/proxmox/vm-storage>
    Block size: 4k (Leave it alone unless you know what you are doing)
    Target: <The target name you created in step 4 above pre-fixed with your iqn. i.e. iqn.YYYY-MM.tld.fqdn:<target name>>
    API Use SSL: <your choice>
    API Username: <root or other privledged user on your FreeNAS>
    Nodes: <Restrict to certain nodes if you'd like>
    Enable: YES (checked)
    Thin Provision: <your choice>
    Write Cache: YES (checked) <Unless you know what you are doing>
    API IPv4 Host: (none) <I left mine as none, but your mileage may vary>
    API Password: <Password of the API Username above>
    Confirm password: <Same as API Password>
    7.) If all goes well you should see your FreeNAS storage populate in your list of storage devices on the left side of the Proxmox GUI.
    8.) Create a new VM as normal, selecting your FreeNAS storage to hold the virtual disk(s).
    9.) To move an existing VM: Select your VM > Hardware > Hard Disk(s) > Move Disk(s) > Target Storage > Your FreeNAS storage
    Below is a log of a successful move.

    Code:
    Dec 12 09:21:42 pve pvedaemon[14881]: <root@pam> move disk VM 107: move --disk scsi0 --storage freenas-vm-storage
    Dec 12 09:21:42 pve pvedaemon[14881]: <root@pam> starting task UPID:pve:000073F9:00CD89B0:5C1143A6:qmmove:107:root@pam:
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::lun_command : create_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::freenas_get_first_available_lunid : return 0
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):name : lun not found
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:21:43 pve pvedaemon[29689]: FreeNAS::create_lu(lun_path=/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0, lun_id=0) : blocksize convert 4k = 4096
    Dec 12 09:21:44 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:45 pve pvedaemon[29689]: FreeNAS::API::create_extent(lun_path=/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0, lun_bs=4096) : sucessfull
    Dec 12 09:21:45 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::API::create_target_to_extent(target_id=5, extent_id=5, lun_id=0) : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::create_lu(lun_path=/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0, lun_id=0) : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::lun_command : add_view()
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::lun_command : list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:21:46 pve pvedaemon[29689]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):name : lun found /dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::lun_command : list_view(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:21:47 pve pvedaemon[29689]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):lun-id : lun found 0
    Starting the moved VM:
    Code:
    Dec 12 09:26:50 pve pvedaemon[28192]: start VM 107: UPID:pve:00006E20:00CE01FD:5C1144DA:qmstart:107:root@pam:
    Dec 12 09:26:50 pve pvedaemon[14881]: <root@pam> starting task UPID:pve:00006E20:00CE01FD:5C1144DA:qmstart:107:root@pam:
    Dec 12 09:26:50 pve pvedaemon[28192]: FreeNAS::lun_command : list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:26:50 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):name : lun found /dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::lun_command : list_view(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):lun-id : lun found 0
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::lun_command : list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):name : lun found /dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0
    Dec 12 09:26:51 pve pvedaemon[28192]: FreeNAS::lun_command : list_view(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:26:52 pve pvedaemon[28192]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):lun-id : lun found 0
    Dec 12 09:26:52 pve systemd[1]: Started 107.scope.
    
    Destroying the moved VM:
    Code:
    Dec 12 09:30:32 pve pvedaemon[28851]: <root@pam> starting task UPID:pve:00007FC9:00CE588B:5C1145B8:qmdestroy:107:root@pam:
    Dec 12 09:30:32 pve pvedaemon[32713]: destroy VM 107: UPID:pve:00007FC9:00CE588B:5C1145B8:qmdestroy:107:root@pam:
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::lun_command : list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):name : lun found /dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::lun_command : list_view(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:30:32 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):lun-id : lun found 0
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::lun_command : list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::list_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0):name : lun found /dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::lun_command : delete_lu(/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0)
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_extent : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::freenas_list_lu : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_globalconfig : target_basename=iqn.2017-12.com.lahansons
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target() : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : sucessfull
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::API::get_target_to_extent() : change undef iscsi_lunid to 0
    Dec 12 09:30:33 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:35 pve pvedaemon[32713]: FreeNAS::API::remove_target_to_extent(link_id=5) : sucessfull
    Dec 12 09:30:35 pve pvedaemon[32713]: FreeNAS::api_call : setup : sucessfull
    Dec 12 09:30:36 pve pvedaemon[32713]: FreeNAS::API::remove_extent(extent_id=5) : sucessfull
    Dec 12 09:30:36 pve pvedaemon[32713]: FreeNAS::delete_lu(lun_path=/dev/zvol/DataDump/data/proxmox-vm/vm-107-disk-0) : sucessfull
    Dec 12 09:30:38 pve zed: eid=26 class=history_event pool_guid=0x6B9F797A3937F4B6
    Dec 12 09:30:38 pve pvedaemon[28851]: <root@pam> end task UPID:pve:00007FC9:00CE588B:5C1145B8:qmdestroy:107:root@pam: OK
    
     
  3. Norman Uittenbogaart

    Joined:
    Feb 28, 2012
    Messages:
    139
    Likes Received:
    4
    Do any of you also use FreeNAS with a dual NIC setup?
    Do you Multipath or LACP?
     
  4. Knuuut

    Knuuut Member

    Joined:
    Jun 7, 2018
    Messages:
    87
    Likes Received:
    8
    I'm using rr (mode=0) with nfs and iSCSI.
    It works.

    Cheers Knuuut
     
  5. Norman Uittenbogaart

    Joined:
    Feb 28, 2012
    Messages:
    139
    Likes Received:
    4
  6. Knuuut

    Knuuut Member

    Joined:
    Jun 7, 2018
    Messages:
    87
    Likes Received:
    8
  7. Norman Uittenbogaart

    Joined:
    Feb 28, 2012
    Messages:
    139
    Likes Received:
    4
    I'm trying this patch of GrandWazoo and it works as a charm, in that it can create VM's and migrate etc etc.
    Everything works.

    BUT......
    The performance crawls, Windows speed is unbearable and in Linux I get a write performence of about 15MB/s.
    While on NFS I get about 80MB/s (1 Gbit line)

    My options on the plugin I have Write cache enabled.
    VM's disks are on caching standard.

    What am I doing wrong?

    Some info on the config,

    Proxmox nodes are connected on a single 1Gbit LAN (for now)
    FreeNAS has aggregated dual NIC with LACP
    ZFS block level is 4k
     
  8. mir

    mir Well-Known Member
    Proxmox Subscriber

    Joined:
    Apr 14, 2012
    Messages:
    3,476
    Likes Received:
    95
    Raid level and number of disks in your Freenas?
    Disk type in Freenas, HDD, SSD etc?
     
  9. Norman Uittenbogaart

    Joined:
    Feb 28, 2012
    Messages:
    139
    Likes Received:
    4
    50% storage in use
    RaidZ2
    6x disks HDD
    1x SDD Samsung 860 Pro as ZIL

    Processor: Intel(R) Xeon(R) CPU 5130 @ 2.00GHz (4 cores)
    Memory: 32 GiB
    CPU Utilization around 4% on average



    Something very strange, I had a SMB service with a test share running.
    I stopped the SMB service and suddenly the speed has increased and is now on a more acceptable 50 MB/s
    Even the Windows VM's are more responsive.

    SMB service was not used.
    Very strange.....
     
  10. Norman Uittenbogaart

    Joined:
    Feb 28, 2012
    Messages:
    139
    Likes Received:
    4
    with the ZFS over iSCSI you have to copy over the authorized keys file to your NAS.
    But this would mean the nodes have root access to my NAS machine, which I do not want.
    What is the command that they need to execute?

    I would like to limit the access to only that command. (I would like my snapshots to be safe on both the NAS and the replication destination in case of hackers)
     
    #50 Norman Uittenbogaart, Feb 26, 2019
    Last edited: Feb 26, 2019
  11. Norman Uittenbogaart

    Joined:
    Feb 28, 2012
    Messages:
    139
    Likes Received:
    4
    To answer my own question, add

    command="zfs get -o value -Hp available,used",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa AAAA

    In the authorization_keys file before the key (until the ssh-rsa part offcourse)
     
  12. LnxBil

    LnxBil Well-Known Member

    Joined:
    Feb 21, 2015
    Messages:
    3,514
    Likes Received:
    315
    Yes, if it needs more, you can just create a wrapper script like it is described here:

    https://binblog.info/2008/10/20/openssh-going-flexible-with-forced-commands/
     
  13. ZS-Man

    ZS-Man New Member

    Joined:
    Oct 18, 2013
    Messages:
    23
    Likes Received:
    0
    Hi, I am testing Freenas API plugin. How to use multipath with this plugin?
    Freenas 11.2, two portals asociated to one target.
    Code:
    root@pve2:~# iscsiadm -m discovery -t st -p 10.0.0.130
    10.0.1.130:3260,2 iqn.2005-10.org.freenas.ctl:proxmox
    10.0.0.130:3260,1 iqn.2005-10.org.freenas.ctl:proxmox
    If I simply add ZFS over ISCSI storage, enter one portal IP, storage is created a is working. But only single path. Is there any aditional configuration needed?

    Thanks.
     
  14. Catwoolfii

    Catwoolfii New Member

    Joined:
    Nov 6, 2016
    Messages:
    20
    Likes Received:
    0
  15. Norman Uittenbogaart

    Joined:
    Feb 28, 2012
    Messages:
    139
    Likes Received:
    4
    I have Proxmox working quite well with FreeNAS through this plugin.
    But for some reason I get a terrible performance for a windows VM.

    Netwerk speeds are ok at 1Gbit (tested both ways)
    Netwerk speeds are also ok from inside a VM
    Reading / Writing are also ok, local, from client and within a linux VM
    But they are terrible for the windows VM

    Anyone have a suggestion?
    Loads are very low for FreeNAS, have 32GB RAM
    Have both a ZIL & L2ARC on a SSD
     
  16. mir

    mir Well-Known Member
    Proxmox Subscriber

    Joined:
    Apr 14, 2012
    Messages:
    3,476
    Likes Received:
    95
    Without performance numbers it is impossible to give you any advice.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice