iscsi over zfs + ovios linux

[plastilin]

Hello everyone, I'm trying to use ovios linux for my tests as a network storage. There was a connection problem. Generated a key and sent it to san. I am trying to add a disk to the virtual machine and I get the error:

No configuration found. Install iet on 192.168.1.169 at /usr/share/perl5/PVE/Storage/LunCmd/Iet.pm line 113. at /usr/share/perl5/PVE/API2/Qemu.pm line 1442. (500)

although the disk was created on san



Do I need to contact the developer of the distribution kit to include iet in the delivery? Or are some other actions necessary?

 

[Stoiko Ivanov]

I have no experience with ovios linux - and hence cannot provide too much info on that one particularly ...

ZFS over iSCSI in PVE creates/removes/resizes/snapshots/... volumes (VM disk-images) by connecting to the iscsi-provider (ovios in that case) via ssh (with the configured key) and running the appropriate commands for the specified target.

So it depends on which target-implementation ovios linux uses - AFAIK the most common ones on linux are:
* LIO (a bit newer - configurable via `targetcli`)
* IET

You need to find out which one ovios uses - and configure it appropriately...


for completeness sake - the documentation for PVE:
https://pve.proxmox.com/wiki/Storage:_ZFS_over_iSCSI

one example config from a test-system here - where the target is a debian machine with LIO:

zfs: iscsilio
    blocksize 4k
    iscsiprovider LIO
    pool zpool
    portal 192.168.0.2
    target iqn.2003-01.org.linux-iscsi.iscsitgt6.x8775:sn.xxxxxxxxxx
    content images
    lio_tpg tpg1
    nowritecache 1
    sparse 1

I hope this helps!

 

[plastilin]

Ovios use tgtd - The SCSI Target Daemon. Maybe problem because used this daemon? Proxmox can be adapted to work with it?

 

[Stoiko Ivanov]

Proxmox can be adapted to work with it?

Not without adding support to this as was done for LIO and the other iSCSI daemons

It's most likely something we'd accept patches for.

* Maybe you can work with an LVM on top of an exported iSCSI LUN from ovios?
* or use some other target OS?

 

[plastilin]

Thank you. I will wait for the plugin to appear for working with the tgtadm configuration utility, and I hope it will not take long. It is possible to work with LVM over iscsi, but then it is not possible to expand the disk without completely removing the PV from the host. + snapshots do not work. You can give directly created LUNs to virtual machines, but it is inconvenient to administer since there is no understanding of which LUN on the NAS belongs to the VM. Also, snapshots do not work, but you can change the size simply by resizing on the SAN and then umap lun map again from OviOS CLI. I reviewed and tried many different ready-made distributions that work with iscsi and only OviOS Linux came up to me for solving all problems, namely iscsi / nfs / smb / ftp and the developer plans to implement tftp support, which is also important for performing a number of tasks. It is also easy to administer using an internal CLI and uses ZFS which is suitable for deploying VM disks, resizing them, and taking snapshots, and you can also build a failover cluster. I really liked the ESOS product but it is based on mdadm and does not provide support for ZFS without rebuilding. Openmediavault does not support iscsi from the web interface, FreeNAS is based on FreeBSD with an overloaded interface and functions I do not need, I would like Linux. XigmaNAS is also FreeBSD based. Rockstor use btrfs. PetaSAN is cool but requires 3 nodes for its work, since it is based on CEPH. I hope that someday the Proxmox team will have a Proxmox Storage Server product, managed via a web interface, only with the functionality of ceph / iscsi / nfs / smb / ftp / tftp / rsync file services, the ability to be used separately and with integration with Proxmox VE and building a failover cluster ... Then it will be a complete Proxmox ecosystem, our favorite one, with the help of which we can create configurations of any complexity. I really hope that when it will be so.

 

[Stoiko Ivanov]

I will wait for the plugin to appear for working with the tgtadm configuration utility, and I hope it will not take long.

Just to not raise your hopes too much - currently it is not on our road-map - and I don't think it will become a high priority in the foreseeable future.

One alternative would be to use a plain debian system (or even a PVE node) as iSCSI target.

Running Ceph hyperconverged is also quite a common deployment (needs 3 PVE nodes)

 

[plastilin]

Just to not raise your hopes too much - currently it is not on our road-map - and I don't think it will become a high priority in the foreseeable future.

I understand. But in my conditions, I would like to get integration with a ready-made distribution, the question is the limited staff who could serve the bundle + I already fully use NFS and SMB on OviOS. So must wait. Unfortunately, I am not a programmer, so there is little I can do to help. Maximum adapt the existing plugin to the tgtadm commands but I don't know where to start.

 

[ovios]

I started working on an agent for this, hopefully it won't take very long to have it ready to go.

 

[ovios]

I have a working plugin for OviOS Linux and if proxmox wants to add, it let me know where and how to submit it.

 

[plastilin]

I have a working plugin for OviOS Linux and if proxmox wants to add, it let me know where and how to submit it.

View attachment 26326

Great news. We will test.

 

[Stoiko Ivanov]

I have a working plugin for OviOS Linux and if proxmox wants to add, it let me know where and how to submit it.

Check out our Developer Documentation https://pve.proxmox.com/wiki/Developer_Documentation
if you want the code to be included in Proxmox VE we'd need a signed CLA - see https://pve.proxmox.com/wiki/Developer_Documentation#Software_License_and_Copyright
for the steps.

Additionally our perl style guide could be helpful:
https://pve.proxmox.com/wiki/Perl_Style_Guide


Thanks!

 

[RolandK]

So it depends on which target-implementation ovios linux uses - AFAIK the most common ones on linux are:
* LIO (a bit newer - configurable via `targetcli`)
* IET

not to forget scst ( http://scst.sourceforge.net/ ) , which was part of proxmox years ago , btw.