ISCSI Storage with LVM Partition

[tirpitz]

hi

we have implemented ISCSI storage with 2 proxmox nodes and using LVM Groups to storage the VM disks. All seems to be okay, but we had 3 incidents where the individual VM disks on LVM partition disappeared and when we rebooted the VM, we got the error

"TASK ERROR: can't activate LV '/dev/images/vm-111-disk-1': One or more specified logical volume(s) not found."

we have checked the possibility of duplication of the VM id, but that is also not the case. I would like to know if

1. LVM group over the ISCSI is a stable solution.
2. What is the solution if such an event reoccurs.
3. has anybody faced similar situation and if there is a way to avoid such incidents.
4. After the disapperance of the VM, when we run lvdisplay, we don't see LVM partition in the list. can we run qm rescan to get the lvm partition back in the system ?

is there a better storage model to work with when using ISCSI storage.

More info. I can see the block device in /dev/

and when I did fdisk , I see that there are 2 partitions but the partition type have changed from Linux LVM and Swap to Windows partition. Has any one experienced this kind of issue ?

I have scanned through syslog and messages but did not get any sort of issue with storage connectivity which might have been the cause of this issue. Since we plan to move all our VMs to central storage and due to this unpredictable behavior, it is now a scary situation if we are doing correct of moving to central storage.

tirpitz

 

[cesarpk]

I am not sure that this idea will be good for you, but if i have only 2 PVE nodes, i prefered have DRBD in synchronous mode in these PVE nodes and not use iSCSI.

Note: DRBD configured in synchronous mode is equal to have mirror of partitions of disks between the PVE Nodes, because each block of disk is replicated synchronously between the nodes

With DRBD in the PVE Nodes, the readings of block of disks are only local, and only the writes are applied for the same PVE node as also for the other PVE node, so the network communication for DRBD is used only for the writes of block of disks, and this type of configuration will use much less bandwith for transmit the data compared with iSCSI, that way, you'll get much more speed in your VMs.

Also of this advantage, also you can have "HA" and "Live migration of VMs". So with less Hardware, you will can enjoy of better performance, and without sacrificing the features of enterprise class

Other tip: As it is highly recommended to have a network independent for DRBD (that is the same case for iSCSI), the difference is that if you disconnects manually the network independent for DRBD, the VMs of the PVE nodes will continue to run normally as if there were no problems, but DRBD will stop the replication, and then when you connect the network communication for DRBD and manually run (by CLI) some commands, the DRBD partitions will be synchronized. And best of all this is that for do it, you don't need power off nothing.

I have this setup in a production enviroment since very years ago, and always get great satisfactions

In the wiki of PVE you will can see how configure it
On the website of linbit you will can see all about of DRBD

If you want use DRBD, my big tip is that you configure two DRBD partitions, the first DRBD partition for the VMs that are running on the first PVE node, and the second DRBD partition for the VMs that are running on the second PVE node, this tip is because if you will have a problem with the replication of DRBD, of this manner, will be very easy correct some problem (if exist) and in very short time.