Iscsi issue with proxmox

[kumarullal]

Hi,
I am using openfiler for the iscsi source. When I select the iscsi target for Proxmox, it does not give me the capacity etc. NFS works fine, but iscsi does not show any capacity. WHen I create a new KVM machine, and for storage, it selects the entire iscsi target disk. How do I resolve this issue. What I want is similat to waht vmware offers. In vmware, you can select the datastore, if it is local storage or iscsi storage. You can allocate a portion of the iscsi storage to the VM. We cant do the same with proxmox. Is there something that I missed?
Please help.

 

[udo]

Hi,
i don't use iScsi, but my lvm-storage shows in the gui the capacity. I think you must add your iscsi-storage. Build a volumegroup on it (pvcreate and vgcreate). If you now add this volumegroup as lvm-storage you should see the capacity...

Udo

 

[kumarullal]

My concern was that the VM takes the entire VG created on iscsi. Is it possible to allocate a portion of the VG (On Iscsi initiator) to the VM, like it does on VMWARE?

 

[udo]

Hi,
perhaps i miss something, but the volumegroup use the full space of the iscsi-disk - thats ok.
After that, the disk of the VMs use only the size (e.g. 10G) in the volumegroup.

See here:

# pvdisplay /dev/sdf1
  --- Physical volume ---
  PV Name               /dev/sdf1
  VG Name               ams200_fc_1_vg
  PV Size               132,82 GB / not usable 2,75 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              34002
  Free PE               10309
  Allocated PE          23693
  PV UUID               8oMaMP-osKR-9WOE-dVNY-yeHC-2qjm-QfdNwY
   
# vgdisplay ams200_fc_1_vg
  --- Volume group ---
  VG Name               ams200_fc_1_vg
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  683
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                13
  Open LV               4
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               132,82 GB
  PE Size               4,00 MB
  Total PE              34002
  Alloc PE / Size       23693 / 92,55 GB
  Free  PE / Size       10309 / 40,27 GB
  VG UUID               fosNqB-YjAB-8g49-3FXc-nEav-mlUJ-eA91Hc
   
# lvdisplay
  --- Logical volume ---
  LV Name                /dev/ams200_fc_1_vg/vm-102-disk-1
  VG Name                ams200_fc_1_vg
  LV UUID                mM29yT-xfXR-5jRu-zPE2-6D2L-H9oN-EmshxG
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                8,00 GB
  Current LE             2048
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:16
   
  --- Logical volume ---
  LV Name                /dev/ams200_fc_1_vg/vm-102-disk-2
  VG Name                ams200_fc_1_vg
  LV UUID                6qViLW-CI5g-P7P5-TeBI-zle8-veR2-6nP5G5
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                3,00 GB
  Current LE             768
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:17
...

There are much more disks (logical volumes) on the vg to use the 92GB.

Udo

 

[kumarullal]

My output is this.
kvm1:/# pvdisplay /dev/sda2
--- Physical volume ---
PV Name /dev/sda2
VG Name pve
PV Size 148.51 GB / not usable 2.09 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 38018
Free PE 1023
Allocated PE 36995
PV UUID HM0Hr0-m9Yx-TEwm-ukHy-KoqR-dpDy-XVEGRY

This is just showing me the local storage, not the isci storage.

Through the web browser, the VM gets the entire vg of isci storage.
I also read somewhere on the this forum that proxmox allocates the entire vg for isci to a VM, which means that in the isci initiator, I have to create multiple vg groups and assign each vg to a VM in proxmox. That is not a very convenient, especially if you want to share the isci storage with other proxmox servers in a cluster.
I hope there is a resolution to this issue.

 

[udo]

My output is this.
kvm1:/# pvdisplay /dev/sda2
--- Physical volume ---
PV Name /dev/sda2
VG Name pve
PV Size 148.51 GB / not usable 2.09 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 38018
Free PE 1023
Allocated PE 36995
PV UUID HM0Hr0-m9Yx-TEwm-ukHy-KoqR-dpDy-XVEGRY

This is just showing me the local storage, not the isci storage.

Through the web browser, the VM gets the entire vg of isci storage.
I also read somewhere on the this forum that proxmox allocates the entire vg for isci to a VM, which means that in the isci initiator, I have to create multiple vg groups and assign each vg to a VM in proxmox. That is not a very convenient, especially if you want to share the isci storage with other proxmox servers in a cluster.
I hope there is a resolution to this issue.

Hi,
if you see with pvdisplay only one device (and with vgdisplay only pve) i assume that your iScsi-Device isn't a lvm-storage.
Do you make a filesystem on the storage? is it mounted?
If it's not mounted, look which device the iscsi-Disk have - in this example i use /dev/sdd.
is a partition on the device? fdisk -l /dev/sdd
create or change to one big partition type 8e. like this:

Disk /dev/sda: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00069f7e

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1       38913   312565952   8e  Linux

pvcreate /dev/sdd1
vgcreate -s 4M iscsivg /dev/sdd1

go to the gui->storage and add a lvm-storage (use iscsivg as existing lvm). If you can access the iscsi-disk from multible storage-nodes select shared.

That should be all.

Udo

 

[vcp_ai]

Hi,

go to the gui->storage and add a lvm-storage (use iscsivg as existing lvm). If you can access the iscsi-disk from multible storage-nodes select shared.

That should be all.

Udo

That is what I'm using OpenFiler & iscsi Target ..
Just two points:
-On Add iSCSI Target screen UNCHECK 'Use LUNS directly' Option. So iscsi Target appears to Proxmox without Size Info..
-Then Add LVM Group and use above Definition as 'Base Storage' Option, and decide if you want to use it as VirtualDisks, Backups...

But that also takes ALL of the iscsi Target as LVM.
I basically followed http://pve.proxmox.com/wiki/Storage_Model

What I've done is to declare (Openfiler) an iSCSI (with needed size) for every application.

Regards

 

[kumarullal]

Hi Udo,
I am using Openfiler as the storage device. I am using LVM
Here is the output of fdisk -l

[root@OPenf ~]# fdisk -l

Disk /dev/hda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 13 104391 83 Linux
/dev/hda2 14 78 522112+ 82 Linux swap / Solaris
/dev/hda3 79 2610 20338290 83 Linux

Disk /dev/hdb: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hdb1 1 13054 104856223+ 8e Linux LVM

Disk /dev/dm-0: 31.7 GB, 31708938240 bytes
255 heads, 63 sectors/track, 3855 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/dm-0p1 * 1 3854 30957223+ 7 HPFS/NTFS

Disk /dev/dm-1: 41.3 GB, 41372614656 bytes
255 heads, 63 sectors/track, 5029 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

vcp_ai,
You are right about VM taking up the entire isci Target.
So is there a way to assign only a portion of the isci volume to the VM, like we do it in vmware?

 

[udo]

Hi,
yes you take the whole disk for lvm, but then you can assign multible VM-disks to this volume group.
It's like vmware. At vmware you have a large vmfs-filesystem with containerfiles for each VM on it and with proxmox you have a large volumegroup with logical volumes for each VM.

I see no big difference (ok, lvm has no underlaying filesystem).

Udo

 

[kumarullal]

Hi Udo,
The only difference is, in vmware, the logical volumes are autometically created by vmware when you assign a VM with a certain portion of the disk space from the volume group. In proxmox, we are expected to assign the logical volume on the remote datastore for each vm. That does not seem to be very user friendly.

 

[udo]

Hi Udo,
The only difference is, in vmware, the logical volumes are autometically created by vmware when you assign a VM with a certain portion of the disk space from the volume group. In proxmox, we are expected to assign the logical volume on the remote datastore for each vm. That does not seem to be very user friendly.

Hm,
it's on click at proxmox to select the right storage (local,lvm...) - see screenshot.

I don't think that's easier with vmware.
Also is proxmox ve not vmware - if someone need the look and feel (and what else) of vmware he schould use it (and pay for it). proxmox ve is not a copy (and i guess it's don't want to be) of vmware.

Udo

 

[kumarullal]

Hi UDO,
Dont get me wrong. I love promox so much that I want to decommission all the VMs running on vmware and bring them all on to Proxmox. I was just evaluating the ease of operation when creating a KVM image on an isci storage.
Another hitch that I faced (Which I have posted on another thread) is that vzdump takes a long time to finish the job for VMs having large disks (500BG or more, a good example would be exchange server).
I love vzdump as a tool very much, if there is a drastic improvement on the speed to finish jobs, then it is an ideal tool.

 

[dietmar]

Another hitch that I faced (Which I have posted on another thread) is that vzdump takes a long time to finish the job for VMs having large disks (500BG or more, a good example would be exchange server).
I love vzdump as a tool very much, if there is a drastic improvement on the speed to finish jobs, then it is an ideal tool.

You are the one who tests performance on an VMWare VM?

 

[kumarullal]

You are the one who tests performance on an VMWare VM?

Yes,
I realize the mistake of using vmware based storage to test the speed out.

Hi,
yes you take the whole disk for lvm, but then you can assign multible VM-disks to this volume group.
It's like vmware. At vmware you have a large vmfs-filesystem with containerfiles for each VM on it and with proxmox you have a large volumegroup with logical volumes for each VM.

I see no big difference (ok, lvm has no underlaying filesystem).

Udo

You are right UDO. I just tried it today and it worked!!!!!!
Please let me know if what I did was what you meant.
First, I added the ISCSI target in the storage section. At this point, you cannot determine the size of the ISCSI target that you just added.
If you create a KVM image and use the ISCSI target, then the entire volume gets allotted to the VM. This is not what we want, We want a situation where we can split the Volume and use it for multiple KVM images.
So I tried your instructions.
Once I added the ISCSI target in the Storage section of the proxmox. Then I added an LVM storage and selected the same ISCSI target that I had previously added, and proxmox accepted that as a LVM storage.
Now when I went to create the new KVM image, and selected the the LVM storage that I created using the ISCSI target in the Disk Storage parameter, and Bingo, it allowed me to use any amount of storage in the Disk Size parameter.
This is exactly what I was looking for. So Now, I am not restricted by one LV on SAN per KVM (As I earlier thought)
Thanks a lot UDO for your suggestion. Thanks a lot dietmar.

 

[udo]

...
You are right UDO. I just tried it today and it worked!!!!!!
Please let me know if what I did was what you meant.
...

Hi,
fine that's worked now! And yes, thats the normal way.

Udo