After having way too many issues with NFS permissions on various storage devices(OmniOS, OpenFiler, FreeNAS), FreeNAS performance is abysmal, OpenFiler seems to have permissions issues causing improper storage calculations, and OmniOS adds ACL issues, which prevent other stuff from happening (hosts and resolv.conf are not created). I'm fairly convinced that we need to move to iSCSI for the storage.
On to the wonderful world of iSCSI...
I've found these articles:
http://pve.proxmox.com/wiki/OpenVZ_on_ISCSI_howto, but this is to move the /var/lib/vz to iSCSI, which is not my intention. Also, does this article need to be updated since the same LUN can be mounted on multiple nodes? Also, not sure how old this article is, as it seems things in Proxmox have changed.
http://pve.proxmox.com/wiki/Storage_Model#LVM_Groups_with_Network_Backing - not too much information here, and seems something is missing between step 1 First and step 1 second, just not sure what...I'm stuck at not having a base volume show up...
http://forum.proxmox.com/threads/9893-Moving-a-Local-VM-to-an-iSCSI-Storage-Device, which seems to be a better fit.
This is what we have so far:
This is where things are getting a bit strange for me. fdisk -l shows dev/sdd and dev/sdc and /dev/mapper/zfs01-01(the name from proxmox UI)
After creating the partition (Lvm type) on the sdc (since it appears sdd is a "clone"). The partition seems to be created correctly on sdc, not on sdd. /dev/mapper/zfs01-01 also updated, so it at least seems to be referencing the right device. After a reboot, sdc and sdd are made the same, as they both display the same partition layout, and have the same Disk identifier. Why is this? Is this because of shared?
The wiki article references changing the startup to automatic, but because we're booting locally, I'm guessing I don't need to change this, as the settings do revert back to manual in the conn[0] section of the node configs.
So at this point, I'm unsure of what to do next. When I go to the GUI to add LVM, it shows the iSCSI storage, but I don't have a base volume to choose.
I can't format the partition sdc1, as it says it's in use by the system. (I can remove from dmadmin to force, but not sure what that will do to proxmox)
Thanks in advance,
Carlos
On to the wonderful world of iSCSI...
I've found these articles:
http://pve.proxmox.com/wiki/OpenVZ_on_ISCSI_howto, but this is to move the /var/lib/vz to iSCSI, which is not my intention. Also, does this article need to be updated since the same LUN can be mounted on multiple nodes? Also, not sure how old this article is, as it seems things in Proxmox have changed.
http://pve.proxmox.com/wiki/Storage_Model#LVM_Groups_with_Network_Backing - not too much information here, and seems something is missing between step 1 First and step 1 second, just not sure what...I'm stuck at not having a base volume show up...
http://forum.proxmox.com/threads/9893-Moving-a-Local-VM-to-an-iSCSI-Storage-Device, which seems to be a better fit.
This is what we have so far:
Code:
root@pve01:~# pvesm iscsiscan --portal 172.30.31.16
iqn.2010-09.org.openindiana:1382567901 172.30.31.16:3260,172.31.31.14:3260,172.30.30.33:3260
Code:
root@pve01:~# pvs PV VG Fmt Attr PSize PFree
/dev/md1 pve lvm2 a-- 465.26g 15.99g
Code:
root@pve01:~# lvs LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
data pve -wi-ao--- 306.27g
root pve -wi-ao--- 96.00g
swap pve -wi-ao--- 47.00g
Code:
root@pve01:~# vgs
VG #PV #LV #SN Attr VSize VFree
pve 1 3 0 wz--n- 465.26g 15.99gThis is where things are getting a bit strange for me. fdisk -l shows dev/sdd and dev/sdc and /dev/mapper/zfs01-01(the name from proxmox UI)
Code:
Disk /dev/sdd: 1073.7 GB, 1073741824000 bytes
255 heads, 63 sectors/track, 130541 cylinders, total 2097152000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3ca0ceb1
Device Boot Start End Blocks Id System
Disk /dev/sdc: 1073.7 GB, 1073741824000 bytes
255 heads, 63 sectors/track, 130541 cylinders, total 2097152000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3ca0ceb1
Device Boot Start End Blocks Id System
Disk /dev/mapper/zfs01-01: 1073.7 GB, 1073741824000 bytes
255 heads, 63 sectors/track, 130541 cylinders, total 2097152000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3ca0ceb1After creating the partition (Lvm type) on the sdc (since it appears sdd is a "clone"). The partition seems to be created correctly on sdc, not on sdd. /dev/mapper/zfs01-01 also updated, so it at least seems to be referencing the right device. After a reboot, sdc and sdd are made the same, as they both display the same partition layout, and have the same Disk identifier. Why is this? Is this because of shared?
The wiki article references changing the startup to automatic, but because we're booting locally, I'm guessing I don't need to change this, as the settings do revert back to manual in the conn[0] section of the node configs.
So at this point, I'm unsure of what to do next. When I go to the GUI to add LVM, it shows the iSCSI storage, but I don't have a base volume to choose.
I can't format the partition sdc1, as it says it's in use by the system. (I can remove from dmadmin to force, but not sure what that will do to proxmox)
Thanks in advance,
Carlos