Proxmox 2.0: node unavailable even if server is up and running / iscsi LVM issue

atinazzi

Active Member
Mar 15, 2010
57
1
26
Hi,
I am testing Proxmox 2.0-18 16283a5a. It looks fantastic, but I am experiencing a few quite annoying issues.

My server has 2 nodes connected to 2 NAS / iscsi via 2 dedicated switches (with 802.3ad enabled)

1) Often but now always, the web GUI shows the second node as unavailable. Likewise if I load the web interface of the second node the first node is shown as unavailable. I can access each node via web GUI and ssh.

2) I have 2 iscsi target which I would like to configure as mirrors (RAID 1). Proxmox web GUI does not seems to support this so I configured it via CLI as follows:
  • i have connected the 2 iscsi targets (3Tb each) via web GUI
  • open up a console (CLI)
  • created GPT partition table with parted
  • #pvcreate /dev/sdb1 /dev/sdc1
  • #vgcreate VMs /dev/sdb1 /dev/sdc1
  • #lvcreate -l 50%FREE -m1 -n vmlg01 VMs --corelog
  • #mkfs.ext4 /dev/VMs/vmlg01
  • #vgchange -ay VMs (on both nodes)
vdisplay shows the new logical volume as 2 x mirrored volumes of 2.94T , R/W and status available, however Proxmox web GUI shows it as a 5.89Tb volume completely full (0 available for storage).

I tried to restart both nodes several times but same result.

Please HELP!
 
  • #mkfs.ext4 /dev/VMs/vmlg01

Seem you try to create an ext4 filesystem (if so, please define a directory storage instead).

Note: There is no need to put an FS onto the storage.

I tried to restart both nodes several times but same result.

Hopefully you do not try to mount a ext4 filesystem on 2 nodes?! This will result in data loss!
 
Thanks for replying...

VG PV LV SN Attr Vsize Vfree
VMs 2 1 0 wz--n- 5.89t 0
pve 1 3 0 wz--n- 136.20g 4.00g

Please note that I can mount /dev/VMs/vmlg01 /mnt without any problem and it shows 2.94Tb. I have been able to mount it in proxmox as a directory and write data on it.
 
OK then I am not doing the right thing.

I wasn't sure about the file system... If I had to create one or not.... I have been trying so many different things...

What I would like to do is create a mirrored volume, available to all nodes, using 2 iscsi targets... any suggestion?
 
Last edited:
I am starting over...

so, I should :


  • #pvcreate /dev/sdb1 /dev/sdc1
  • #vgcreate VMs /dev/sdb1 /dev/sdc1
  • #vgchange -ay VMs (on both nodes)
you said not to create the LV ... (#lvcreate -l 50%FREE -m1 -n vmlg01 VMs --corelog) however if I do not create the LV then the 2 targets are not mirrored....
how should I enable mirroring? any suggestion?
 
however if I do not create the LV then the 2 targets are not mirrored....
how should I enable mirroring? any suggestion?

I do not really understand what you want to do with dm mirror over iscsi - duplicate network traffic??

I suggest you setup RAID on you storage server instead (or better, use an reasonable HW raid controller).
 
No, I am not trying to duplicate network traffic. I am trying to create a high-availability cluster with fully redundant hardware.
I have 2 HP DL380-G7 with 2 NC382i Dual Port Multifunction Gigabit adapters. One dual port adapter is dedicated to cluster management and VMs and the other is dedicated to iscsi with the two ports aggregated with 802.3ad.
I have 2 HP V1910 switches and 2 NAS servers. Server1 and NAS1 connect to Switch1 using 802.3ad and Server2 and NAS2 connect to Switch2 using 802.3ad. There is an aggregated uplink between the 2 switches.

I want to do mirror over ISCSI to keep the cluster running in the event of one of the NAS server to become unavailable. It is quite basic design I think... I implemented something similar a few years ago using FC and ESXi.

Now... i did:

  • #pvcreate /dev/sdb1 /dev/sdc1
  • #vgcreate VMs /dev/sdb1 /dev/sdc1
  • #vgchange -ay VMs (on both nodes)
  • #lvcreate -l 50%FREE -m1 -n vmlg01 VMs --corelog)
without creating a file system this time.... but proxmox web GUI does not seems to pick up the LVM groups I created using the CLI... any suggestion?
 
You say that proxmox does not uses LV but to my knowledge swap root and data are LV under the VG pve... could you please elaborate?

I am currently looking to various alternatives such as GNBD or OCFS2 which should be supported by debian or DRBD but as I am not too familiar with these technologies any help will be strongly appreciated.

Also... please check this http://lawrit.lawr.ucdavis.edu/itdo...mirror-to-an-existing-lvm-volume-proxmox/view

I am more than happy to purchase a support ticket if you (Dietmar) tell me that it can be done.
Thanks
 
Last edited:
You say that proxmox does not uses LV but to my knowledge swap root and data are LV under the VG pve... could you please elaborate?

Those are local disks with a filesystem on top.

I am currently looking to various alternatives such as GNBD or OCFS2 which should be supported by debian or DRBD but as I am not too familiar with these technologies any help will be strongly appreciated.

Seems you are looking for some replicated, HA storage device. There are many vendors of such products, and you need to choose the product (I can't do that for you).

I am more than happy to purchase a support ticket if you (Dietmar) tell me that it can be done.

A support ticket is for single PVE related bugs/problems. It does not include consultation and planning of a complete HA enabled storage infrastructure.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!